2024-05-09T10:40:31.841 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-09T10:40:31.841 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_neesingh@teuthology --archive /home/teuthworker/archive/neesingh-2024-05-09_10:23:18-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7699683 --verbose --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}} --name neesingh-2024-05-09_10:23:18-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi -- /home/teuthworker/archive/neesingh-2024-05-09_10:23:18-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7699683/orig.config.yaml 2024-05-09T10:40:31.885 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T10:40:31.970 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2024-05-09_10:23:18-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7699683 branch: wip-neesingh-testing1-20240507 description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}} email: neesingh@redhat.com first_in_suite: false job_id: '7699683' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2024-05-09_10:23:18-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-neesingh-testing1-20240507 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20 kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-58090 sha1: 1836a1f5ed9f69ee7eef7aa7d49d42f0600dc280 owner: scheduled_neesingh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6829 sha1: 4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20 sleep_before_teardown: 0 suite: fs:volumes suite_branch: wip-58090 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_1836a1f5ed9f69ee7eef7aa7d49d42f0600dc280/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: 1836a1f5ed9f69ee7eef7aa7d49d42f0600dc280 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi098.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBU5z6SJ/twu2gk90TtXpSa7p3EIXZVNKxZdCLlaeuFaXfwaRIP6RqKEOLouwyyyQMHNzmoebWboRk+iJDMUGeg= smithi105.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLWdhHoI+mO2K+gFcG8zkd/kk0Xk2KNsMlsL2SxeUB5GE0PwLlokVNRdvL+7C2zbG4Ata0XcRL5tgNl9Mb6iqvY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - check-counter: counters: mgr: - min: 4 name: finisher-volumes.complete_latency.avgcount - expected_val: 0 name: finisher-volumes.queue_len - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestPerModuleFinsherThread teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-09_10:23:18 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-09T10:40:31.971 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_1836a1f5ed9f69ee7eef7aa7d49d42f0600dc280/qa; will attempt to use it 2024-05-09T10:40:31.971 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_1836a1f5ed9f69ee7eef7aa7d49d42f0600dc280/qa/tasks 2024-05-09T10:40:31.971 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-09T10:40:31.973 INFO:teuthology.task.internal:Checking packages... 2024-05-09T10:40:31.996 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20' 2024-05-09T10:40:31.996 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T10:40:31.997 INFO:teuthology.packaging:ref: None 2024-05-09T10:40:31.997 INFO:teuthology.packaging:tag: None 2024-05-09T10:40:31.997 INFO:teuthology.packaging:branch: wip-neesingh-testing1-20240507 2024-05-09T10:40:31.997 INFO:teuthology.packaging:sha1: 4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20 2024-05-09T10:40:31.997 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-neesingh-testing1-20240507 2024-05-09T10:40:32.139 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1629.g4f7075d1 2024-05-09T10:40:32.141 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-09T10:40:32.147 INFO:teuthology.task.internal:no buildpackages task found 2024-05-09T10:40:32.148 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-09T10:40:32.204 INFO:teuthology.task.internal:Saving configuration 2024-05-09T10:40:32.220 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-09T10:40:32.226 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T10:40:32.252 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi098.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-05-09_10:23:18-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7699683', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 10:35:47.346923', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBU5z6SJ/twu2gk90TtXpSa7p3EIXZVNKxZdCLlaeuFaXfwaRIP6RqKEOLouwyyyQMHNzmoebWboRk+iJDMUGeg='} 2024-05-09T10:40:32.275 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-05-09_10:23:18-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7699683', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 10:35:47.345188', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLWdhHoI+mO2K+gFcG8zkd/kk0Xk2KNsMlsL2SxeUB5GE0PwLlokVNRdvL+7C2zbG4Ata0XcRL5tgNl9Mb6iqvY='} 2024-05-09T10:40:32.275 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-09T10:40:32.281 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-09T10:40:32.281 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-09T10:40:32.281 INFO:teuthology.run_tasks:Running task console_log... 2024-05-09T10:40:32.409 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7faca86d2160>, signals=[15]) 2024-05-09T10:40:32.410 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-09T10:40:32.417 INFO:teuthology.task.internal:Opening connections... 2024-05-09T10:40:32.417 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi098.front.sepia.ceph.com 2024-05-09T10:40:32.418 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T10:40:32.493 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi105.front.sepia.ceph.com 2024-05-09T10:40:32.494 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T10:40:32.567 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-09T10:40:32.573 DEBUG:teuthology.orchestra.run.smithi098:> uname -m 2024-05-09T10:40:32.591 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64 2024-05-09T10:40:32.591 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2024-05-09T10:40:32.649 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Stream" 2024-05-09T10:40:32.649 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="9" 2024-05-09T10:40:32.649 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos" 2024-05-09T10:40:32.649 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora" 2024-05-09T10:40:32.650 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="9" 2024-05-09T10:40:32.650 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el9" 2024-05-09T10:40:32.650 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T10:40:32.650 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2024-05-09T10:40:32.650 INFO:teuthology.orchestra.run.smithi098.stdout:LOGO="fedora-logo-icon" 2024-05-09T10:40:32.650 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T10:40:32.650 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://centos.org/" 2024-05-09T10:40:32.650 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T10:40:32.650 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T10:40:32.650 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T10:40:32.651 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2024-05-09T10:40:32.676 DEBUG:teuthology.orchestra.run.smithi105:> uname -m 2024-05-09T10:40:32.694 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2024-05-09T10:40:32.694 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2024-05-09T10:40:32.752 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="CentOS Stream" 2024-05-09T10:40:32.752 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="9" 2024-05-09T10:40:32.752 INFO:teuthology.orchestra.run.smithi105.stdout:ID="centos" 2024-05-09T10:40:32.752 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="rhel fedora" 2024-05-09T10:40:32.752 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="9" 2024-05-09T10:40:32.752 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el9" 2024-05-09T10:40:32.752 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T10:40:32.752 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2024-05-09T10:40:32.752 INFO:teuthology.orchestra.run.smithi105.stdout:LOGO="fedora-logo-icon" 2024-05-09T10:40:32.752 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T10:40:32.752 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://centos.org/" 2024-05-09T10:40:32.752 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T10:40:32.753 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T10:40:32.753 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T10:40:32.753 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2024-05-09T10:40:32.776 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-09T10:40:32.787 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-09T10:40:32.805 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-09T10:40:32.805 DEBUG:teuthology.orchestra.run.smithi098:> test '!' -e /home/ubuntu/cephtest 2024-05-09T10:40:32.807 DEBUG:teuthology.orchestra.run.smithi105:> test '!' -e /home/ubuntu/cephtest 2024-05-09T10:40:32.823 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-09T10:40:32.839 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-09T10:40:32.839 DEBUG:teuthology.orchestra.run.smithi098:> test -z $(ls -A /var/lib/ceph) 2024-05-09T10:40:32.864 DEBUG:teuthology.orchestra.run.smithi105:> test -z $(ls -A /var/lib/ceph) 2024-05-09T10:40:32.975 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-09T10:40:33.032 INFO:teuthology.run_tasks:Running task kernel... 2024-05-09T10:40:33.046 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-05-09T10:40:33.046 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-05-09T10:40:33.046 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-05-09T10:40:33.047 DEBUG:teuthology.orchestra.run.smithi098:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T10:40:33.047 DEBUG:teuthology.orchestra.run.smithi105:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T10:40:33.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T10:40:33.063 DEBUG:teuthology.orchestra.run.smithi098:> uname -r 2024-05-09T10:40:33.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T10:40:33.064 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2024-05-09T10:40:33.121 INFO:teuthology.orchestra.run.smithi098.stdout:5.14.0-437.el9.x86_64 2024-05-09T10:40:33.122 INFO:teuthology.task.kernel:Running kernel on smithi098: 5.14.0-437.el9.x86_64 2024-05-09T10:40:33.122 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum install -y kernel 2024-05-09T10:40:33.123 INFO:teuthology.orchestra.run.smithi105.stdout:5.14.0-437.el9.x86_64 2024-05-09T10:40:33.123 INFO:teuthology.task.kernel:Running kernel on smithi105: 5.14.0-437.el9.x86_64 2024-05-09T10:40:33.123 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum install -y kernel 2024-05-09T10:40:36.456 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.1 MB 00:01 2024-05-09T10:40:36.617 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.1 MB 00:01 2024-05-09T10:40:38.610 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-05-09T10:40:39.031 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-05-09T10:40:42.686 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-05-09T10:40:43.157 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-05-09T10:40:44.583 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 16 kB 00:00 2024-05-09T10:40:44.728 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-05-09T10:40:45.655 INFO:teuthology.orchestra.run.smithi098.stdout:Extra Packages for Enterprise Linux 27 MB/s | 21 MB 00:00 2024-05-09T10:40:46.861 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 9.9 MB/s | 21 MB 00:02 2024-05-09T10:40:50.813 INFO:teuthology.orchestra.run.smithi098.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-09T10:40:51.984 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-09T10:40:52.852 INFO:teuthology.orchestra.run.smithi098.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T10:40:52.853 INFO:teuthology.orchestra.run.smithi098.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T10:40:52.949 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-05-09T10:40:52.960 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-05-09T10:40:52.960 INFO:teuthology.orchestra.run.smithi098.stdout: Package Architecture Version Repository Size 2024-05-09T10:40:52.960 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-05-09T10:40:52.961 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-05-09T10:40:52.961 INFO:teuthology.orchestra.run.smithi098.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T10:40:52.961 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2024-05-09T10:40:52.961 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T10:40:52.961 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T10:40:52.961 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T10:40:52.961 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:40:52.961 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-05-09T10:40:52.961 INFO:teuthology.orchestra.run.smithi098.stdout:================================================================================ 2024-05-09T10:40:52.961 INFO:teuthology.orchestra.run.smithi098.stdout:Install 4 Packages 2024-05-09T10:40:52.961 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:40:52.962 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 78 M 2024-05-09T10:40:52.962 INFO:teuthology.orchestra.run.smithi098.stdout:Installed size: 123 M 2024-05-09T10:40:52.962 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-05-09T10:40:54.007 INFO:teuthology.orchestra.run.smithi105.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T10:40:54.007 INFO:teuthology.orchestra.run.smithi105.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T10:40:54.092 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-05-09T10:40:54.103 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-05-09T10:40:54.103 INFO:teuthology.orchestra.run.smithi105.stdout: Package Architecture Version Repository Size 2024-05-09T10:40:54.103 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-05-09T10:40:54.103 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2024-05-09T10:40:54.103 INFO:teuthology.orchestra.run.smithi105.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T10:40:54.103 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2024-05-09T10:40:54.103 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T10:40:54.103 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T10:40:54.103 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T10:40:54.103 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:40:54.103 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2024-05-09T10:40:54.104 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-05-09T10:40:54.104 INFO:teuthology.orchestra.run.smithi105.stdout:Install 4 Packages 2024-05-09T10:40:54.104 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:40:54.104 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 78 M 2024-05-09T10:40:54.104 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 123 M 2024-05-09T10:40:54.104 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2024-05-09T10:40:54.344 INFO:teuthology.orchestra.run.smithi105.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 2.8 MB/s | 484 kB 00:00 2024-05-09T10:40:54.464 INFO:teuthology.orchestra.run.smithi098.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 353 kB/s | 484 kB 00:01 2024-05-09T10:40:55.053 INFO:teuthology.orchestra.run.smithi105.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 18 MB/s | 15 MB 00:00 2024-05-09T10:40:55.703 INFO:teuthology.orchestra.run.smithi105.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 21 MB/s | 28 MB 00:01 2024-05-09T10:40:56.137 INFO:teuthology.orchestra.run.smithi105.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 17 MB/s | 34 MB 00:01 2024-05-09T10:40:56.137 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-05-09T10:40:56.137 INFO:teuthology.orchestra.run.smithi105.stdout:Total 38 MB/s | 78 MB 00:02 2024-05-09T10:40:56.814 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2024-05-09T10:40:57.150 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2024-05-09T10:40:57.150 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2024-05-09T10:40:58.082 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2024-05-09T10:40:58.083 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2024-05-09T10:40:58.256 INFO:teuthology.orchestra.run.smithi098.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 3.0 MB/s | 15 MB 00:05 2024-05-09T10:40:59.823 INFO:teuthology.orchestra.run.smithi098.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 5.0 MB/s | 34 MB 00:06 2024-05-09T10:41:00.474 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2024-05-09T10:41:01.112 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T10:41:01.195 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T10:41:01.623 INFO:teuthology.orchestra.run.smithi098.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 3.9 MB/s | 28 MB 00:07 2024-05-09T10:41:01.623 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-05-09T10:41:01.623 INFO:teuthology.orchestra.run.smithi098.stdout:Total 9.0 MB/s | 78 MB 00:08 2024-05-09T10:41:02.305 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-05-09T10:41:02.647 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-05-09T10:41:02.647 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-05-09T10:41:03.588 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-05-09T10:41:03.589 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-05-09T10:41:03.875 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T10:41:04.005 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T10:41:05.997 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-05-09T10:41:06.776 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T10:41:06.882 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T10:41:08.712 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T10:41:08.825 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T10:41:09.578 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T10:41:09.706 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T10:41:13.550 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T10:41:14.376 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T10:41:14.425 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T10:41:19.195 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T10:41:43.007 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T10:41:43.045 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T10:41:43.351 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T10:41:43.351 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T10:41:43.351 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T10:41:43.351 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T10:41:44.155 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T10:41:44.156 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:41:44.156 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2024-05-09T10:41:44.156 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T10:41:44.156 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T10:41:44.156 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T10:41:44.156 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T10:41:44.156 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:41:44.156 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2024-05-09T10:41:44.493 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T10:41:44.575 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T10:41:44.576 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T10:41:44.576 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T10:41:44.576 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T10:41:44.576 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T10:41:44.576 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-05-09T10:41:44.576 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi105.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T10:41:44.576 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T10:41:44.626 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T10:41:44.626 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T10:41:44.626 DEBUG:teuthology.orchestra.run.smithi105:> sudo rpm -qi grub2-tools 2024-05-09T10:41:44.656 INFO:teuthology.orchestra.run.smithi105.stdout:Name : grub2-tools 2024-05-09T10:41:44.656 INFO:teuthology.orchestra.run.smithi105.stdout:Epoch : 1 2024-05-09T10:41:44.656 INFO:teuthology.orchestra.run.smithi105.stdout:Version : 2.06 2024-05-09T10:41:44.656 INFO:teuthology.orchestra.run.smithi105.stdout:Release : 68.el9 2024-05-09T10:41:44.657 INFO:teuthology.orchestra.run.smithi105.stdout:Architecture: x86_64 2024-05-09T10:41:44.657 INFO:teuthology.orchestra.run.smithi105.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T10:41:44.657 INFO:teuthology.orchestra.run.smithi105.stdout:Group : Unspecified 2024-05-09T10:41:44.657 INFO:teuthology.orchestra.run.smithi105.stdout:Size : 8264254 2024-05-09T10:41:44.657 INFO:teuthology.orchestra.run.smithi105.stdout:License : GPLv3+ 2024-05-09T10:41:44.657 INFO:teuthology.orchestra.run.smithi105.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T10:41:44.657 INFO:teuthology.orchestra.run.smithi105.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T10:41:44.657 INFO:teuthology.orchestra.run.smithi105.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T10:41:44.657 INFO:teuthology.orchestra.run.smithi105.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T10:41:44.657 INFO:teuthology.orchestra.run.smithi105.stdout:Packager : builder@centos.org 2024-05-09T10:41:44.657 INFO:teuthology.orchestra.run.smithi105.stdout:Vendor : CentOS 2024-05-09T10:41:44.657 INFO:teuthology.orchestra.run.smithi105.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T10:41:44.657 INFO:teuthology.orchestra.run.smithi105.stdout:Summary : Support tools for GRUB. 2024-05-09T10:41:44.657 INFO:teuthology.orchestra.run.smithi105.stdout:Description : 2024-05-09T10:41:44.657 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:41:44.658 INFO:teuthology.orchestra.run.smithi105.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T10:41:44.658 INFO:teuthology.orchestra.run.smithi105.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T10:41:44.658 INFO:teuthology.orchestra.run.smithi105.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T10:41:44.658 INFO:teuthology.orchestra.run.smithi105.stdout:hardware devices. 2024-05-09T10:41:44.658 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:41:44.658 INFO:teuthology.orchestra.run.smithi105.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T10:41:44.659 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T10:41:44.659 INFO:teuthology.task.kernel:Updating grub on smithi105 to boot 5.14.0-444.el9.x86_64 2024-05-09T10:41:44.659 DEBUG:teuthology.orchestra.run.smithi105:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T10:41:45.325 INFO:teuthology.orchestra.run.smithi105.stderr:Generating grub configuration file ... 2024-05-09T10:41:46.735 INFO:teuthology.orchestra.run.smithi105.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T10:41:46.762 INFO:teuthology.orchestra.run.smithi105.stderr:done 2024-05-09T10:41:46.764 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-05-09T10:41:46.821 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.2LJDRjpjiI 2024-05-09T10:41:46.822 DEBUG:teuthology.orchestra.run.smithi105:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.2LJDRjpjiI 2024-05-09T10:41:46.886 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.2LJDRjpjiI 2024-05-09T10:41:47.025 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.2LJDRjpjiI is 6KB 2024-05-09T10:41:47.073 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.2LJDRjpjiI 2024-05-09T10:41:47.088 DEBUG:teuthology.orchestra.run.smithi105:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T10:41:47.149 INFO:teuthology.orchestra.run.smithi105.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T10:41:47.149 INFO:teuthology.orchestra.run.smithi105.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T10:41:47.149 INFO:teuthology.orchestra.run.smithi105.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T10:41:47.149 INFO:teuthology.orchestra.run.smithi105.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T10:41:47.150 DEBUG:teuthology.orchestra.run.smithi105:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T10:41:47.278 DEBUG:teuthology.orchestra.run.smithi105:> sudo shutdown -r now 2024-05-09T10:41:48.722 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T10:41:48.769 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T10:41:49.097 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T10:41:49.097 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T10:41:49.097 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T10:41:49.097 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T10:41:49.902 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T10:41:49.902 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:41:49.902 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-05-09T10:41:49.902 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T10:41:49.903 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T10:41:49.903 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T10:41:49.903 INFO:teuthology.orchestra.run.smithi098.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T10:41:49.903 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:41:49.903 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-05-09T10:41:50.255 DEBUG:teuthology.orchestra.run.smithi098:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T10:41:50.329 INFO:teuthology.orchestra.run.smithi098.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T10:41:50.329 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T10:41:50.329 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T10:41:50.329 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T10:41:50.330 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T10:41:50.330 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-09T10:41:50.330 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi098.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T10:41:50.330 DEBUG:teuthology.orchestra.run.smithi098:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T10:41:50.379 INFO:teuthology.orchestra.run.smithi098.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T10:41:50.379 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T10:41:50.379 DEBUG:teuthology.orchestra.run.smithi098:> sudo rpm -qi grub2-tools 2024-05-09T10:41:50.417 INFO:teuthology.orchestra.run.smithi098.stdout:Name : grub2-tools 2024-05-09T10:41:50.417 INFO:teuthology.orchestra.run.smithi098.stdout:Epoch : 1 2024-05-09T10:41:50.417 INFO:teuthology.orchestra.run.smithi098.stdout:Version : 2.06 2024-05-09T10:41:50.417 INFO:teuthology.orchestra.run.smithi098.stdout:Release : 68.el9 2024-05-09T10:41:50.417 INFO:teuthology.orchestra.run.smithi098.stdout:Architecture: x86_64 2024-05-09T10:41:50.417 INFO:teuthology.orchestra.run.smithi098.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T10:41:50.417 INFO:teuthology.orchestra.run.smithi098.stdout:Group : Unspecified 2024-05-09T10:41:50.417 INFO:teuthology.orchestra.run.smithi098.stdout:Size : 8264254 2024-05-09T10:41:50.417 INFO:teuthology.orchestra.run.smithi098.stdout:License : GPLv3+ 2024-05-09T10:41:50.417 INFO:teuthology.orchestra.run.smithi098.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T10:41:50.417 INFO:teuthology.orchestra.run.smithi098.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T10:41:50.417 INFO:teuthology.orchestra.run.smithi098.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T10:41:50.418 INFO:teuthology.orchestra.run.smithi098.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T10:41:50.418 INFO:teuthology.orchestra.run.smithi098.stdout:Packager : builder@centos.org 2024-05-09T10:41:50.418 INFO:teuthology.orchestra.run.smithi098.stdout:Vendor : CentOS 2024-05-09T10:41:50.418 INFO:teuthology.orchestra.run.smithi098.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T10:41:50.418 INFO:teuthology.orchestra.run.smithi098.stdout:Summary : Support tools for GRUB. 2024-05-09T10:41:50.418 INFO:teuthology.orchestra.run.smithi098.stdout:Description : 2024-05-09T10:41:50.418 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:41:50.418 INFO:teuthology.orchestra.run.smithi098.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T10:41:50.418 INFO:teuthology.orchestra.run.smithi098.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T10:41:50.418 INFO:teuthology.orchestra.run.smithi098.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T10:41:50.418 INFO:teuthology.orchestra.run.smithi098.stdout:hardware devices. 2024-05-09T10:41:50.418 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:41:50.418 INFO:teuthology.orchestra.run.smithi098.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T10:41:50.419 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T10:41:50.419 INFO:teuthology.task.kernel:Updating grub on smithi098 to boot 5.14.0-444.el9.x86_64 2024-05-09T10:41:50.419 DEBUG:teuthology.orchestra.run.smithi098:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T10:41:51.005 INFO:teuthology.orchestra.run.smithi098.stderr:Generating grub configuration file ... 2024-05-09T10:41:52.377 INFO:teuthology.orchestra.run.smithi098.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T10:41:52.405 INFO:teuthology.orchestra.run.smithi098.stderr:done 2024-05-09T10:41:52.407 DEBUG:teuthology.orchestra.run.smithi098:> mktemp 2024-05-09T10:41:52.423 INFO:teuthology.orchestra.run.smithi098.stdout:/tmp/tmp.DnvroExuV0 2024-05-09T10:41:52.424 DEBUG:teuthology.orchestra.run.smithi098:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.DnvroExuV0 2024-05-09T10:41:52.489 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 0666 /tmp/tmp.DnvroExuV0 2024-05-09T10:41:52.621 DEBUG:teuthology.orchestra.remote:smithi098:/tmp/tmp.DnvroExuV0 is 6KB 2024-05-09T10:41:52.641 DEBUG:teuthology.orchestra.run.smithi098:> rm -fr /tmp/tmp.DnvroExuV0 2024-05-09T10:41:52.655 DEBUG:teuthology.orchestra.run.smithi098:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T10:41:52.723 INFO:teuthology.orchestra.run.smithi098.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T10:41:52.723 INFO:teuthology.orchestra.run.smithi098.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T10:41:52.723 INFO:teuthology.orchestra.run.smithi098.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T10:41:52.723 INFO:teuthology.orchestra.run.smithi098.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T10:41:52.725 DEBUG:teuthology.orchestra.run.smithi098:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T10:41:52.872 DEBUG:teuthology.orchestra.run.smithi098:> sudo shutdown -r now 2024-05-09T10:42:17.306 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-05-09T10:42:17.307 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-05-09T10:42:17.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T10:42:22.878 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-09T10:42:22.878 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-05-09T10:42:22.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T10:43:17.363 DEBUG:teuthology.orchestra.remote:timed out 2024-05-09T10:43:22.886 DEBUG:teuthology.orchestra.remote:timed out 2024-05-09T10:43:26.366 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-05-09T10:43:26.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T10:43:26.997 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-05-09T10:43:27.407 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-05-09T10:43:27.407 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-444.el9.x86_64"... 2024-05-09T10:43:27.407 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2024-05-09T10:43:27.422 INFO:teuthology.orchestra.run.smithi105.stdout:5.14.0-444.el9.x86_64 2024-05-09T10:43:27.422 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T10:43:27.422 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T10:43:27.422 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T10:43:28.424 DEBUG:teuthology.parallel:result is None 2024-05-09T10:43:31.887 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-05-09T10:43:31.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T10:43:32.254 DEBUG:teuthology.orchestra.run.smithi098:> true 2024-05-09T10:43:32.707 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-05-09T10:43:32.707 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-444.el9.x86_64"... 2024-05-09T10:43:32.707 DEBUG:teuthology.orchestra.run.smithi098:> uname -r 2024-05-09T10:43:32.722 INFO:teuthology.orchestra.run.smithi098.stdout:5.14.0-444.el9.x86_64 2024-05-09T10:43:32.722 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T10:43:32.722 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T10:43:32.722 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T10:43:33.723 DEBUG:teuthology.parallel:result is None 2024-05-09T10:43:33.723 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-09T10:43:33.730 INFO:teuthology.task.internal:Creating test directory... 2024-05-09T10:43:33.730 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T10:43:33.732 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T10:43:33.751 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-09T10:43:33.766 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-09T10:43:33.817 INFO:teuthology.task.internal:Creating archive directory... 2024-05-09T10:43:33.817 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T10:43:33.819 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T10:43:33.859 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-09T10:43:33.866 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-09T10:43:33.866 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T10:43:33.876 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T10:43:33.957 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T10:43:33.979 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T10:43:34.034 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T10:43:34.100 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T10:43:34.102 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-09T10:43:34.796 INFO:teuthology.task.internal:Configuring sudo... 2024-05-09T10:43:34.796 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T10:43:34.799 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T10:43:34.828 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-09T10:43:34.907 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-09T10:43:34.907 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T10:43:34.910 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T10:43:34.927 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T10:43:35.005 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T10:43:35.046 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T10:43:35.132 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T10:43:35.167 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:43:35.167 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T10:43:35.232 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T10:43:35.277 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T10:43:35.329 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T10:43:35.411 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T10:43:35.454 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:43:35.454 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T10:43:35.519 DEBUG:teuthology.orchestra.run.smithi098:> sudo service rsyslog restart 2024-05-09T10:43:35.522 DEBUG:teuthology.orchestra.run.smithi105:> sudo service rsyslog restart 2024-05-09T10:43:35.559 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T10:43:35.610 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T10:43:36.116 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-09T10:43:36.125 INFO:teuthology.task.internal:Starting timer... 2024-05-09T10:43:36.125 INFO:teuthology.run_tasks:Running task pcp... 2024-05-09T10:43:36.165 INFO:teuthology.run_tasks:Running task selinux... 2024-05-09T10:43:36.194 DEBUG:teuthology.orchestra.run.smithi098:> sudo service auditd rotate 2024-05-09T10:43:36.288 INFO:teuthology.orchestra.run.smithi098.stdout:Rotating logs: 2024-05-09T10:43:36.290 DEBUG:teuthology.orchestra.run.smithi105:> sudo service auditd rotate 2024-05-09T10:43:36.333 INFO:teuthology.orchestra.run.smithi105.stdout:Rotating logs: 2024-05-09T10:43:36.335 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-09T10:43:36.335 DEBUG:teuthology.orchestra.run.smithi098:> /usr/sbin/getenforce 2024-05-09T10:43:36.358 INFO:teuthology.orchestra.run.smithi098.stdout:Permissive 2024-05-09T10:43:36.359 DEBUG:teuthology.orchestra.run.smithi105:> /usr/sbin/getenforce 2024-05-09T10:43:36.408 INFO:teuthology.orchestra.run.smithi105.stdout:Permissive 2024-05-09T10:43:36.408 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi098.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi105.front.sepia.ceph.com': 'permissive'} 2024-05-09T10:43:36.409 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-09T10:43:36.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T10:43:36.434 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-09T10:43:36.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T10:43:36.476 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-09T10:43:36.476 DEBUG:teuthology.orchestra.run.smithi098:> sudo /usr/sbin/setenforce permissive 2024-05-09T10:43:36.553 DEBUG:teuthology.orchestra.run.smithi105:> sudo /usr/sbin/setenforce permissive 2024-05-09T10:43:36.582 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-09T10:43:36.592 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-09T10:43:36.597 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-09T10:43:36.725 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-09T10:43:36.741 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-09T10:43:36.742 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi098.front.sepia.ceph.com,smithi105.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-09T10:47:32.721 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi098.front.sepia.ceph.com'), Remote(name='ubuntu@smithi105.front.sepia.ceph.com')] 2024-05-09T10:47:32.722 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-05-09T10:47:32.724 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T10:47:32.842 DEBUG:teuthology.orchestra.run.smithi098:> true 2024-05-09T10:47:32.920 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-05-09T10:47:32.921 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-05-09T10:47:32.921 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T10:47:32.995 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-05-09T10:47:33.073 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-05-09T10:47:33.073 INFO:teuthology.run_tasks:Running task clock... 2024-05-09T10:47:33.084 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-09T10:47:33.084 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T10:47:33.085 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T10:47:33.087 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T10:47:33.087 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T10:47:33.112 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T10:47:33.126 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T10:47:33.141 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T10:47:33.150 INFO:teuthology.orchestra.run.smithi098.stderr:sudo: ntpd: command not found 2024-05-09T10:47:33.155 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T10:47:33.161 INFO:teuthology.orchestra.run.smithi098.stdout:506 Cannot talk to daemon 2024-05-09T10:47:33.175 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T10:47:33.178 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: ntpd: command not found 2024-05-09T10:47:33.189 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T10:47:33.189 INFO:teuthology.orchestra.run.smithi105.stdout:506 Cannot talk to daemon 2024-05-09T10:47:33.203 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T10:47:33.217 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T10:47:33.246 INFO:teuthology.orchestra.run.smithi098.stderr:bash: line 1: ntpq: command not found 2024-05-09T10:47:33.250 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T10:47:33.250 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2024-05-09T10:47:33.250 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T10:47:33.250 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T10:47:33.250 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T10:47:33.250 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T10:47:33.272 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-05-09T10:47:33.275 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T10:47:33.275 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-05-09T10:47:33.275 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T10:47:33.275 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T10:47:33.275 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T10:47:33.275 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T10:47:33.276 INFO:teuthology.run_tasks:Running task install... 2024-05-09T10:47:33.285 DEBUG:teuthology.task.install:project ceph 2024-05-09T10:47:33.285 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20'}} 2024-05-09T10:47:33.285 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20'} 2024-05-09T10:47:33.285 INFO:teuthology.task.install:Using flavor: default 2024-05-09T10:47:33.291 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-09T10:47:33.291 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-09T10:47:33.291 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20', 'tag': None, 'wait_for_package': False} 2024-05-09T10:47:33.291 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20 2024-05-09T10:47:33.293 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20', 'tag': None, 'wait_for_package': False} 2024-05-09T10:47:33.293 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20 2024-05-09T10:47:33.517 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/centos/9/flavors/default/ 2024-05-09T10:47:33.517 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1629.g4f7075d1 2024-05-09T10:47:33.528 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/centos/9/flavors/default/ 2024-05-09T10:47:33.529 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1629.g4f7075d1 2024-05-09T10:47:33.627 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T10:47:33.627 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:47:33.627 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T10:47:33.640 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-neesingh-testing1-20240507/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T10:47:33.641 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:47:33.641 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T10:47:33.653 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T10:47:33.654 DEBUG:teuthology.orchestra.run.smithi098:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T10:47:33.666 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T10:47:33.666 DEBUG:teuthology.orchestra.run.smithi105:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T10:47:33.719 DEBUG:teuthology.orchestra.run.smithi098:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T10:47:33.732 DEBUG:teuthology.orchestra.run.smithi105:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T10:47:33.794 DEBUG:teuthology.orchestra.run.smithi098:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T10:47:33.807 DEBUG:teuthology.orchestra.run.smithi105:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T10:47:33.858 INFO:teuthology.orchestra.run.smithi098.stdout:check_obsoletes = 1 2024-05-09T10:47:33.860 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum clean all 2024-05-09T10:47:33.871 INFO:teuthology.orchestra.run.smithi105.stdout:check_obsoletes = 1 2024-05-09T10:47:33.873 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum clean all 2024-05-09T10:47:34.107 INFO:teuthology.orchestra.run.smithi098.stdout:54 files removed 2024-05-09T10:47:34.111 INFO:teuthology.orchestra.run.smithi105.stdout:54 files removed 2024-05-09T10:47:34.138 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T10:47:34.139 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T10:47:34.769 INFO:teuthology.orchestra.run.smithi105.stdout:ceph packages for x86_64 230 kB/s | 82 kB 00:00 2024-05-09T10:47:34.782 INFO:teuthology.orchestra.run.smithi098.stdout:ceph packages for x86_64 222 kB/s | 82 kB 00:00 2024-05-09T10:47:35.136 INFO:teuthology.orchestra.run.smithi105.stdout:ceph noarch packages 32 kB/s | 10 kB 00:00 2024-05-09T10:47:35.249 INFO:teuthology.orchestra.run.smithi098.stdout:ceph noarch packages 24 kB/s | 10 kB 00:00 2024-05-09T10:47:35.567 INFO:teuthology.orchestra.run.smithi098.stdout:ceph source packages 6.9 kB/s | 1.9 kB 00:00 2024-05-09T10:47:35.570 INFO:teuthology.orchestra.run.smithi105.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-05-09T10:47:36.409 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.1 MB 00:00 2024-05-09T10:47:36.528 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - BaseOS 8.8 MB/s | 8.1 MB 00:00 2024-05-09T10:47:38.760 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-05-09T10:47:40.246 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - AppStream 7.3 MB/s | 19 MB 00:02 2024-05-09T10:47:43.953 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.2 MB 00:01 2024-05-09T10:47:45.587 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 16 kB 00:00 2024-05-09T10:47:45.655 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.2 MB 00:01 2024-05-09T10:47:47.481 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-05-09T10:47:47.841 INFO:teuthology.orchestra.run.smithi098.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-05-09T10:47:48.994 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-05-09T10:47:52.972 INFO:teuthology.orchestra.run.smithi098.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-09T10:47:54.150 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-09T10:47:55.340 INFO:teuthology.orchestra.run.smithi098.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T10:47:55.341 INFO:teuthology.orchestra.run.smithi098.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T10:47:55.352 INFO:teuthology.orchestra.run.smithi098.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T10:47:55.353 INFO:teuthology.orchestra.run.smithi098.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T10:47:55.354 INFO:teuthology.orchestra.run.smithi098.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T10:47:55.355 INFO:teuthology.orchestra.run.smithi098.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T10:47:55.357 INFO:teuthology.orchestra.run.smithi098.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T10:47:55.359 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T10:47:55.359 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T10:47:55.421 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-05-09T10:47:55.448 INFO:teuthology.orchestra.run.smithi098.stdout:======================================================================================= 2024-05-09T10:47:55.449 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2024-05-09T10:47:55.449 INFO:teuthology.orchestra.run.smithi098.stdout:======================================================================================= 2024-05-09T10:47:55.449 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-05-09T10:47:55.449 INFO:teuthology.orchestra.run.smithi098.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T10:47:55.449 INFO:teuthology.orchestra.run.smithi098.stdout: ceph x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 6.6 k 2024-05-09T10:47:55.449 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 5.3 M 2024-05-09T10:47:55.449 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 887 k 2024-05-09T10:47:55.449 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 145 k 2024-05-09T10:47:55.449 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 1.5 M 2024-05-09T10:47:55.449 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1629.g4f7075d1.el9 ceph-noarch 138 k 2024-05-09T10:47:55.449 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1629.g4f7075d1.el9 ceph-noarch 1.8 M 2024-05-09T10:47:55.449 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1629.g4f7075d1.el9 ceph-noarch 7.4 M 2024-05-09T10:47:55.450 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook noarch 2:19.0.0-1629.g4f7075d1.el9 ceph-noarch 49 k 2024-05-09T10:47:55.450 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 9.8 M 2024-05-09T10:47:55.450 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 47 M 2024-05-09T10:47:55.450 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume noarch 2:19.0.0-1629.g4f7075d1.el9 ceph-noarch 264 k 2024-05-09T10:47:55.450 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm noarch 2:19.0.0-1629.g4f7075d1.el9 ceph-noarch 748 k 2024-05-09T10:47:55.450 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 220 k 2024-05-09T10:47:55.450 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-shell x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 27 k 2024-05-09T10:47:55.450 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top noarch 2:19.0.0-1629.g4f7075d1.el9 ceph-noarch 18 k 2024-05-09T10:47:55.450 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T10:47:55.450 INFO:teuthology.orchestra.run.smithi098.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T10:47:55.450 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T10:47:55.450 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 31 k 2024-05-09T10:47:55.450 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2 x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 744 k 2024-05-09T10:47:55.450 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 127 k 2024-05-09T10:47:55.451 INFO:teuthology.orchestra.run.smithi098.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T10:47:55.451 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T10:47:55.451 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T10:47:55.451 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T10:47:55.451 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T10:47:55.451 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T10:47:55.451 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 162 k 2024-05-09T10:47:55.451 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 321 k 2024-05-09T10:47:55.451 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 298 k 2024-05-09T10:47:55.451 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 99 k 2024-05-09T10:47:55.451 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 85 k 2024-05-09T10:47:55.451 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 3.1 M 2024-05-09T10:47:55.452 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 170 k 2024-05-09T10:47:55.452 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T10:47:55.452 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrading: 2024-05-09T10:47:55.452 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T10:47:55.452 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T10:47:55.452 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T10:47:55.452 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T10:47:55.452 INFO:teuthology.orchestra.run.smithi098.stdout: librados2 x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 3.4 M 2024-05-09T10:47:55.452 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1 x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 3.1 M 2024-05-09T10:47:55.452 INFO:teuthology.orchestra.run.smithi098.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T10:47:55.452 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T10:47:55.452 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2024-05-09T10:47:55.452 INFO:teuthology.orchestra.run.smithi098.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T10:47:55.453 INFO:teuthology.orchestra.run.smithi098.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T10:47:55.453 INFO:teuthology.orchestra.run.smithi098.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T10:47:55.453 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 20 M 2024-05-09T10:47:55.453 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1629.g4f7075d1.el9 ceph-noarch 31 k 2024-05-09T10:47:55.453 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 2.1 M 2024-05-09T10:47:55.453 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1629.g4f7075d1.el9 ceph-noarch 248 k 2024-05-09T10:47:55.453 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 4.7 M 2024-05-09T10:47:55.453 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 17 M 2024-05-09T10:47:55.453 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1629.g4f7075d1.el9 ceph-noarch 15 k 2024-05-09T10:47:55.453 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 25 k 2024-05-09T10:47:55.453 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T10:47:55.453 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T10:47:55.453 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T10:47:55.453 INFO:teuthology.orchestra.run.smithi098.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T10:47:55.454 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T10:47:55.454 INFO:teuthology.orchestra.run.smithi098.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T10:47:55.454 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T10:47:55.454 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T10:47:55.454 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 163 k 2024-05-09T10:47:55.454 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T10:47:55.454 INFO:teuthology.orchestra.run.smithi098.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T10:47:55.454 INFO:teuthology.orchestra.run.smithi098.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T10:47:55.459 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T10:47:55.459 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T10:47:55.459 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1 x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 501 k 2024-05-09T10:47:55.459 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T10:47:55.459 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2 x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 5.1 M 2024-05-09T10:47:55.459 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T10:47:55.459 INFO:teuthology.orchestra.run.smithi098.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T10:47:55.459 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T10:47:55.459 INFO:teuthology.orchestra.run.smithi098.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T10:47:55.459 INFO:teuthology.orchestra.run.smithi098.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T10:47:55.459 INFO:teuthology.orchestra.run.smithi098.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T10:47:55.459 INFO:teuthology.orchestra.run.smithi098.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T10:47:55.459 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T10:47:55.460 INFO:teuthology.orchestra.run.smithi098.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T10:47:55.460 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T10:47:55.460 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T10:47:55.460 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T10:47:55.460 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T10:47:55.460 INFO:teuthology.orchestra.run.smithi098.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T10:47:55.460 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T10:47:55.460 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T10:47:55.460 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T10:47:55.460 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 45 k 2024-05-09T10:47:55.460 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 131 k 2024-05-09T10:47:55.460 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T10:47:55.460 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T10:47:55.461 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T10:47:55.461 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T10:47:55.461 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T10:47:55.461 INFO:teuthology.orchestra.run.smithi098.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T10:47:55.461 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T10:47:55.461 INFO:teuthology.orchestra.run.smithi098.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T10:47:55.461 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T10:47:55.461 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T10:47:55.461 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T10:47:55.461 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T10:47:55.461 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T10:47:55.461 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T10:47:55.461 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T10:47:55.461 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T10:47:55.462 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T10:47:55.462 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T10:47:55.462 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T10:47:55.462 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T10:47:55.462 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T10:47:55.462 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T10:47:55.462 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T10:47:55.462 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T10:47:55.462 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T10:47:55.462 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T10:47:55.462 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T10:47:55.462 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T10:47:55.462 INFO:teuthology.orchestra.run.smithi098.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T10:47:55.462 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T10:47:55.463 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T10:47:55.463 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T10:47:55.463 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T10:47:55.463 INFO:teuthology.orchestra.run.smithi098.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T10:47:55.463 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T10:47:55.463 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T10:47:55.463 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T10:47:55.463 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T10:47:55.463 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T10:47:55.463 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T10:47:55.463 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T10:47:55.463 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T10:47:55.463 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T10:47:55.463 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T10:47:55.463 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T10:47:55.463 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T10:47:55.464 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T10:47:55.464 INFO:teuthology.orchestra.run.smithi098.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T10:47:55.464 INFO:teuthology.orchestra.run.smithi098.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T10:47:55.464 INFO:teuthology.orchestra.run.smithi098.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T10:47:55.464 INFO:teuthology.orchestra.run.smithi098.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T10:47:55.464 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T10:47:55.464 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T10:47:55.464 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T10:47:55.464 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T10:47:55.464 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T10:47:55.464 INFO:teuthology.orchestra.run.smithi098.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T10:47:55.464 INFO:teuthology.orchestra.run.smithi098.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T10:47:55.464 INFO:teuthology.orchestra.run.smithi098.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T10:47:55.464 INFO:teuthology.orchestra.run.smithi098.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T10:47:55.464 INFO:teuthology.orchestra.run.smithi098.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T10:47:55.464 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T10:47:55.464 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T10:47:55.465 INFO:teuthology.orchestra.run.smithi098.stdout:Installing weak dependencies: 2024-05-09T10:47:55.465 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T10:47:55.465 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:47:55.465 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-05-09T10:47:55.465 INFO:teuthology.orchestra.run.smithi098.stdout:======================================================================================= 2024-05-09T10:47:55.465 INFO:teuthology.orchestra.run.smithi098.stdout:Install 148 Packages 2024-05-09T10:47:55.465 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrade 8 Packages 2024-05-09T10:47:55.465 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:47:55.466 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 220 M 2024-05-09T10:47:55.466 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-05-09T10:47:56.432 INFO:teuthology.orchestra.run.smithi098.stdout:(1/156): ceph-19.0.0-1629.g4f7075d1.el9.x86_64. 36 kB/s | 6.6 kB 00:00 2024-05-09T10:47:56.493 INFO:teuthology.orchestra.run.smithi105.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T10:47:56.494 INFO:teuthology.orchestra.run.smithi105.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T10:47:56.505 INFO:teuthology.orchestra.run.smithi105.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T10:47:56.505 INFO:teuthology.orchestra.run.smithi105.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T10:47:56.506 INFO:teuthology.orchestra.run.smithi105.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T10:47:56.507 INFO:teuthology.orchestra.run.smithi105.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T10:47:56.509 INFO:teuthology.orchestra.run.smithi105.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T10:47:56.511 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T10:47:56.512 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T10:47:56.572 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-05-09T10:47:56.591 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================= 2024-05-09T10:47:56.591 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2024-05-09T10:47:56.591 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================= 2024-05-09T10:47:56.591 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2024-05-09T10:47:56.591 INFO:teuthology.orchestra.run.smithi105.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T10:47:56.592 INFO:teuthology.orchestra.run.smithi105.stdout: ceph x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 6.6 k 2024-05-09T10:47:56.592 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 5.3 M 2024-05-09T10:47:56.592 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 887 k 2024-05-09T10:47:56.592 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 145 k 2024-05-09T10:47:56.592 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 1.5 M 2024-05-09T10:47:56.592 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1629.g4f7075d1.el9 ceph-noarch 138 k 2024-05-09T10:47:56.592 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1629.g4f7075d1.el9 ceph-noarch 1.8 M 2024-05-09T10:47:56.592 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1629.g4f7075d1.el9 ceph-noarch 7.4 M 2024-05-09T10:47:56.592 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook noarch 2:19.0.0-1629.g4f7075d1.el9 ceph-noarch 49 k 2024-05-09T10:47:56.592 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 9.8 M 2024-05-09T10:47:56.592 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 47 M 2024-05-09T10:47:56.592 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume noarch 2:19.0.0-1629.g4f7075d1.el9 ceph-noarch 264 k 2024-05-09T10:47:56.592 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm noarch 2:19.0.0-1629.g4f7075d1.el9 ceph-noarch 748 k 2024-05-09T10:47:56.593 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 220 k 2024-05-09T10:47:56.593 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 27 k 2024-05-09T10:47:56.593 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top noarch 2:19.0.0-1629.g4f7075d1.el9 ceph-noarch 18 k 2024-05-09T10:47:56.593 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T10:47:56.593 INFO:teuthology.orchestra.run.smithi105.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T10:47:56.593 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T10:47:56.593 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 31 k 2024-05-09T10:47:56.593 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2 x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 744 k 2024-05-09T10:47:56.593 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 127 k 2024-05-09T10:47:56.593 INFO:teuthology.orchestra.run.smithi105.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T10:47:56.593 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T10:47:56.593 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T10:47:56.593 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T10:47:56.593 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T10:47:56.594 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T10:47:56.594 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 162 k 2024-05-09T10:47:56.594 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 321 k 2024-05-09T10:47:56.594 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 298 k 2024-05-09T10:47:56.594 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 99 k 2024-05-09T10:47:56.594 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 85 k 2024-05-09T10:47:56.594 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 3.1 M 2024-05-09T10:47:56.594 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 170 k 2024-05-09T10:47:56.594 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T10:47:56.594 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrading: 2024-05-09T10:47:56.594 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T10:47:56.594 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T10:47:56.594 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T10:47:56.594 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T10:47:56.595 INFO:teuthology.orchestra.run.smithi105.stdout: librados2 x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 3.4 M 2024-05-09T10:47:56.595 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1 x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 3.1 M 2024-05-09T10:47:56.595 INFO:teuthology.orchestra.run.smithi105.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T10:47:56.595 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T10:47:56.595 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2024-05-09T10:47:56.595 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T10:47:56.595 INFO:teuthology.orchestra.run.smithi105.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T10:47:56.595 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T10:47:56.595 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 20 M 2024-05-09T10:47:56.595 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1629.g4f7075d1.el9 ceph-noarch 31 k 2024-05-09T10:47:56.595 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 2.1 M 2024-05-09T10:47:56.595 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1629.g4f7075d1.el9 ceph-noarch 248 k 2024-05-09T10:47:56.595 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 4.7 M 2024-05-09T10:47:56.595 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 17 M 2024-05-09T10:47:56.596 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1629.g4f7075d1.el9 ceph-noarch 15 k 2024-05-09T10:47:56.596 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 25 k 2024-05-09T10:47:56.596 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T10:47:56.596 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T10:47:56.596 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T10:47:56.596 INFO:teuthology.orchestra.run.smithi105.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T10:47:56.596 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T10:47:56.596 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T10:47:56.596 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T10:47:56.596 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T10:47:56.596 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 163 k 2024-05-09T10:47:56.596 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T10:47:56.596 INFO:teuthology.orchestra.run.smithi105.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T10:47:56.596 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T10:47:56.597 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T10:47:56.597 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T10:47:56.597 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1 x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 501 k 2024-05-09T10:47:56.597 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T10:47:56.597 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2 x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 5.1 M 2024-05-09T10:47:56.597 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T10:47:56.597 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T10:47:56.597 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T10:47:56.597 INFO:teuthology.orchestra.run.smithi105.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T10:47:56.597 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T10:47:56.597 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T10:47:56.597 INFO:teuthology.orchestra.run.smithi105.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T10:47:56.597 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T10:47:56.598 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T10:47:56.598 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T10:47:56.598 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T10:47:56.598 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T10:47:56.598 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T10:47:56.598 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T10:47:56.598 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T10:47:56.598 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T10:47:56.598 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T10:47:56.598 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 45 k 2024-05-09T10:47:56.598 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common x86_64 2:19.0.0-1629.g4f7075d1.el9 ceph 131 k 2024-05-09T10:47:56.598 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T10:47:56.598 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T10:47:56.599 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T10:47:56.599 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T10:47:56.599 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T10:47:56.599 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T10:47:56.599 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T10:47:56.599 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T10:47:56.599 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T10:47:56.599 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T10:47:56.599 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T10:47:56.599 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T10:47:56.599 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T10:47:56.599 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T10:47:56.599 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T10:47:56.599 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T10:47:56.600 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T10:47:56.600 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T10:47:56.600 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T10:47:56.600 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T10:47:56.600 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T10:47:56.600 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T10:47:56.600 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T10:47:56.600 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T10:47:56.600 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T10:47:56.600 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T10:47:56.600 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T10:47:56.600 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T10:47:56.600 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T10:47:56.600 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T10:47:56.600 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T10:47:56.601 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T10:47:56.601 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T10:47:56.601 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T10:47:56.601 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T10:47:56.601 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T10:47:56.601 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T10:47:56.601 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T10:47:56.601 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T10:47:56.601 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T10:47:56.601 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T10:47:56.601 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T10:47:56.601 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T10:47:56.601 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T10:47:56.601 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T10:47:56.601 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T10:47:56.602 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T10:47:56.602 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T10:47:56.602 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T10:47:56.602 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T10:47:56.602 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T10:47:56.602 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T10:47:56.602 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T10:47:56.602 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T10:47:56.602 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T10:47:56.602 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T10:47:56.602 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T10:47:56.602 INFO:teuthology.orchestra.run.smithi105.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T10:47:56.602 INFO:teuthology.orchestra.run.smithi105.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T10:47:56.603 INFO:teuthology.orchestra.run.smithi105.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T10:47:56.603 INFO:teuthology.orchestra.run.smithi105.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T10:47:56.603 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T10:47:56.603 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T10:47:56.603 INFO:teuthology.orchestra.run.smithi105.stdout:Installing weak dependencies: 2024-05-09T10:47:56.603 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T10:47:56.603 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:47:56.603 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2024-05-09T10:47:56.603 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================= 2024-05-09T10:47:56.603 INFO:teuthology.orchestra.run.smithi105.stdout:Install 148 Packages 2024-05-09T10:47:56.603 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrade 8 Packages 2024-05-09T10:47:56.603 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:47:56.604 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 220 M 2024-05-09T10:47:56.604 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2024-05-09T10:47:56.666 INFO:teuthology.orchestra.run.smithi098.stdout:(2/156): ceph-fuse-19.0.0-1629.g4f7075d1.el9.x8 3.7 MB/s | 887 kB 00:00 2024-05-09T10:47:56.733 INFO:teuthology.orchestra.run.smithi098.stdout:(3/156): ceph-immutable-object-cache-19.0.0-162 2.1 MB/s | 145 kB 00:00 2024-05-09T10:47:57.017 INFO:teuthology.orchestra.run.smithi098.stdout:(4/156): ceph-mds-19.0.0-1629.g4f7075d1.el9.x86 7.5 MB/s | 2.1 MB 00:00 2024-05-09T10:47:57.233 INFO:teuthology.orchestra.run.smithi098.stdout:(5/156): ceph-mgr-19.0.0-1629.g4f7075d1.el9.x86 6.9 MB/s | 1.5 MB 00:00 2024-05-09T10:47:57.234 INFO:teuthology.orchestra.run.smithi105.stdout:(1/156): ceph-19.0.0-1629.g4f7075d1.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-05-09T10:47:57.642 INFO:teuthology.orchestra.run.smithi098.stdout:(6/156): ceph-base-19.0.0-1629.g4f7075d1.el9.x8 3.8 MB/s | 5.3 MB 00:01 2024-05-09T10:47:57.986 INFO:teuthology.orchestra.run.smithi098.stdout:(7/156): ceph-common-19.0.0-1629.g4f7075d1.el9. 12 MB/s | 20 MB 00:01 2024-05-09T10:47:58.101 INFO:teuthology.orchestra.run.smithi098.stdout:(8/156): ceph-mon-19.0.0-1629.g4f7075d1.el9.x86 5.4 MB/s | 4.7 MB 00:00 2024-05-09T10:47:58.168 INFO:teuthology.orchestra.run.smithi098.stdout:(9/156): ceph-selinux-19.0.0-1629.g4f7075d1.el9 378 kB/s | 25 kB 00:00 2024-05-09T10:47:58.301 INFO:teuthology.orchestra.run.smithi105.stdout:(2/156): ceph-fuse-19.0.0-1629.g4f7075d1.el9.x8 831 kB/s | 887 kB 00:01 2024-05-09T10:47:58.485 INFO:teuthology.orchestra.run.smithi105.stdout:(3/156): ceph-immutable-object-cache-19.0.0-162 792 kB/s | 145 kB 00:00 2024-05-09T10:47:58.818 INFO:teuthology.orchestra.run.smithi098.stdout:(10/156): ceph-radosgw-19.0.0-1629.g4f7075d1.el 12 MB/s | 9.8 MB 00:00 2024-05-09T10:47:58.885 INFO:teuthology.orchestra.run.smithi098.stdout:(11/156): cephfs-mirror-19.0.0-1629.g4f7075d1.e 3.2 MB/s | 220 kB 00:00 2024-05-09T10:47:58.952 INFO:teuthology.orchestra.run.smithi098.stdout:(12/156): cephfs-shell-19.0.0-1629.g4f7075d1.el 406 kB/s | 27 kB 00:00 2024-05-09T10:47:59.002 INFO:teuthology.orchestra.run.smithi098.stdout:(13/156): libcephfs-devel-19.0.0-1629.g4f7075d1 622 kB/s | 31 kB 00:00 2024-05-09T10:47:59.102 INFO:teuthology.orchestra.run.smithi098.stdout:(14/156): libcephfs2-19.0.0-1629.g4f7075d1.el9. 7.3 MB/s | 744 kB 00:00 2024-05-09T10:47:59.178 INFO:teuthology.orchestra.run.smithi098.stdout:(15/156): libcephsqlite-19.0.0-1629.g4f7075d1.e 2.1 MB/s | 163 kB 00:00 2024-05-09T10:47:59.245 INFO:teuthology.orchestra.run.smithi098.stdout:(16/156): librados-devel-19.0.0-1629.g4f7075d1. 1.9 MB/s | 127 kB 00:00 2024-05-09T10:47:59.345 INFO:teuthology.orchestra.run.smithi098.stdout:(17/156): libradosstriper1-19.0.0-1629.g4f7075d 4.9 MB/s | 501 kB 00:00 2024-05-09T10:47:59.660 INFO:teuthology.orchestra.run.smithi105.stdout:(4/156): ceph-base-19.0.0-1629.g4f7075d1.el9.x8 1.9 MB/s | 5.3 MB 00:02 2024-05-09T10:47:59.702 INFO:teuthology.orchestra.run.smithi105.stdout:(5/156): ceph-mds-19.0.0-1629.g4f7075d1.el9.x86 1.7 MB/s | 2.1 MB 00:01 2024-05-09T10:47:59.937 INFO:teuthology.orchestra.run.smithi098.stdout:(18/156): librgw2-19.0.0-1629.g4f7075d1.el9.x86 8.6 MB/s | 5.1 MB 00:00 2024-05-09T10:48:00.004 INFO:teuthology.orchestra.run.smithi098.stdout:(19/156): python3-ceph-argparse-19.0.0-1629.g4f 675 kB/s | 45 kB 00:00 2024-05-09T10:48:00.010 INFO:teuthology.orchestra.run.smithi105.stdout:(6/156): ceph-mgr-19.0.0-1629.g4f7075d1.el9.x86 4.3 MB/s | 1.5 MB 00:00 2024-05-09T10:48:00.063 INFO:teuthology.orchestra.run.smithi098.stdout:(20/156): python3-ceph-common-19.0.0-1629.g4f70 2.2 MB/s | 131 kB 00:00 2024-05-09T10:48:00.138 INFO:teuthology.orchestra.run.smithi098.stdout:(21/156): python3-cephfs-19.0.0-1629.g4f7075d1. 2.1 MB/s | 162 kB 00:00 2024-05-09T10:48:00.238 INFO:teuthology.orchestra.run.smithi098.stdout:(22/156): python3-rados-19.0.0-1629.g4f7075d1.e 3.1 MB/s | 321 kB 00:00 2024-05-09T10:48:00.322 INFO:teuthology.orchestra.run.smithi098.stdout:(23/156): python3-rbd-19.0.0-1629.g4f7075d1.el9 3.5 MB/s | 298 kB 00:00 2024-05-09T10:48:00.398 INFO:teuthology.orchestra.run.smithi098.stdout:(24/156): python3-rgw-19.0.0-1629.g4f7075d1.el9 1.3 MB/s | 99 kB 00:00 2024-05-09T10:48:00.473 INFO:teuthology.orchestra.run.smithi098.stdout:(25/156): rbd-fuse-19.0.0-1629.g4f7075d1.el9.x8 1.1 MB/s | 85 kB 00:00 2024-05-09T10:48:00.932 INFO:teuthology.orchestra.run.smithi098.stdout:(26/156): ceph-osd-19.0.0-1629.g4f7075d1.el9.x8 5.1 MB/s | 17 MB 00:03 2024-05-09T10:48:00.986 INFO:teuthology.orchestra.run.smithi105.stdout:(7/156): ceph-mon-19.0.0-1629.g4f7075d1.el9.x86 3.6 MB/s | 4.7 MB 00:01 2024-05-09T10:48:00.999 INFO:teuthology.orchestra.run.smithi098.stdout:(27/156): rbd-mirror-19.0.0-1629.g4f7075d1.el9. 5.9 MB/s | 3.1 MB 00:00 2024-05-09T10:48:01.024 INFO:teuthology.orchestra.run.smithi098.stdout:(28/156): rbd-nbd-19.0.0-1629.g4f7075d1.el9.x86 1.8 MB/s | 170 kB 00:00 2024-05-09T10:48:01.074 INFO:teuthology.orchestra.run.smithi098.stdout:(29/156): ceph-grafana-dashboards-19.0.0-1629.g 418 kB/s | 31 kB 00:00 2024-05-09T10:48:01.116 INFO:teuthology.orchestra.run.smithi098.stdout:(30/156): ceph-mgr-cephadm-19.0.0-1629.g4f7075d 1.5 MB/s | 138 kB 00:00 2024-05-09T10:48:01.525 INFO:teuthology.orchestra.run.smithi098.stdout:(31/156): ceph-mgr-dashboard-19.0.0-1629.g4f707 4.0 MB/s | 1.8 MB 00:00 2024-05-09T10:48:01.625 INFO:teuthology.orchestra.run.smithi098.stdout:(32/156): ceph-mgr-modules-core-19.0.0-1629.g4f 2.4 MB/s | 248 kB 00:00 2024-05-09T10:48:01.709 INFO:teuthology.orchestra.run.smithi098.stdout:(33/156): ceph-mgr-rook-19.0.0-1629.g4f7075d1.e 591 kB/s | 49 kB 00:00 2024-05-09T10:48:01.792 INFO:teuthology.orchestra.run.smithi098.stdout:(34/156): ceph-prometheus-alerts-19.0.0-1629.g4 184 kB/s | 15 kB 00:00 2024-05-09T10:48:01.884 INFO:teuthology.orchestra.run.smithi098.stdout:(35/156): ceph-volume-19.0.0-1629.g4f7075d1.el9 2.8 MB/s | 264 kB 00:00 2024-05-09T10:48:02.176 INFO:teuthology.orchestra.run.smithi098.stdout:(36/156): cephadm-19.0.0-1629.g4f7075d1.el9.noa 2.5 MB/s | 748 kB 00:00 2024-05-09T10:48:02.226 INFO:teuthology.orchestra.run.smithi098.stdout:(37/156): cephfs-top-19.0.0-1629.g4f7075d1.el9. 366 kB/s | 18 kB 00:00 2024-05-09T10:48:02.469 INFO:teuthology.orchestra.run.smithi105.stdout:(8/156): ceph-osd-19.0.0-1629.g4f7075d1.el9.x86 6.8 MB/s | 17 MB 00:02 2024-05-09T10:48:02.586 INFO:teuthology.orchestra.run.smithi105.stdout:(9/156): ceph-radosgw-19.0.0-1629.g4f7075d1.el9 6.1 MB/s | 9.8 MB 00:01 2024-05-09T10:48:02.612 INFO:teuthology.orchestra.run.smithi105.stdout:(10/156): ceph-selinux-19.0.0-1629.g4f7075d1.el 177 kB/s | 25 kB 00:00 2024-05-09T10:48:02.678 INFO:teuthology.orchestra.run.smithi105.stdout:(11/156): cephfs-mirror-19.0.0-1629.g4f7075d1.e 3.2 MB/s | 220 kB 00:00 2024-05-09T10:48:02.685 INFO:teuthology.orchestra.run.smithi098.stdout:(38/156): ceph-mgr-diskprediction-local-19.0.0- 4.7 MB/s | 7.4 MB 00:01 2024-05-09T10:48:02.745 INFO:teuthology.orchestra.run.smithi105.stdout:(12/156): cephfs-shell-19.0.0-1629.g4f7075d1.el 406 kB/s | 27 kB 00:00 2024-05-09T10:48:02.812 INFO:teuthology.orchestra.run.smithi105.stdout:(13/156): libcephfs-devel-19.0.0-1629.g4f7075d1 468 kB/s | 31 kB 00:00 2024-05-09T10:48:02.963 INFO:teuthology.orchestra.run.smithi105.stdout:(14/156): libcephfs2-19.0.0-1629.g4f7075d1.el9. 4.8 MB/s | 744 kB 00:00 2024-05-09T10:48:02.969 INFO:teuthology.orchestra.run.smithi098.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 108 kB/s | 80 kB 00:00 2024-05-09T10:48:03.038 INFO:teuthology.orchestra.run.smithi105.stdout:(15/156): libcephsqlite-19.0.0-1629.g4f7075d1.e 2.1 MB/s | 163 kB 00:00 2024-05-09T10:48:03.113 INFO:teuthology.orchestra.run.smithi105.stdout:(16/156): librados-devel-19.0.0-1629.g4f7075d1. 1.7 MB/s | 127 kB 00:00 2024-05-09T10:48:03.206 INFO:teuthology.orchestra.run.smithi105.stdout:(17/156): libradosstriper1-19.0.0-1629.g4f7075d 5.3 MB/s | 501 kB 00:00 2024-05-09T10:48:03.619 INFO:teuthology.orchestra.run.smithi098.stdout:(40/156): libgfortran-11.4.1-3.el9.x86_64.rpm 860 kB/s | 803 kB 00:00 2024-05-09T10:48:03.761 INFO:teuthology.orchestra.run.smithi098.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 247 kB/s | 195 kB 00:00 2024-05-09T10:48:03.797 INFO:teuthology.orchestra.run.smithi105.stdout:(18/156): librgw2-19.0.0-1629.g4f7075d1.el9.x86 8.7 MB/s | 5.1 MB 00:00 2024-05-09T10:48:03.864 INFO:teuthology.orchestra.run.smithi105.stdout:(19/156): python3-ceph-argparse-19.0.0-1629.g4f 679 kB/s | 45 kB 00:00 2024-05-09T10:48:03.964 INFO:teuthology.orchestra.run.smithi105.stdout:(20/156): python3-ceph-common-19.0.0-1629.g4f70 1.3 MB/s | 131 kB 00:00 2024-05-09T10:48:04.040 INFO:teuthology.orchestra.run.smithi105.stdout:(21/156): python3-cephfs-19.0.0-1629.g4f7075d1. 2.1 MB/s | 162 kB 00:00 2024-05-09T10:48:04.115 INFO:teuthology.orchestra.run.smithi105.stdout:(22/156): python3-rados-19.0.0-1629.g4f7075d1.e 4.2 MB/s | 321 kB 00:00 2024-05-09T10:48:04.207 INFO:teuthology.orchestra.run.smithi105.stdout:(23/156): python3-rbd-19.0.0-1629.g4f7075d1.el9 3.2 MB/s | 298 kB 00:00 2024-05-09T10:48:04.291 INFO:teuthology.orchestra.run.smithi105.stdout:(24/156): python3-rgw-19.0.0-1629.g4f7075d1.el9 1.2 MB/s | 99 kB 00:00 2024-05-09T10:48:04.320 INFO:teuthology.orchestra.run.smithi098.stdout:(42/156): ceph-test-19.0.0-1629.g4f7075d1.el9.x 7.7 MB/s | 47 MB 00:06 2024-05-09T10:48:04.374 INFO:teuthology.orchestra.run.smithi105.stdout:(25/156): rbd-fuse-19.0.0-1629.g4f7075d1.el9.x8 1.0 MB/s | 85 kB 00:00 2024-05-09T10:48:04.708 INFO:teuthology.orchestra.run.smithi105.stdout:(26/156): rbd-mirror-19.0.0-1629.g4f7075d1.el9. 9.3 MB/s | 3.1 MB 00:00 2024-05-09T10:48:04.775 INFO:teuthology.orchestra.run.smithi105.stdout:(27/156): rbd-nbd-19.0.0-1629.g4f7075d1.el9.x86 2.5 MB/s | 170 kB 00:00 2024-05-09T10:48:04.820 INFO:teuthology.orchestra.run.smithi098.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 211 kB/s | 253 kB 00:01 2024-05-09T10:48:04.842 INFO:teuthology.orchestra.run.smithi105.stdout:(28/156): ceph-grafana-dashboards-19.0.0-1629.g 472 kB/s | 31 kB 00:00 2024-05-09T10:48:04.964 INFO:teuthology.orchestra.run.smithi105.stdout:(29/156): ceph-mgr-cephadm-19.0.0-1629.g4f7075d 1.1 MB/s | 138 kB 00:00 2024-05-09T10:48:05.029 INFO:teuthology.orchestra.run.smithi098.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 151 kB/s | 106 kB 00:00 2024-05-09T10:48:05.184 INFO:teuthology.orchestra.run.smithi105.stdout:(30/156): ceph-mgr-dashboard-19.0.0-1629.g4f707 8.1 MB/s | 1.8 MB 00:00 2024-05-09T10:48:05.213 INFO:teuthology.orchestra.run.smithi098.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 865 kB/s | 1.2 MB 00:01 2024-05-09T10:48:05.376 INFO:teuthology.orchestra.run.smithi105.stdout:(31/156): ceph-common-19.0.0-1629.g4f7075d1.el9 2.3 MB/s | 20 MB 00:08 2024-05-09T10:48:05.526 INFO:teuthology.orchestra.run.smithi105.stdout:(32/156): ceph-mgr-modules-core-19.0.0-1629.g4f 1.6 MB/s | 248 kB 00:00 2024-05-09T10:48:05.571 INFO:teuthology.orchestra.run.smithi098.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 180 kB/s | 135 kB 00:00 2024-05-09T10:48:05.593 INFO:teuthology.orchestra.run.smithi105.stdout:(33/156): ceph-mgr-rook-19.0.0-1629.g4f7075d1.e 741 kB/s | 49 kB 00:00 2024-05-09T10:48:05.644 INFO:teuthology.orchestra.run.smithi105.stdout:(34/156): ceph-prometheus-alerts-19.0.0-1629.g4 307 kB/s | 15 kB 00:00 2024-05-09T10:48:05.663 INFO:teuthology.orchestra.run.smithi098.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 7.5 MB/s | 689 kB 00:00 2024-05-09T10:48:05.752 INFO:teuthology.orchestra.run.smithi105.stdout:(35/156): ceph-volume-19.0.0-1629.g4f7075d1.el9 2.4 MB/s | 264 kB 00:00 2024-05-09T10:48:05.763 INFO:teuthology.orchestra.run.smithi098.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 9.2 MB/s | 943 kB 00:00 2024-05-09T10:48:05.830 INFO:teuthology.orchestra.run.smithi098.stdout:(49/156): python3-requests-2.25.1-8.el9.noarch. 156 kB/s | 125 kB 00:00 2024-05-09T10:48:05.855 INFO:teuthology.orchestra.run.smithi098.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-05-09T10:48:05.897 INFO:teuthology.orchestra.run.smithi098.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 323 kB/s | 21 kB 00:00 2024-05-09T10:48:05.911 INFO:teuthology.orchestra.run.smithi105.stdout:(36/156): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-05-09T10:48:05.948 INFO:teuthology.orchestra.run.smithi098.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-05-09T10:48:05.973 INFO:teuthology.orchestra.run.smithi098.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 434 kB/s | 33 kB 00:00 2024-05-09T10:48:05.994 INFO:teuthology.orchestra.run.smithi105.stdout:(37/156): cephfs-top-19.0.0-1629.g4f7075d1.el9. 219 kB/s | 18 kB 00:00 2024-05-09T10:48:06.020 INFO:teuthology.orchestra.run.smithi105.stdout:(38/156): cephadm-19.0.0-1629.g4f7075d1.el9.noa 2.7 MB/s | 748 kB 00:00 2024-05-09T10:48:06.023 INFO:teuthology.orchestra.run.smithi098.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-05-09T10:48:06.065 INFO:teuthology.orchestra.run.smithi098.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-05-09T10:48:06.090 INFO:teuthology.orchestra.run.smithi098.stdout:(56/156): python3-urllib3-1.26.5-5.el9.noarch.r 245 kB/s | 215 kB 00:00 2024-05-09T10:48:06.145 INFO:teuthology.orchestra.run.smithi105.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 535 kB/s | 80 kB 00:00 2024-05-09T10:48:06.166 INFO:teuthology.orchestra.run.smithi098.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-05-09T10:48:06.191 INFO:teuthology.orchestra.run.smithi098.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 450 kB/s | 45 kB 00:00 2024-05-09T10:48:06.216 INFO:teuthology.orchestra.run.smithi098.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-05-09T10:48:06.267 INFO:teuthology.orchestra.run.smithi098.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 511 kB/s | 25 kB 00:00 2024-05-09T10:48:06.300 INFO:teuthology.orchestra.run.smithi098.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-05-09T10:48:06.334 INFO:teuthology.orchestra.run.smithi098.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 4.0 MB/s | 582 kB 00:00 2024-05-09T10:48:06.359 INFO:teuthology.orchestra.run.smithi098.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 515 kB/s | 47 kB 00:00 2024-05-09T10:48:06.410 INFO:teuthology.orchestra.run.smithi098.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-09T10:48:06.435 INFO:teuthology.orchestra.run.smithi098.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 468 kB/s | 35 kB 00:00 2024-05-09T10:48:06.460 INFO:teuthology.orchestra.run.smithi098.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-05-09T10:48:06.594 INFO:teuthology.orchestra.run.smithi098.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-05-09T10:48:06.612 INFO:teuthology.orchestra.run.smithi105.stdout:(40/156): ceph-test-19.0.0-1629.g4f7075d1.el9.x 12 MB/s | 47 MB 00:04 2024-05-09T10:48:06.637 INFO:teuthology.orchestra.run.smithi105.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 397 kB/s | 195 kB 00:00 2024-05-09T10:48:06.662 INFO:teuthology.orchestra.run.smithi105.stdout:(42/156): libgfortran-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 803 kB 00:00 2024-05-09T10:48:06.677 INFO:teuthology.orchestra.run.smithi098.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-05-09T10:48:06.714 INFO:teuthology.orchestra.run.smithi105.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-05-09T10:48:06.744 INFO:teuthology.orchestra.run.smithi098.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-05-09T10:48:06.763 INFO:teuthology.orchestra.run.smithi105.stdout:(44/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-05-09T10:48:06.789 INFO:teuthology.orchestra.run.smithi105.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-05-09T10:48:06.803 INFO:teuthology.orchestra.run.smithi098.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-05-09T10:48:06.814 INFO:teuthology.orchestra.run.smithi105.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-09T10:48:06.836 INFO:teuthology.orchestra.run.smithi098.stdout:(71/156): postgresql-contrib-13.14-1.el9.x86_64 3.7 MB/s | 911 kB 00:00 2024-05-09T10:48:06.847 INFO:teuthology.orchestra.run.smithi105.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 5.8 MB/s | 1.2 MB 00:00 2024-05-09T10:48:06.873 INFO:teuthology.orchestra.run.smithi105.stdout:(48/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-05-09T10:48:06.903 INFO:teuthology.orchestra.run.smithi098.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-05-09T10:48:06.954 INFO:teuthology.orchestra.run.smithi098.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-05-09T10:48:07.015 INFO:teuthology.orchestra.run.smithi105.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 755 kB/s | 107 kB 00:00 2024-05-09T10:48:07.029 INFO:teuthology.orchestra.run.smithi098.stdout:(74/156): postgresql-server-13.14-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-05-09T10:48:07.065 INFO:teuthology.orchestra.run.smithi105.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 431 kB/s | 21 kB 00:00 2024-05-09T10:48:07.096 INFO:teuthology.orchestra.run.smithi098.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 1.7 MB/s | 244 kB 00:00 2024-05-09T10:48:07.130 INFO:teuthology.orchestra.run.smithi098.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-05-09T10:48:07.140 INFO:teuthology.orchestra.run.smithi105.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 943 kB 00:00 2024-05-09T10:48:07.166 INFO:teuthology.orchestra.run.smithi105.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 3.1 MB/s | 313 kB 00:00 2024-05-09T10:48:07.171 INFO:teuthology.orchestra.run.smithi098.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 340 kB/s | 26 kB 00:00 2024-05-09T10:48:07.197 INFO:teuthology.orchestra.run.smithi098.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 286 kB/s | 19 kB 00:00 2024-05-09T10:48:07.199 INFO:teuthology.orchestra.run.smithi105.stdout:(53/156): automake-1.16.2-8.el9.noarch.rpm 1.7 MB/s | 689 kB 00:00 2024-05-09T10:48:07.247 INFO:teuthology.orchestra.run.smithi098.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-05-09T10:48:07.258 INFO:teuthology.orchestra.run.smithi105.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-05-09T10:48:07.272 INFO:teuthology.orchestra.run.smithi098.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-05-09T10:48:07.283 INFO:teuthology.orchestra.run.smithi105.stdout:(55/156): flexiblas-3.0.4-8.el9.x86_64.rpm 229 kB/s | 33 kB 00:00 2024-05-09T10:48:07.328 INFO:teuthology.orchestra.run.smithi105.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 166 kB/s | 11 kB 00:00 2024-05-09T10:48:07.348 INFO:teuthology.orchestra.run.smithi098.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-05-09T10:48:07.384 INFO:teuthology.orchestra.run.smithi105.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 777 kB/s | 45 kB 00:00 2024-05-09T10:48:07.409 INFO:teuthology.orchestra.run.smithi105.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-05-09T10:48:07.415 INFO:teuthology.orchestra.run.smithi098.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-05-09T10:48:07.440 INFO:teuthology.orchestra.run.smithi098.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-05-09T10:48:07.499 INFO:teuthology.orchestra.run.smithi098.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 923 kB/s | 77 kB 00:00 2024-05-09T10:48:07.524 INFO:teuthology.orchestra.run.smithi098.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 504 kB/s | 42 kB 00:00 2024-05-09T10:48:07.551 INFO:teuthology.orchestra.run.smithi105.stdout:(59/156): libtool-2.4.6-45.el9.x86_64.rpm 4.0 MB/s | 582 kB 00:00 2024-05-09T10:48:07.585 INFO:teuthology.orchestra.run.smithi105.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-05-09T10:48:07.591 INFO:teuthology.orchestra.run.smithi098.stdout:(86/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-05-09T10:48:07.610 INFO:teuthology.orchestra.run.smithi105.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-05-09T10:48:07.635 INFO:teuthology.orchestra.run.smithi105.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-05-09T10:48:07.641 INFO:teuthology.orchestra.run.smithi098.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 1.9 MB/s | 269 kB 00:00 2024-05-09T10:48:07.675 INFO:teuthology.orchestra.run.smithi098.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-05-09T10:48:07.694 INFO:teuthology.orchestra.run.smithi105.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-05-09T10:48:07.719 INFO:teuthology.orchestra.run.smithi105.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-05-09T10:48:07.725 INFO:teuthology.orchestra.run.smithi098.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.0 MB/s | 279 kB 00:00 2024-05-09T10:48:07.750 INFO:teuthology.orchestra.run.smithi098.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 468 kB/s | 51 kB 00:00 2024-05-09T10:48:07.776 INFO:teuthology.orchestra.run.smithi098.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 533 kB/s | 54 kB 00:00 2024-05-09T10:48:07.811 INFO:teuthology.orchestra.run.smithi105.stdout:(65/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.6 MB/s | 3.0 MB 00:00 2024-05-09T10:48:07.818 INFO:teuthology.orchestra.run.smithi098.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 622 kB/s | 42 kB 00:00 2024-05-09T10:48:07.836 INFO:teuthology.orchestra.run.smithi105.stdout:(66/156): openblas-0.3.21-2.el9.x86_64.rpm 247 kB/s | 35 kB 00:00 2024-05-09T10:48:07.843 INFO:teuthology.orchestra.run.smithi098.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 658 kB/s | 44 kB 00:00 2024-05-09T10:48:07.893 INFO:teuthology.orchestra.run.smithi098.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-05-09T10:48:07.919 INFO:teuthology.orchestra.run.smithi098.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-05-09T10:48:07.969 INFO:teuthology.orchestra.run.smithi098.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 630 kB/s | 47 kB 00:00 2024-05-09T10:48:08.019 INFO:teuthology.orchestra.run.smithi098.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-05-09T10:48:08.036 INFO:teuthology.orchestra.run.smithi098.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 863 kB/s | 58 kB 00:00 2024-05-09T10:48:08.095 INFO:teuthology.orchestra.run.smithi098.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.2 MB/s | 94 kB 00:00 2024-05-09T10:48:08.112 INFO:teuthology.orchestra.run.smithi098.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 841 kB/s | 64 kB 00:00 2024-05-09T10:48:08.162 INFO:teuthology.orchestra.run.smithi098.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 668 kB/s | 45 kB 00:00 2024-05-09T10:48:08.279 INFO:teuthology.orchestra.run.smithi105.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 3.6 MB/s | 1.6 MB 00:00 2024-05-09T10:48:08.321 INFO:teuthology.orchestra.run.smithi098.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-05-09T10:48:08.512 INFO:teuthology.orchestra.run.smithi105.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 3.8 MB/s | 911 kB 00:00 2024-05-09T10:48:08.571 INFO:teuthology.orchestra.run.smithi105.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.8 MB/s | 4.4 MB 00:00 2024-05-09T10:48:08.596 INFO:teuthology.orchestra.run.smithi105.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-05-09T10:48:08.646 INFO:teuthology.orchestra.run.smithi098.stdout:(103/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-05-09T10:48:08.729 INFO:teuthology.orchestra.run.smithi105.stdout:(71/156): openblas-openmp-0.3.21-2.el9.x86_64.r 4.7 MB/s | 4.7 MB 00:01 2024-05-09T10:48:08.772 INFO:teuthology.orchestra.run.smithi098.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 2.4 MB/s | 309 kB 00:00 2024-05-09T10:48:08.788 INFO:teuthology.orchestra.run.smithi105.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-09T10:48:08.822 INFO:teuthology.orchestra.run.smithi098.stdout:(105/156): lua-devel-5.4.4-4.el9.x86_64.rpm 31 kB/s | 22 kB 00:00 2024-05-09T10:48:08.847 INFO:teuthology.orchestra.run.smithi098.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 258 kB/s | 19 kB 00:00 2024-05-09T10:48:08.897 INFO:teuthology.orchestra.run.smithi105.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 3.4 MB/s | 1.0 MB 00:00 2024-05-09T10:48:08.922 INFO:teuthology.orchestra.run.smithi098.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 347 kB/s | 26 kB 00:00 2024-05-09T10:48:08.981 INFO:teuthology.orchestra.run.smithi105.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-05-09T10:48:09.014 INFO:teuthology.orchestra.run.smithi098.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 529 kB/s | 49 kB 00:00 2024-05-09T10:48:09.064 INFO:teuthology.orchestra.run.smithi105.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-05-09T10:48:09.073 INFO:teuthology.orchestra.run.smithi098.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-05-09T10:48:09.122 INFO:teuthology.orchestra.run.smithi105.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 439 kB/s | 26 kB 00:00 2024-05-09T10:48:09.132 INFO:teuthology.orchestra.run.smithi098.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-05-09T10:48:09.173 INFO:teuthology.orchestra.run.smithi105.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-05-09T10:48:09.241 INFO:teuthology.orchestra.run.smithi098.stdout:(111/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.6 MB/s | 839 kB 00:00 2024-05-09T10:48:09.256 INFO:teuthology.orchestra.run.smithi105.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-05-09T10:48:09.282 INFO:teuthology.orchestra.run.smithi098.stdout:(112/156): protobuf-compiler-3.14.0-13.el9.x86_ 771 kB/s | 863 kB 00:01 2024-05-09T10:48:09.315 INFO:teuthology.orchestra.run.smithi105.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 596 kB/s | 35 kB 00:00 2024-05-09T10:48:09.341 INFO:teuthology.orchestra.run.smithi098.stdout:(113/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.5 MB/s | 4.4 MB 00:00 2024-05-09T10:48:09.371 INFO:teuthology.orchestra.run.smithi098.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-05-09T10:48:09.392 INFO:teuthology.orchestra.run.smithi098.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 639 kB/s | 32 kB 00:00 2024-05-09T10:48:09.417 INFO:teuthology.orchestra.run.smithi098.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-05-09T10:48:09.451 INFO:teuthology.orchestra.run.smithi098.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-05-09T10:48:09.476 INFO:teuthology.orchestra.run.smithi098.stdout:(118/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 225 kB/s | 43 kB 00:00 2024-05-09T10:48:09.499 INFO:teuthology.orchestra.run.smithi105.stdout:(80/156): postgresql-server-13.14-1.el9.x86_64. 6.2 MB/s | 5.8 MB 00:00 2024-05-09T10:48:09.501 INFO:teuthology.orchestra.run.smithi098.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-05-09T10:48:09.535 INFO:teuthology.orchestra.run.smithi098.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-05-09T10:48:09.560 INFO:teuthology.orchestra.run.smithi098.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 420 kB/s | 35 kB 00:00 2024-05-09T10:48:09.594 INFO:teuthology.orchestra.run.smithi098.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-05-09T10:48:09.599 INFO:teuthology.orchestra.run.smithi105.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-05-09T10:48:09.661 INFO:teuthology.orchestra.run.smithi098.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 16 MB/s | 2.0 MB 00:00 2024-05-09T10:48:09.666 INFO:teuthology.orchestra.run.smithi105.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-05-09T10:48:09.686 INFO:teuthology.orchestra.run.smithi098.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-05-09T10:48:09.711 INFO:teuthology.orchestra.run.smithi098.stdout:(125/156): python3-grpcio-tools-1.46.7-10.el9.x 958 kB/s | 144 kB 00:00 2024-05-09T10:48:09.716 INFO:teuthology.orchestra.run.smithi105.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-09T10:48:09.767 INFO:teuthology.orchestra.run.smithi105.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-05-09T10:48:09.816 INFO:teuthology.orchestra.run.smithi098.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 114 kB/s | 18 kB 00:00 2024-05-09T10:48:09.825 INFO:teuthology.orchestra.run.smithi105.stdout:(85/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-05-09T10:48:09.854 INFO:teuthology.orchestra.run.smithi098.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 138 kB/s | 23 kB 00:00 2024-05-09T10:48:09.871 INFO:teuthology.orchestra.run.smithi098.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 122 kB/s | 19 kB 00:00 2024-05-09T10:48:09.884 INFO:teuthology.orchestra.run.smithi105.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-05-09T10:48:09.896 INFO:teuthology.orchestra.run.smithi098.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 248 kB/s | 20 kB 00:00 2024-05-09T10:48:09.930 INFO:teuthology.orchestra.run.smithi098.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 794 kB/s | 46 kB 00:00 2024-05-09T10:48:09.955 INFO:teuthology.orchestra.run.smithi098.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-05-09T10:48:09.968 INFO:teuthology.orchestra.run.smithi105.stdout:(87/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-05-09T10:48:09.989 INFO:teuthology.orchestra.run.smithi098.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-05-09T10:48:10.006 INFO:teuthology.orchestra.run.smithi098.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 766 kB/s | 58 kB 00:00 2024-05-09T10:48:10.031 INFO:teuthology.orchestra.run.smithi098.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-05-09T10:48:10.043 INFO:teuthology.orchestra.run.smithi105.stdout:(88/156): python3-babel-2.9.1-2.el9.noarch.rpm 4.8 MB/s | 6.0 MB 00:01 2024-05-09T10:48:10.048 INFO:teuthology.orchestra.run.smithi098.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 279 kB/s | 16 kB 00:00 2024-05-09T10:48:10.068 INFO:teuthology.orchestra.run.smithi105.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 509 kB/s | 51 kB 00:00 2024-05-09T10:48:10.073 INFO:teuthology.orchestra.run.smithi098.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-05-09T10:48:10.090 INFO:teuthology.orchestra.run.smithi098.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 478 kB/s | 28 kB 00:00 2024-05-09T10:48:10.093 INFO:teuthology.orchestra.run.smithi105.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-09T10:48:10.115 INFO:teuthology.orchestra.run.smithi098.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 458 kB/s | 31 kB 00:00 2024-05-09T10:48:10.141 INFO:teuthology.orchestra.run.smithi098.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-05-09T10:48:10.144 INFO:teuthology.orchestra.run.smithi105.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-05-09T10:48:10.166 INFO:teuthology.orchestra.run.smithi098.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-05-09T10:48:10.191 INFO:teuthology.orchestra.run.smithi098.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 472 kB/s | 36 kB 00:00 2024-05-09T10:48:10.202 INFO:teuthology.orchestra.run.smithi105.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 758 kB/s | 44 kB 00:00 2024-05-09T10:48:10.216 INFO:teuthology.orchestra.run.smithi098.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-05-09T10:48:10.242 INFO:teuthology.orchestra.run.smithi098.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-09T10:48:10.267 INFO:teuthology.orchestra.run.smithi098.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-05-09T10:48:10.286 INFO:teuthology.orchestra.run.smithi105.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-05-09T10:48:10.301 INFO:teuthology.orchestra.run.smithi098.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-05-09T10:48:10.326 INFO:teuthology.orchestra.run.smithi098.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-05-09T10:48:10.345 INFO:teuthology.orchestra.run.smithi105.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 616 kB/s | 36 kB 00:00 2024-05-09T10:48:10.351 INFO:teuthology.orchestra.run.smithi098.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-05-09T10:48:10.393 INFO:teuthology.orchestra.run.smithi098.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-05-09T10:48:10.412 INFO:teuthology.orchestra.run.smithi105.stdout:(95/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.6 MB/s | 5.1 MB 00:01 2024-05-09T10:48:10.437 INFO:teuthology.orchestra.run.smithi105.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 514 kB/s | 47 kB 00:00 2024-05-09T10:48:10.488 INFO:teuthology.orchestra.run.smithi105.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-09T10:48:10.513 INFO:teuthology.orchestra.run.smithi105.stdout:(98/156): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-05-09T10:48:10.538 INFO:teuthology.orchestra.run.smithi105.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-09T10:48:10.563 INFO:teuthology.orchestra.run.smithi105.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-09T10:48:10.589 INFO:teuthology.orchestra.run.smithi105.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-05-09T10:48:10.685 INFO:teuthology.orchestra.run.smithi098.stdout:(149/156): librados2-19.0.0-1629.g4f7075d1.el9. 9.4 MB/s | 3.4 MB 00:00 2024-05-09T10:48:10.911 INFO:teuthology.orchestra.run.smithi098.stdout:(150/156): librbd1-19.0.0-1629.g4f7075d1.el9.x8 5.6 MB/s | 3.1 MB 00:00 2024-05-09T10:48:11.159 INFO:teuthology.orchestra.run.smithi105.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 37 kB/s | 22 kB 00:00 2024-05-09T10:48:11.548 INFO:teuthology.orchestra.run.smithi105.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 901 kB/s | 863 kB 00:00 2024-05-09T10:48:11.552 INFO:teuthology.orchestra.run.smithi098.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 65 kB/s | 42 kB 00:00 2024-05-09T10:48:11.594 INFO:teuthology.orchestra.run.smithi098.stdout:(152/156): NetworkManager-1.47.5-1.el9.x86_64.r 1.9 MB/s | 2.3 MB 00:01 2024-05-09T10:48:11.748 INFO:teuthology.orchestra.run.smithi105.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 931 kB/s | 548 kB 00:00 2024-05-09T10:48:11.845 INFO:teuthology.orchestra.run.smithi098.stdout:(153/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.6 MB/s | 1.8 MB 00:01 2024-05-09T10:48:12.015 INFO:teuthology.orchestra.run.smithi105.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-05-09T10:48:12.048 INFO:teuthology.orchestra.run.smithi105.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 618 kB/s | 309 kB 00:00 2024-05-09T10:48:12.315 INFO:teuthology.orchestra.run.smithi105.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 97 kB/s | 26 kB 00:00 2024-05-09T10:48:12.353 INFO:teuthology.orchestra.run.smithi098.stdout:(154/156): NetworkManager-tui-1.47.5-1.el9.x86_ 310 kB/s | 248 kB 00:00 2024-05-09T10:48:12.637 INFO:teuthology.orchestra.run.smithi098.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 1.3 MB/s | 1.3 MB 00:01 2024-05-09T10:48:12.674 INFO:teuthology.orchestra.run.smithi105.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 135 kB/s | 49 kB 00:00 2024-05-09T10:48:12.954 INFO:teuthology.orchestra.run.smithi098.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.2 MB/s | 2.4 MB 00:01 2024-05-09T10:48:12.958 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-05-09T10:48:12.958 INFO:teuthology.orchestra.run.smithi098.stdout:Total 13 MB/s | 220 MB 00:17 2024-05-09T10:48:13.024 INFO:teuthology.orchestra.run.smithi105.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 192 kB/s | 67 kB 00:00 2024-05-09T10:48:13.291 INFO:teuthology.orchestra.run.smithi105.stdout:(110/156): libarrow-9.0.0-11.el9.x86_64.rpm 3.5 MB/s | 4.4 MB 00:01 2024-05-09T10:48:13.617 INFO:teuthology.orchestra.run.smithi105.stdout:(111/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.6 MB/s | 16 MB 00:03 2024-05-09T10:48:13.642 INFO:teuthology.orchestra.run.smithi105.stdout:(112/156): luarocks-3.9.2-1.el9.noarch.rpm 245 kB/s | 151 kB 00:00 2024-05-09T10:48:13.992 INFO:teuthology.orchestra.run.smithi105.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 124 kB/s | 43 kB 00:00 2024-05-09T10:48:14.034 INFO:teuthology.orchestra.run.smithi105.stdout:(114/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.1 MB/s | 839 kB 00:00 2024-05-09T10:48:14.173 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-05-09T10:48:14.201 INFO:teuthology.orchestra.run.smithi105.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 939 kB/s | 548 kB 00:00 2024-05-09T10:48:14.259 INFO:teuthology.orchestra.run.smithi105.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 120 kB/s | 32 kB 00:00 2024-05-09T10:48:14.293 INFO:teuthology.orchestra.run.smithi105.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 58 kB/s | 15 kB 00:00 2024-05-09T10:48:14.323 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-05-09T10:48:14.323 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-05-09T10:48:14.635 INFO:teuthology.orchestra.run.smithi105.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 396 kB/s | 172 kB 00:00 2024-05-09T10:48:14.802 INFO:teuthology.orchestra.run.smithi105.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 661 kB/s | 358 kB 00:00 2024-05-09T10:48:14.827 INFO:teuthology.orchestra.run.smithi105.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 568 kB/s | 303 kB 00:00 2024-05-09T10:48:14.894 INFO:teuthology.orchestra.run.smithi105.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 136 kB/s | 35 kB 00:00 2024-05-09T10:48:15.328 INFO:teuthology.orchestra.run.smithi105.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 334 kB/s | 144 kB 00:00 2024-05-09T10:48:15.353 INFO:teuthology.orchestra.run.smithi105.stdout:(123/156): python3-google-auth-2.29.0-1.el9.noa 373 kB/s | 206 kB 00:00 2024-05-09T10:48:15.512 INFO:teuthology.orchestra.run.smithi105.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 58 kB/s | 11 kB 00:00 2024-05-09T10:48:15.628 INFO:teuthology.orchestra.run.smithi105.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 64 kB/s | 18 kB 00:00 2024-05-09T10:48:15.754 INFO:teuthology.orchestra.run.smithi105.stdout:(126/156): python3-grpcio-1.46.7-10.el9.x86_64. 2.2 MB/s | 2.0 MB 00:00 2024-05-09T10:48:15.779 INFO:teuthology.orchestra.run.smithi105.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 87 kB/s | 23 kB 00:00 2024-05-09T10:48:15.938 INFO:teuthology.orchestra.run.smithi105.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 63 kB/s | 19 kB 00:00 2024-05-09T10:48:16.021 INFO:teuthology.orchestra.run.smithi105.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 74 kB/s | 20 kB 00:00 2024-05-09T10:48:16.288 INFO:teuthology.orchestra.run.smithi105.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 132 kB/s | 46 kB 00:00 2024-05-09T10:48:16.363 INFO:teuthology.orchestra.run.smithi105.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 231 kB/s | 79 kB 00:00 2024-05-09T10:48:16.440 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-05-09T10:48:16.440 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-05-09T10:48:16.539 INFO:teuthology.orchestra.run.smithi105.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 1.3 MB/s | 1.0 MB 00:00 2024-05-09T10:48:16.639 INFO:teuthology.orchestra.run.smithi105.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 165 kB/s | 58 kB 00:00 2024-05-09T10:48:16.823 INFO:teuthology.orchestra.run.smithi105.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 58 kB/s | 16 kB 00:00 2024-05-09T10:48:16.915 INFO:teuthology.orchestra.run.smithi105.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 494 kB/s | 272 kB 00:00 2024-05-09T10:48:17.065 INFO:teuthology.orchestra.run.smithi105.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 211 kB/s | 90 kB 00:00 2024-05-09T10:48:17.098 INFO:teuthology.orchestra.run.smithi105.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 102 kB/s | 28 kB 00:00 2024-05-09T10:48:17.174 INFO:teuthology.orchestra.run.smithi105.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 119 kB/s | 31 kB 00:00 2024-05-09T10:48:17.441 INFO:teuthology.orchestra.run.smithi105.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 134 kB/s | 36 kB 00:00 2024-05-09T10:48:17.466 INFO:teuthology.orchestra.run.smithi105.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 161 kB/s | 59 kB 00:00 2024-05-09T10:48:17.574 INFO:teuthology.orchestra.run.smithi105.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 370 kB/s | 188 kB 00:00 2024-05-09T10:48:17.791 INFO:teuthology.orchestra.run.smithi105.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 145 kB/s | 51 kB 00:00 2024-05-09T10:48:17.992 INFO:teuthology.orchestra.run.smithi105.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 215 kB/s | 90 kB 00:00 2024-05-09T10:48:18.017 INFO:teuthology.orchestra.run.smithi105.stdout:(144/156): python3-webob-1.8.7-6.el9.noarch.rpm 417 kB/s | 230 kB 00:00 2024-05-09T10:48:18.267 INFO:teuthology.orchestra.run.smithi105.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 73 kB/s | 20 kB 00:00 2024-05-09T10:48:18.401 INFO:teuthology.orchestra.run.smithi105.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 701 kB/s | 427 kB 00:00 2024-05-09T10:48:18.526 INFO:teuthology.orchestra.run.smithi105.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 376 kB/s | 192 kB 00:00 2024-05-09T10:48:18.835 INFO:teuthology.orchestra.run.smithi105.stdout:(148/156): librados2-19.0.0-1629.g4f7075d1.el9. 7.8 MB/s | 3.4 MB 00:00 2024-05-09T10:48:18.952 INFO:teuthology.orchestra.run.smithi105.stdout:(149/156): librbd1-19.0.0-1629.g4f7075d1.el9.x8 7.4 MB/s | 3.1 MB 00:00 2024-05-09T10:48:19.035 INFO:teuthology.orchestra.run.smithi105.stdout:(150/156): NetworkManager-1.47.5-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-05-09T10:48:19.069 INFO:teuthology.orchestra.run.smithi105.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 1.2 MB/s | 42 kB 00:00 2024-05-09T10:48:19.119 INFO:teuthology.orchestra.run.smithi105.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 4.9 MB/s | 248 kB 00:00 2024-05-09T10:48:19.153 INFO:teuthology.orchestra.run.smithi105.stdout:(153/156): thrift-0.15.0-2.el9.x86_64.rpm 1.8 MB/s | 1.6 MB 00:00 2024-05-09T10:48:19.186 INFO:teuthology.orchestra.run.smithi105.stdout:(154/156): NetworkManager-libnm-1.47.5-1.el9.x8 7.8 MB/s | 1.8 MB 00:00 2024-05-09T10:48:19.220 INFO:teuthology.orchestra.run.smithi105.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 13 MB/s | 1.3 MB 00:00 2024-05-09T10:48:19.370 INFO:teuthology.orchestra.run.smithi105.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-05-09T10:48:19.373 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-05-09T10:48:19.373 INFO:teuthology.orchestra.run.smithi105.stdout:Total 9.7 MB/s | 220 MB 00:22 2024-05-09T10:48:20.585 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2024-05-09T10:48:20.734 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2024-05-09T10:48:20.734 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2024-05-09T10:48:22.526 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-05-09T10:48:22.609 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T10:48:22.651 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T10:48:22.718 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T10:48:22.751 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T10:48:22.776 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T10:48:22.845 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T10:48:22.851 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2024-05-09T10:48:22.852 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2024-05-09T10:48:23.172 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T10:48:23.286 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librados2-2:19.0.0-1629.g4f7075d1.el9.x86_64 8/164 2024-05-09T10:48:23.424 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-2:19.0.0-1629.g4f7075d1.el9.x86_64 8/164 2024-05-09T10:48:23.454 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs2-2:19.0.0-1629.g4f7075d1.el9.x86_64 9/164 2024-05-09T10:48:23.545 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephfs2-2:19.0.0-1629.g4f7075d1.el9.x86_64 9/164 2024-05-09T10:48:23.575 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rados-2:19.0.0-1629.g4f7075d1.el9.x86_64 10/164 2024-05-09T10:48:23.756 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T10:48:23.818 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T10:48:23.841 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephsqlite-2:19.0.0-1629.g4f7075d1.el9.x86_64 13/164 2024-05-09T10:48:23.928 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1629.g4f7075d1.el9.x86_64 13/164 2024-05-09T10:48:23.951 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libradosstriper1-2:19.0.0-1629.g4f7075d1.el9.x86 14/164 2024-05-09T10:48:23.997 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1629.g4f7075d1.el9.x86 14/164 2024-05-09T10:48:24.027 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T10:48:24.047 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T10:48:24.195 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T10:48:24.375 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T10:48:24.525 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T10:48:24.585 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T10:48:24.702 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T10:48:24.770 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T10:48:24.993 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T10:48:25.104 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T10:48:25.198 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T10:48:25.234 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-09T10:48:25.269 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-09T10:48:25.343 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-09T10:48:25.377 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-09T10:48:25.420 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-09T10:48:25.453 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-09T10:48:25.480 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-09T10:48:25.550 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-09T10:48:25.604 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-09T10:48:25.634 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-09T10:48:25.677 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-09T10:48:25.753 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-09T10:48:25.796 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-common-2:19.0.0-1629.g4f7075d1.el9. 33/164 2024-05-09T10:48:25.840 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-argparse-2:19.0.0-1629.g4f7075d1.el 34/164 2024-05-09T10:48:25.868 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cephfs-2:19.0.0-1629.g4f7075d1.el9.x86_6 35/164 2024-05-09T10:48:25.938 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-09T10:48:25.983 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-09T10:48:26.057 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-09T10:48:26.198 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-09T10:48:26.253 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-09T10:48:26.773 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-09T10:48:26.857 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-09T10:48:26.886 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-09T10:48:26.928 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-09T10:48:26.966 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librados-devel-2:19.0.0-1629.g4f7075d1.el9.x86_6 45/164 2024-05-09T10:48:27.009 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-09T10:48:27.178 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-09T10:48:27.237 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-09T10:48:27.299 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-09T10:48:27.374 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T10:48:27.412 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T10:48:27.449 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T10:48:27.495 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T10:48:27.532 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T10:48:27.575 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T10:48:27.617 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T10:48:27.753 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T10:48:27.806 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T10:48:27.836 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T10:48:27.878 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T10:48:27.973 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T10:48:28.011 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T10:48:28.053 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T10:48:28.093 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T10:48:28.118 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T10:48:28.440 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T10:48:28.572 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T10:48:28.618 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T10:48:28.646 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T10:48:28.684 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T10:48:28.719 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T10:48:28.750 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T10:48:28.780 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T10:48:28.815 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T10:48:28.848 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T10:48:28.890 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T10:48:28.924 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T10:48:29.011 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T10:48:29.055 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T10:48:29.081 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2024-05-09T10:48:29.086 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T10:48:29.133 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T10:48:29.167 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T10:48:29.212 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T10:48:29.233 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T10:48:29.279 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T10:48:29.288 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T10:48:29.317 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T10:48:29.342 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T10:48:29.397 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T10:48:29.719 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T10:48:29.835 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librados2-2:19.0.0-1629.g4f7075d1.el9.x86_64 8/164 2024-05-09T10:48:29.940 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:19.0.0-1629.g4f7075d1.el9.x86_64 8/164 2024-05-09T10:48:29.970 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs2-2:19.0.0-1629.g4f7075d1.el9.x86_64 9/164 2024-05-09T10:48:30.008 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T10:48:30.060 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T10:48:30.078 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephfs2-2:19.0.0-1629.g4f7075d1.el9.x86_64 9/164 2024-05-09T10:48:30.094 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T10:48:30.112 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rados-2:19.0.0-1629.g4f7075d1.el9.x86_64 10/164 2024-05-09T10:48:30.195 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T10:48:30.260 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T10:48:30.289 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T10:48:30.351 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T10:48:30.355 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T10:48:30.382 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephsqlite-2:19.0.0-1629.g4f7075d1.el9.x86_64 13/164 2024-05-09T10:48:30.387 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T10:48:30.485 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1629.g4f7075d1.el9.x86_64 13/164 2024-05-09T10:48:30.508 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libradosstriper1-2:19.0.0-1629.g4f7075d1.el9.x86 14/164 2024-05-09T10:48:30.563 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T10:48:30.571 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1629.g4f7075d1.el9.x86 14/164 2024-05-09T10:48:30.594 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T10:48:30.601 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T10:48:30.623 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T10:48:30.672 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T10:48:30.810 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T10:48:30.997 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T10:48:31.140 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T10:48:31.192 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T10:48:31.299 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T10:48:31.334 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T10:48:31.368 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T10:48:31.451 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T10:48:31.582 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T10:48:31.701 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T10:48:31.794 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T10:48:31.836 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-09T10:48:31.875 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-09T10:48:31.940 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-09T10:48:31.984 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-09T10:48:32.026 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-09T10:48:32.060 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-09T10:48:32.087 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-09T10:48:32.155 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-09T10:48:32.208 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-09T10:48:32.239 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-09T10:48:32.283 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-09T10:48:32.358 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-09T10:48:32.394 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-common-2:19.0.0-1629.g4f7075d1.el9. 33/164 2024-05-09T10:48:32.437 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-argparse-2:19.0.0-1629.g4f7075d1.el 34/164 2024-05-09T10:48:32.466 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cephfs-2:19.0.0-1629.g4f7075d1.el9.x86_6 35/164 2024-05-09T10:48:32.536 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-09T10:48:32.580 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-09T10:48:32.655 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-09T10:48:32.722 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T10:48:32.799 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T10:48:32.812 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-09T10:48:32.826 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T10:48:32.857 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-09T10:48:32.862 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T10:48:33.175 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T10:48:33.225 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librbd1-2:19.0.0-1629.g4f7075d1.el9.x86_64 101/164 2024-05-09T10:48:33.298 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-2:19.0.0-1629.g4f7075d1.el9.x86_64 101/164 2024-05-09T10:48:33.330 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rbd-2:19.0.0-1629.g4f7075d1.el9.x86_64 102/164 2024-05-09T10:48:33.366 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T10:48:33.388 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-09T10:48:33.470 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-09T10:48:33.499 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-09T10:48:33.542 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-09T10:48:33.580 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librados-devel-2:19.0.0-1629.g4f7075d1.el9.x86_6 45/164 2024-05-09T10:48:33.622 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-09T10:48:33.774 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-09T10:48:33.819 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T10:48:33.825 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-09T10:48:33.845 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librgw2-2:19.0.0-1629.g4f7075d1.el9.x86_64 105/164 2024-05-09T10:48:33.879 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-09T10:48:33.899 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librgw2-2:19.0.0-1629.g4f7075d1.el9.x86_64 105/164 2024-05-09T10:48:33.921 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rgw-2:19.0.0-1629.g4f7075d1.el9.x86_64 106/164 2024-05-09T10:48:33.952 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T10:48:33.983 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T10:48:34.020 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T10:48:34.066 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T10:48:34.103 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T10:48:34.146 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T10:48:34.188 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T10:48:34.322 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T10:48:34.377 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T10:48:34.407 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T10:48:34.448 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T10:48:34.543 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T10:48:34.582 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T10:48:34.627 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T10:48:34.656 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T10:48:34.681 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T10:48:34.766 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T10:48:34.889 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T10:48:34.921 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T10:48:34.981 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T10:48:35.009 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T10:48:35.048 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T10:48:35.088 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T10:48:35.117 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T10:48:35.161 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T10:48:35.194 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T10:48:35.235 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T10:48:35.270 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T10:48:35.356 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T10:48:35.400 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T10:48:35.439 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T10:48:35.486 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T10:48:35.583 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T10:48:35.635 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T10:48:36.076 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:19.0.0-1629.g4f7075d1.el9.x86_64 107/164 2024-05-09T10:48:36.118 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-common-2:19.0.0-1629.g4f7075d1.el9.x86_64 107/164 2024-05-09T10:48:36.350 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T10:48:36.555 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T10:48:36.588 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T10:48:36.660 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:19.0.0-1629.g4f7075d1.el9.x86_64 107/164 2024-05-09T10:48:36.687 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T10:48:36.697 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-base-2:19.0.0-1629.g4f7075d1.el9.x86_64 108/164 2024-05-09T10:48:36.738 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T10:48:36.772 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-base-2:19.0.0-1629.g4f7075d1.el9.x86_64 108/164 2024-05-09T10:48:36.772 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T10:48:36.772 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T10:48:36.772 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:48:36.815 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-selinux-2:19.0.0-1629.g4f7075d1.el9.x86_64 109/164 2024-05-09T10:48:36.824 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T10:48:36.874 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T10:48:37.052 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T10:48:37.080 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T10:48:37.165 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T10:48:37.810 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T10:48:37.918 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T10:48:39.153 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T10:48:39.234 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T10:48:39.261 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T10:48:39.294 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T10:48:39.604 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T10:48:39.649 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librbd1-2:19.0.0-1629.g4f7075d1.el9.x86_64 101/164 2024-05-09T10:48:39.739 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:19.0.0-1629.g4f7075d1.el9.x86_64 101/164 2024-05-09T10:48:39.770 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rbd-2:19.0.0-1629.g4f7075d1.el9.x86_64 102/164 2024-05-09T10:48:39.806 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T10:48:40.268 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T10:48:40.294 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librgw2-2:19.0.0-1629.g4f7075d1.el9.x86_64 105/164 2024-05-09T10:48:40.365 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librgw2-2:19.0.0-1629.g4f7075d1.el9.x86_64 105/164 2024-05-09T10:48:40.395 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rgw-2:19.0.0-1629.g4f7075d1.el9.x86_64 106/164 2024-05-09T10:48:42.626 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.0.0-1629.g4f7075d1.el9.x86_64 107/164 2024-05-09T10:48:42.658 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-common-2:19.0.0-1629.g4f7075d1.el9.x86_64 107/164 2024-05-09T10:48:43.220 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.0.0-1629.g4f7075d1.el9.x86_64 107/164 2024-05-09T10:48:43.253 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-base-2:19.0.0-1629.g4f7075d1.el9.x86_64 108/164 2024-05-09T10:48:43.345 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-base-2:19.0.0-1629.g4f7075d1.el9.x86_64 108/164 2024-05-09T10:48:43.345 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T10:48:43.345 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T10:48:43.345 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:48:43.379 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-selinux-2:19.0.0-1629.g4f7075d1.el9.x86_64 109/164 2024-05-09T10:48:48.668 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1629.g4f7075d1.el9.x86_64 109/164 2024-05-09T10:48:48.668 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /sys 2024-05-09T10:48:48.668 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /proc 2024-05-09T10:48:48.668 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /mnt 2024-05-09T10:48:48.668 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /var/tmp 2024-05-09T10:48:48.668 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /home 2024-05-09T10:48:48.668 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /root 2024-05-09T10:48:48.668 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /tmp 2024-05-09T10:48:48.668 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:48:48.831 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-volume-2:19.0.0-1629.g4f7075d1.el9.noarch 110/164 2024-05-09T10:48:48.848 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-volume-2:19.0.0-1629.g4f7075d1.el9.noarch 110/164 2024-05-09T10:48:48.848 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T10:48:48.848 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T10:48:48.848 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:48:50.361 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-osd-2:19.0.0-1629.g4f7075d1.el9.x86_64 111/164 2024-05-09T10:48:50.388 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-osd-2:19.0.0-1629.g4f7075d1.el9.x86_64 111/164 2024-05-09T10:48:50.388 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T10:48:50.388 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T10:48:50.388 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T10:48:50.388 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T10:48:50.388 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:48:50.594 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mds-2:19.0.0-1629.g4f7075d1.el9.x86_64 112/164 2024-05-09T10:48:50.618 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mds-2:19.0.0-1629.g4f7075d1.el9.x86_64 112/164 2024-05-09T10:48:50.618 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T10:48:50.618 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T10:48:50.618 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T10:48:50.618 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T10:48:50.618 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:48:51.053 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mon-2:19.0.0-1629.g4f7075d1.el9.x86_64 113/164 2024-05-09T10:48:51.077 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mon-2:19.0.0-1629.g4f7075d1.el9.x86_64 113/164 2024-05-09T10:48:51.077 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T10:48:51.077 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T10:48:51.077 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T10:48:51.077 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T10:48:51.077 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:48:51.175 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T10:48:51.223 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T10:48:51.279 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T10:48:51.463 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T10:48:51.513 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T10:48:51.598 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T10:48:51.688 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T10:48:51.825 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T10:48:51.874 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T10:48:51.912 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T10:48:51.940 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T10:48:52.335 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephadm-2:19.0.0-1629.g4f7075d1.el9.noarch 125/164 2024-05-09T10:48:52.363 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephadm-2:19.0.0-1629.g4f7075d1.el9.noarch 125/164 2024-05-09T10:48:52.395 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1629.g4f7075d1.e 126/164 2024-05-09T10:48:52.457 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1629.g4f7075d1. 127/164 2024-05-09T10:48:52.487 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1629.g4f7075d1.el9.noa 128/164 2024-05-09T10:48:52.715 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1629.g4f7075d1.el9.noa 128/164 2024-05-09T10:48:52.746 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1629.g4f7075d1.el9.n 129/164 2024-05-09T10:48:53.717 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1629.g4f7075d1.el9.n 129/164 2024-05-09T10:48:53.746 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1629.g4f7 130/164 2024-05-09T10:48:53.848 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1629.g4f7 130/164 2024-05-09T10:48:54.027 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1629.g4f7075d1.el 131/164 2024-05-09T10:48:54.065 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-2:19.0.0-1629.g4f7075d1.el9.x86_64 132/164 2024-05-09T10:48:54.089 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1629.g4f7075d1.el9.x86_64 132/164 2024-05-09T10:48:54.089 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T10:48:54.089 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T10:48:54.090 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T10:48:54.090 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T10:48:54.090 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:48:54.141 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-rook-2:19.0.0-1629.g4f7075d1.el9.noarch 133/164 2024-05-09T10:48:54.153 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1629.g4f7075d1.el9.noarch 133/164 2024-05-09T10:48:54.252 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-2:19.0.0-1629.g4f7075d1.el9.x86_64 134/164 2024-05-09T10:48:54.308 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-fuse-2:19.0.0-1629.g4f7075d1.el9.x86_64 135/164 2024-05-09T10:48:55.072 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1629.g4f7075d1.el9.x86_64 109/164 2024-05-09T10:48:55.072 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /sys 2024-05-09T10:48:55.072 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /proc 2024-05-09T10:48:55.073 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /mnt 2024-05-09T10:48:55.073 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /var/tmp 2024-05-09T10:48:55.073 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /home 2024-05-09T10:48:55.073 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /root 2024-05-09T10:48:55.073 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /tmp 2024-05-09T10:48:55.073 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:48:55.171 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T10:48:55.202 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-radosgw-2:19.0.0-1629.g4f7075d1.el9.x86_64 137/164 2024-05-09T10:48:55.226 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1629.g4f7075d1.el9.x86_64 137/164 2024-05-09T10:48:55.226 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T10:48:55.226 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T10:48:55.226 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T10:48:55.227 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T10:48:55.227 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:48:55.237 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-volume-2:19.0.0-1629.g4f7075d1.el9.noarch 110/164 2024-05-09T10:48:55.253 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-volume-2:19.0.0-1629.g4f7075d1.el9.noarch 110/164 2024-05-09T10:48:55.254 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T10:48:55.254 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T10:48:55.254 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:48:55.269 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1629.g4f707 138/164 2024-05-09T10:48:55.292 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1629.g4f707 138/164 2024-05-09T10:48:55.292 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T10:48:55.292 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T10:48:55.292 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:48:55.337 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-mirror-2:19.0.0-1629.g4f7075d1.el9.x86_64 139/164 2024-05-09T10:48:55.360 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1629.g4f7075d1.el9.x86_64 139/164 2024-05-09T10:48:55.360 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T10:48:55.360 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T10:48:55.360 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T10:48:55.361 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T10:48:55.361 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:48:55.662 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-mirror-2:19.0.0-1629.g4f7075d1.el9.x86_64 140/164 2024-05-09T10:48:55.686 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1629.g4f7075d1.el9.x86_64 140/164 2024-05-09T10:48:55.686 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T10:48:55.686 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T10:48:55.686 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T10:48:55.686 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T10:48:55.686 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:48:56.766 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-osd-2:19.0.0-1629.g4f7075d1.el9.x86_64 111/164 2024-05-09T10:48:56.792 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-osd-2:19.0.0-1629.g4f7075d1.el9.x86_64 111/164 2024-05-09T10:48:56.792 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T10:48:56.792 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T10:48:56.792 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T10:48:56.793 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T10:48:56.793 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:48:56.999 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mds-2:19.0.0-1629.g4f7075d1.el9.x86_64 112/164 2024-05-09T10:48:57.026 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mds-2:19.0.0-1629.g4f7075d1.el9.x86_64 112/164 2024-05-09T10:48:57.027 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T10:48:57.027 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T10:48:57.027 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T10:48:57.027 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T10:48:57.027 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:48:57.475 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mon-2:19.0.0-1629.g4f7075d1.el9.x86_64 113/164 2024-05-09T10:48:57.499 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mon-2:19.0.0-1629.g4f7075d1.el9.x86_64 113/164 2024-05-09T10:48:57.499 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T10:48:57.499 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T10:48:57.499 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T10:48:57.499 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T10:48:57.499 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:48:57.605 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T10:48:57.653 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T10:48:57.709 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T10:48:57.895 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T10:48:57.942 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T10:48:58.019 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T10:48:58.108 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T10:48:58.249 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T10:48:58.302 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T10:48:58.333 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T10:48:58.360 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T10:48:58.738 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephadm-2:19.0.0-1629.g4f7075d1.el9.noarch 125/164 2024-05-09T10:48:58.784 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephadm-2:19.0.0-1629.g4f7075d1.el9.noarch 125/164 2024-05-09T10:48:58.815 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1629.g4f7075d1.e 126/164 2024-05-09T10:48:58.875 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1629.g4f7075d1. 127/164 2024-05-09T10:48:58.899 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1629.g4f7075d1.el9.noa 128/164 2024-05-09T10:48:59.125 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1629.g4f7075d1.el9.noa 128/164 2024-05-09T10:48:59.158 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1629.g4f7075d1.el9.n 129/164 2024-05-09T10:49:00.125 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1629.g4f7075d1.el9.n 129/164 2024-05-09T10:49:00.134 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-test-2:19.0.0-1629.g4f7075d1.el9.x86_64 141/164 2024-05-09T10:49:00.150 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1629.g4f7 130/164 2024-05-09T10:49:00.174 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-fuse-2:19.0.0-1629.g4f7075d1.el9.x86_64 142/164 2024-05-09T10:49:00.215 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-nbd-2:19.0.0-1629.g4f7075d1.el9.x86_64 143/164 2024-05-09T10:49:00.251 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1629.g4f7 130/164 2024-05-09T10:49:00.256 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T10:49:00.283 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T10:49:00.414 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1629.g4f7075d1.el 131/164 2024-05-09T10:49:00.452 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-2:19.0.0-1629.g4f7075d1.el9.x86_64 132/164 2024-05-09T10:49:00.476 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1629.g4f7075d1.el9.x86_64 132/164 2024-05-09T10:49:00.476 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T10:49:00.476 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T10:49:00.476 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T10:49:00.476 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T10:49:00.476 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:00.482 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-shell-2:19.0.0-1629.g4f7075d1.el9.x86_64 146/164 2024-05-09T10:49:00.519 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-rook-2:19.0.0-1629.g4f7075d1.el9.noarch 133/164 2024-05-09T10:49:00.530 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T10:49:00.531 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1629.g4f7075d1.el9.noarch 133/164 2024-05-09T10:49:00.560 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs-devel-2:19.0.0-1629.g4f7075d1.el9.x86_ 148/164 2024-05-09T10:49:00.589 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T10:49:00.627 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T10:49:00.628 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-2:19.0.0-1629.g4f7075d1.el9.x86_64 134/164 2024-05-09T10:49:00.682 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-fuse-2:19.0.0-1629.g4f7075d1.el9.x86_64 135/164 2024-05-09T10:49:01.246 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T10:49:01.284 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T10:49:01.305 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T10:49:01.539 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T10:49:01.580 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-radosgw-2:19.0.0-1629.g4f7075d1.el9.x86_64 137/164 2024-05-09T10:49:01.604 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1629.g4f7075d1.el9.x86_64 137/164 2024-05-09T10:49:01.604 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T10:49:01.604 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T10:49:01.604 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T10:49:01.604 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T10:49:01.604 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:01.656 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1629.g4f707 138/164 2024-05-09T10:49:01.679 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1629.g4f707 138/164 2024-05-09T10:49:01.679 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T10:49:01.679 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T10:49:01.679 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:01.724 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-mirror-2:19.0.0-1629.g4f7075d1.el9.x86_64 139/164 2024-05-09T10:49:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1629.g4f7075d1.el9.x86_64 139/164 2024-05-09T10:49:01.747 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T10:49:01.747 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T10:49:01.747 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T10:49:01.747 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T10:49:01.747 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:02.049 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-mirror-2:19.0.0-1629.g4f7075d1.el9.x86_64 140/164 2024-05-09T10:49:02.072 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1629.g4f7075d1.el9.x86_64 140/164 2024-05-09T10:49:02.072 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T10:49:02.072 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T10:49:02.073 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T10:49:02.073 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T10:49:02.073 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:02.138 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-top-2:19.0.0-1629.g4f7075d1.el9.noarch 152/164 2024-05-09T10:49:02.223 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T10:49:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T10:49:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T10:49:02.385 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T10:49:02.411 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T10:49:02.411 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T10:49:02.487 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T10:49:02.528 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T10:49:02.528 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T10:49:02.596 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T10:49:02.621 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T10:49:02.627 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T10:49:02.627 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T10:49:02.807 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T10:49:02.807 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T10:49:02.891 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T10:49:02.957 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T10:49:06.494 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-test-2:19.0.0-1629.g4f7075d1.el9.x86_64 141/164 2024-05-09T10:49:06.535 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-fuse-2:19.0.0-1629.g4f7075d1.el9.x86_64 142/164 2024-05-09T10:49:06.575 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-nbd-2:19.0.0-1629.g4f7075d1.el9.x86_64 143/164 2024-05-09T10:49:06.608 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T10:49:06.652 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T10:49:06.859 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-shell-2:19.0.0-1629.g4f7075d1.el9.x86_64 146/164 2024-05-09T10:49:06.915 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T10:49:06.946 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs-devel-2:19.0.0-1629.g4f7075d1.el9.x86_ 148/164 2024-05-09T10:49:06.975 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T10:49:07.021 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T10:49:07.629 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T10:49:07.670 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T10:49:07.690 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T10:49:08.007 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T10:49:08.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-2:19.0.0-1629.g4f7075d1.el9.x86_64 1/164 2024-05-09T10:49:08.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-base-2:19.0.0-1629.g4f7075d1.el9.x86_64 2/164 2024-05-09T10:49:08.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-common-2:19.0.0-1629.g4f7075d1.el9.x86_64 3/164 2024-05-09T10:49:08.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-fuse-2:19.0.0-1629.g4f7075d1.el9.x86_64 4/164 2024-05-09T10:49:08.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1629.g4f707 5/164 2024-05-09T10:49:08.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mds-2:19.0.0-1629.g4f7075d1.el9.x86_64 6/164 2024-05-09T10:49:08.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-2:19.0.0-1629.g4f7075d1.el9.x86_64 7/164 2024-05-09T10:49:08.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mon-2:19.0.0-1629.g4f7075d1.el9.x86_64 8/164 2024-05-09T10:49:08.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-osd-2:19.0.0-1629.g4f7075d1.el9.x86_64 9/164 2024-05-09T10:49:08.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-radosgw-2:19.0.0-1629.g4f7075d1.el9.x86_64 10/164 2024-05-09T10:49:08.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-selinux-2:19.0.0-1629.g4f7075d1.el9.x86_64 11/164 2024-05-09T10:49:08.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-test-2:19.0.0-1629.g4f7075d1.el9.x86_64 12/164 2024-05-09T10:49:08.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-mirror-2:19.0.0-1629.g4f7075d1.el9.x86_64 13/164 2024-05-09T10:49:08.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-shell-2:19.0.0-1629.g4f7075d1.el9.x86_64 14/164 2024-05-09T10:49:08.008 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs-devel-2:19.0.0-1629.g4f7075d1.el9.x86_ 15/164 2024-05-09T10:49:08.009 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs2-2:19.0.0-1629.g4f7075d1.el9.x86_64 16/164 2024-05-09T10:49:08.009 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephsqlite-2:19.0.0-1629.g4f7075d1.el9.x86_64 17/164 2024-05-09T10:49:08.009 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados-devel-2:19.0.0-1629.g4f7075d1.el9.x86_6 18/164 2024-05-09T10:49:08.009 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libradosstriper1-2:19.0.0-1629.g4f7075d1.el9.x86 19/164 2024-05-09T10:49:08.009 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librgw2-2:19.0.0-1629.g4f7075d1.el9.x86_64 20/164 2024-05-09T10:49:08.009 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1629.g4f7075d1.el 21/164 2024-05-09T10:49:08.009 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-common-2:19.0.0-1629.g4f7075d1.el9. 22/164 2024-05-09T10:49:08.009 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cephfs-2:19.0.0-1629.g4f7075d1.el9.x86_6 23/164 2024-05-09T10:49:08.009 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rados-2:19.0.0-1629.g4f7075d1.el9.x86_64 24/164 2024-05-09T10:49:08.009 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rbd-2:19.0.0-1629.g4f7075d1.el9.x86_64 25/164 2024-05-09T10:49:08.009 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rgw-2:19.0.0-1629.g4f7075d1.el9.x86_64 26/164 2024-05-09T10:49:08.009 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-fuse-2:19.0.0-1629.g4f7075d1.el9.x86_64 27/164 2024-05-09T10:49:08.009 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-mirror-2:19.0.0-1629.g4f7075d1.el9.x86_64 28/164 2024-05-09T10:49:08.009 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-nbd-2:19.0.0-1629.g4f7075d1.el9.x86_64 29/164 2024-05-09T10:49:08.009 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1629.g4f7075d1. 30/164 2024-05-09T10:49:08.010 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1629.g4f7075d1.el9.noa 31/164 2024-05-09T10:49:08.025 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1629.g4f7075d1.el9.n 32/164 2024-05-09T10:49:08.025 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1629.g4f7 33/164 2024-05-09T10:49:08.026 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1629.g4f7075d1.el 34/164 2024-05-09T10:49:08.026 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1629.g4f7075d1.el9.noarch 35/164 2024-05-09T10:49:08.026 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1629.g4f7075d1.e 36/164 2024-05-09T10:49:08.026 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-volume-2:19.0.0-1629.g4f7075d1.el9.noarch 37/164 2024-05-09T10:49:08.026 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephadm-2:19.0.0-1629.g4f7075d1.el9.noarch 38/164 2024-05-09T10:49:08.026 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-top-2:19.0.0-1629.g4f7075d1.el9.noarch 39/164 2024-05-09T10:49:08.027 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T10:49:08.027 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T10:49:08.027 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T10:49:08.027 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T10:49:08.027 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T10:49:08.027 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T10:49:08.028 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T10:49:08.028 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T10:49:08.028 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T10:49:08.028 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T10:49:08.028 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T10:49:08.028 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T10:49:08.029 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T10:49:08.029 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T10:49:08.029 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T10:49:08.029 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T10:49:08.029 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T10:49:08.029 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T10:49:08.029 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T10:49:08.030 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T10:49:08.030 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T10:49:08.030 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T10:49:08.030 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T10:49:08.030 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T10:49:08.031 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T10:49:08.031 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T10:49:08.031 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T10:49:08.031 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T10:49:08.031 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T10:49:08.031 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T10:49:08.032 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T10:49:08.032 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T10:49:08.032 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T10:49:08.032 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T10:49:08.032 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T10:49:08.033 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T10:49:08.033 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T10:49:08.033 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T10:49:08.033 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T10:49:08.033 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T10:49:08.033 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T10:49:08.034 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T10:49:08.034 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T10:49:08.034 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T10:49:08.034 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T10:49:08.034 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T10:49:08.034 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T10:49:08.035 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T10:49:08.035 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T10:49:08.035 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T10:49:08.035 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T10:49:08.035 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T10:49:08.035 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T10:49:08.036 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T10:49:08.036 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T10:49:08.036 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T10:49:08.036 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T10:49:08.036 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T10:49:08.036 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T10:49:08.036 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T10:49:08.037 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T10:49:08.037 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T10:49:08.037 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T10:49:08.037 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T10:49:08.037 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T10:49:08.037 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T10:49:08.038 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T10:49:08.038 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T10:49:08.038 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T10:49:08.038 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T10:49:08.038 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T10:49:08.039 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T10:49:08.039 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T10:49:08.039 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T10:49:08.039 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T10:49:08.039 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T10:49:08.039 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T10:49:08.039 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T10:49:08.040 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T10:49:08.040 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T10:49:08.040 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T10:49:08.040 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T10:49:08.040 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T10:49:08.040 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T10:49:08.041 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T10:49:08.041 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T10:49:08.041 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T10:49:08.041 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T10:49:08.041 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T10:49:08.042 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T10:49:08.042 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T10:49:08.042 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T10:49:08.042 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T10:49:08.042 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T10:49:08.042 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T10:49:08.042 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T10:49:08.042 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T10:49:08.043 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T10:49:08.043 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T10:49:08.043 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T10:49:08.043 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T10:49:08.043 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T10:49:08.043 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T10:49:08.043 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T10:49:08.043 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T10:49:08.044 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T10:49:08.044 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T10:49:08.044 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T10:49:08.044 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T10:49:08.044 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-2:19.0.0-1629.g4f7075d1.el9.x86_64 149/164 2024-05-09T10:49:08.044 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T10:49:08.044 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-2:19.0.0-1629.g4f7075d1.el9.x86_64 151/164 2024-05-09T10:49:08.044 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T10:49:08.044 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T10:49:08.045 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T10:49:08.045 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T10:49:08.045 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T10:49:08.045 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T10:49:08.045 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T10:49:08.045 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T10:49:08.045 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T10:49:08.045 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T10:49:08.045 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T10:49:08.046 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T10:49:08.513 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-top-2:19.0.0-1629.g4f7075d1.el9.noarch 152/164 2024-05-09T10:49:08.599 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T10:49:08.663 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T10:49:08.759 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T10:49:08.794 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T10:49:08.820 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T10:49:08.820 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T10:49:08.906 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T10:49:08.962 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T10:49:08.962 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T10:49:09.030 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T10:49:09.055 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T10:49:09.061 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T10:49:09.061 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T10:49:09.262 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T10:49:09.262 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T10:49:09.350 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T10:49:09.414 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T10:49:10.868 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T10:49:10.868 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:10.868 INFO:teuthology.orchestra.run.smithi098.stdout:Upgraded: 2024-05-09T10:49:10.868 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T10:49:10.868 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T10:49:10.868 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T10:49:10.868 INFO:teuthology.orchestra.run.smithi098.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T10:49:10.868 INFO:teuthology.orchestra.run.smithi098.stdout: librados2-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.868 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.868 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T10:49:10.869 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T10:49:10.869 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-05-09T10:49:10.869 INFO:teuthology.orchestra.run.smithi098.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T10:49:10.869 INFO:teuthology.orchestra.run.smithi098.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T10:49:10.869 INFO:teuthology.orchestra.run.smithi098.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T10:49:10.869 INFO:teuthology.orchestra.run.smithi098.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T10:49:10.869 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.869 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.869 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.869 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.869 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards-2:19.0.0-1629.g4f7075d1.el9.noarch 2024-05-09T10:49:10.869 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.869 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.869 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.869 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm-2:19.0.0-1629.g4f7075d1.el9.noarch 2024-05-09T10:49:10.870 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard-2:19.0.0-1629.g4f7075d1.el9.noarch 2024-05-09T10:49:10.870 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1629.g4f7075d1.el9.noarch 2024-05-09T10:49:10.870 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core-2:19.0.0-1629.g4f7075d1.el9.noarch 2024-05-09T10:49:10.870 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook-2:19.0.0-1629.g4f7075d1.el9.noarch 2024-05-09T10:49:10.870 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.870 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.870 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts-2:19.0.0-1629.g4f7075d1.el9.noarch 2024-05-09T10:49:10.870 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.870 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.870 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.870 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume-2:19.0.0-1629.g4f7075d1.el9.noarch 2024-05-09T10:49:10.870 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm-2:19.0.0-1629.g4f7075d1.el9.noarch 2024-05-09T10:49:10.871 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.871 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-shell-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.871 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top-2:19.0.0-1629.g4f7075d1.el9.noarch 2024-05-09T10:49:10.871 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T10:49:10.871 INFO:teuthology.orchestra.run.smithi098.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T10:49:10.871 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T10:49:10.871 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T10:49:10.871 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T10:49:10.871 INFO:teuthology.orchestra.run.smithi098.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T10:49:10.871 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T10:49:10.871 INFO:teuthology.orchestra.run.smithi098.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T10:49:10.871 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T10:49:10.871 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T10:49:10.871 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T10:49:10.871 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.872 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.872 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.872 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T10:49:10.872 INFO:teuthology.orchestra.run.smithi098.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T10:49:10.872 INFO:teuthology.orchestra.run.smithi098.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T10:49:10.872 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T10:49:10.872 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T10:49:10.872 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.872 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.872 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T10:49:10.872 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.872 INFO:teuthology.orchestra.run.smithi098.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T10:49:10.872 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T10:49:10.872 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T10:49:10.872 INFO:teuthology.orchestra.run.smithi098.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T10:49:10.873 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T10:49:10.873 INFO:teuthology.orchestra.run.smithi098.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T10:49:10.873 INFO:teuthology.orchestra.run.smithi098.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T10:49:10.873 INFO:teuthology.orchestra.run.smithi098.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T10:49:10.873 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T10:49:10.873 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T10:49:10.873 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T10:49:10.873 INFO:teuthology.orchestra.run.smithi098.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T10:49:10.873 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T10:49:10.873 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T10:49:10.874 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T10:49:10.874 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T10:49:10.874 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T10:49:10.874 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T10:49:10.874 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T10:49:10.874 INFO:teuthology.orchestra.run.smithi098.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T10:49:10.874 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T10:49:10.874 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T10:49:10.874 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T10:49:10.874 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.874 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.874 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.874 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T10:49:10.874 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T10:49:10.874 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T10:49:10.875 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T10:49:10.875 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T10:49:10.875 INFO:teuthology.orchestra.run.smithi098.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T10:49:10.875 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T10:49:10.875 INFO:teuthology.orchestra.run.smithi098.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T10:49:10.875 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T10:49:10.875 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T10:49:10.875 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T10:49:10.875 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T10:49:10.875 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T10:49:10.875 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T10:49:10.875 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T10:49:10.875 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T10:49:10.875 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T10:49:10.876 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T10:49:10.876 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T10:49:10.876 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T10:49:10.876 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T10:49:10.876 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T10:49:10.876 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T10:49:10.876 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T10:49:10.876 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T10:49:10.876 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T10:49:10.876 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T10:49:10.876 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T10:49:10.876 INFO:teuthology.orchestra.run.smithi098.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T10:49:10.876 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T10:49:10.877 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T10:49:10.877 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T10:49:10.877 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T10:49:10.877 INFO:teuthology.orchestra.run.smithi098.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T10:49:10.877 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T10:49:10.877 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T10:49:10.877 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T10:49:10.877 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T10:49:10.877 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T10:49:10.877 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T10:49:10.877 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.877 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.877 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T10:49:10.877 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T10:49:10.877 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T10:49:10.877 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.878 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T10:49:10.878 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T10:49:10.878 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T10:49:10.878 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T10:49:10.878 INFO:teuthology.orchestra.run.smithi098.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T10:49:10.878 INFO:teuthology.orchestra.run.smithi098.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T10:49:10.878 INFO:teuthology.orchestra.run.smithi098.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T10:49:10.878 INFO:teuthology.orchestra.run.smithi098.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T10:49:10.878 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T10:49:10.878 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T10:49:10.878 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T10:49:10.878 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T10:49:10.878 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T10:49:10.878 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T10:49:10.879 INFO:teuthology.orchestra.run.smithi098.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T10:49:10.879 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.879 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.879 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:10.879 INFO:teuthology.orchestra.run.smithi098.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T10:49:10.879 INFO:teuthology.orchestra.run.smithi098.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T10:49:10.879 INFO:teuthology.orchestra.run.smithi098.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T10:49:10.879 INFO:teuthology.orchestra.run.smithi098.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T10:49:10.879 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T10:49:10.879 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T10:49:10.880 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T10:49:10.880 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:10.880 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-05-09T10:49:11.288 DEBUG:teuthology.parallel:result is None 2024-05-09T10:49:14.371 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T10:49:14.371 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-2:19.0.0-1629.g4f7075d1.el9.x86_64 1/164 2024-05-09T10:49:14.371 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-base-2:19.0.0-1629.g4f7075d1.el9.x86_64 2/164 2024-05-09T10:49:14.371 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-common-2:19.0.0-1629.g4f7075d1.el9.x86_64 3/164 2024-05-09T10:49:14.371 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-fuse-2:19.0.0-1629.g4f7075d1.el9.x86_64 4/164 2024-05-09T10:49:14.371 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1629.g4f707 5/164 2024-05-09T10:49:14.371 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mds-2:19.0.0-1629.g4f7075d1.el9.x86_64 6/164 2024-05-09T10:49:14.371 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-2:19.0.0-1629.g4f7075d1.el9.x86_64 7/164 2024-05-09T10:49:14.371 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mon-2:19.0.0-1629.g4f7075d1.el9.x86_64 8/164 2024-05-09T10:49:14.372 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-osd-2:19.0.0-1629.g4f7075d1.el9.x86_64 9/164 2024-05-09T10:49:14.372 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-radosgw-2:19.0.0-1629.g4f7075d1.el9.x86_64 10/164 2024-05-09T10:49:14.372 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-selinux-2:19.0.0-1629.g4f7075d1.el9.x86_64 11/164 2024-05-09T10:49:14.372 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-test-2:19.0.0-1629.g4f7075d1.el9.x86_64 12/164 2024-05-09T10:49:14.372 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-mirror-2:19.0.0-1629.g4f7075d1.el9.x86_64 13/164 2024-05-09T10:49:14.372 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-shell-2:19.0.0-1629.g4f7075d1.el9.x86_64 14/164 2024-05-09T10:49:14.372 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs-devel-2:19.0.0-1629.g4f7075d1.el9.x86_ 15/164 2024-05-09T10:49:14.372 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs2-2:19.0.0-1629.g4f7075d1.el9.x86_64 16/164 2024-05-09T10:49:14.372 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephsqlite-2:19.0.0-1629.g4f7075d1.el9.x86_64 17/164 2024-05-09T10:49:14.372 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados-devel-2:19.0.0-1629.g4f7075d1.el9.x86_6 18/164 2024-05-09T10:49:14.372 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libradosstriper1-2:19.0.0-1629.g4f7075d1.el9.x86 19/164 2024-05-09T10:49:14.372 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librgw2-2:19.0.0-1629.g4f7075d1.el9.x86_64 20/164 2024-05-09T10:49:14.372 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1629.g4f7075d1.el 21/164 2024-05-09T10:49:14.372 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-common-2:19.0.0-1629.g4f7075d1.el9. 22/164 2024-05-09T10:49:14.373 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cephfs-2:19.0.0-1629.g4f7075d1.el9.x86_6 23/164 2024-05-09T10:49:14.373 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rados-2:19.0.0-1629.g4f7075d1.el9.x86_64 24/164 2024-05-09T10:49:14.373 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rbd-2:19.0.0-1629.g4f7075d1.el9.x86_64 25/164 2024-05-09T10:49:14.373 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rgw-2:19.0.0-1629.g4f7075d1.el9.x86_64 26/164 2024-05-09T10:49:14.373 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-fuse-2:19.0.0-1629.g4f7075d1.el9.x86_64 27/164 2024-05-09T10:49:14.386 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-mirror-2:19.0.0-1629.g4f7075d1.el9.x86_64 28/164 2024-05-09T10:49:14.387 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-nbd-2:19.0.0-1629.g4f7075d1.el9.x86_64 29/164 2024-05-09T10:49:14.387 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1629.g4f7075d1. 30/164 2024-05-09T10:49:14.387 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1629.g4f7075d1.el9.noa 31/164 2024-05-09T10:49:14.387 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1629.g4f7075d1.el9.n 32/164 2024-05-09T10:49:14.387 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1629.g4f7 33/164 2024-05-09T10:49:14.387 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1629.g4f7075d1.el 34/164 2024-05-09T10:49:14.387 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1629.g4f7075d1.el9.noarch 35/164 2024-05-09T10:49:14.387 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1629.g4f7075d1.e 36/164 2024-05-09T10:49:14.387 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-volume-2:19.0.0-1629.g4f7075d1.el9.noarch 37/164 2024-05-09T10:49:14.387 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephadm-2:19.0.0-1629.g4f7075d1.el9.noarch 38/164 2024-05-09T10:49:14.387 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-top-2:19.0.0-1629.g4f7075d1.el9.noarch 39/164 2024-05-09T10:49:14.387 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T10:49:14.387 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T10:49:14.388 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T10:49:14.388 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T10:49:14.388 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T10:49:14.388 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T10:49:14.388 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T10:49:14.388 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T10:49:14.388 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T10:49:14.388 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T10:49:14.388 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T10:49:14.388 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T10:49:14.388 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T10:49:14.388 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T10:49:14.388 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T10:49:14.388 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T10:49:14.388 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T10:49:14.389 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T10:49:14.389 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T10:49:14.389 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T10:49:14.389 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T10:49:14.389 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T10:49:14.389 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T10:49:14.389 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T10:49:14.389 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T10:49:14.389 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T10:49:14.389 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T10:49:14.389 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T10:49:14.389 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T10:49:14.389 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T10:49:14.389 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T10:49:14.390 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T10:49:14.390 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T10:49:14.390 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T10:49:14.390 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T10:49:14.390 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T10:49:14.390 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T10:49:14.390 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T10:49:14.390 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T10:49:14.390 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T10:49:14.390 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T10:49:14.390 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T10:49:14.390 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T10:49:14.390 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T10:49:14.390 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T10:49:14.391 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T10:49:14.391 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T10:49:14.391 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T10:49:14.391 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T10:49:14.391 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T10:49:14.391 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T10:49:14.391 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T10:49:14.391 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T10:49:14.391 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T10:49:14.391 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T10:49:14.391 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T10:49:14.391 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T10:49:14.391 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T10:49:14.391 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T10:49:14.391 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T10:49:14.392 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T10:49:14.392 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T10:49:14.392 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T10:49:14.392 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T10:49:14.392 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T10:49:14.392 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T10:49:14.392 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T10:49:14.392 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T10:49:14.392 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T10:49:14.392 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T10:49:14.392 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T10:49:14.392 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T10:49:14.392 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T10:49:14.392 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T10:49:14.392 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T10:49:14.393 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T10:49:14.393 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T10:49:14.393 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T10:49:14.393 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T10:49:14.393 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T10:49:14.393 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T10:49:14.393 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T10:49:14.393 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T10:49:14.393 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T10:49:14.393 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T10:49:14.393 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T10:49:14.393 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T10:49:14.393 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T10:49:14.393 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T10:49:14.393 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T10:49:14.394 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T10:49:14.394 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T10:49:14.394 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T10:49:14.394 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T10:49:14.394 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T10:49:14.394 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T10:49:14.394 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T10:49:14.394 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T10:49:14.394 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T10:49:14.394 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T10:49:14.394 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T10:49:14.394 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T10:49:14.394 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T10:49:14.394 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T10:49:14.394 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T10:49:14.395 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T10:49:14.395 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T10:49:14.395 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T10:49:14.395 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T10:49:14.395 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:19.0.0-1629.g4f7075d1.el9.x86_64 149/164 2024-05-09T10:49:14.395 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T10:49:14.395 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:19.0.0-1629.g4f7075d1.el9.x86_64 151/164 2024-05-09T10:49:14.395 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T10:49:14.395 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T10:49:14.395 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T10:49:14.395 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T10:49:14.395 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T10:49:14.395 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T10:49:14.395 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T10:49:14.395 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T10:49:14.396 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T10:49:14.396 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T10:49:14.396 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T10:49:14.396 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T10:49:17.027 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T10:49:17.028 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:17.028 INFO:teuthology.orchestra.run.smithi105.stdout:Upgraded: 2024-05-09T10:49:17.028 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T10:49:17.028 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T10:49:17.028 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T10:49:17.028 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T10:49:17.028 INFO:teuthology.orchestra.run.smithi105.stdout: librados2-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.028 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.028 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T10:49:17.028 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T10:49:17.028 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2024-05-09T10:49:17.028 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T10:49:17.029 INFO:teuthology.orchestra.run.smithi105.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T10:49:17.029 INFO:teuthology.orchestra.run.smithi105.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T10:49:17.029 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T10:49:17.029 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.029 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.029 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.029 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.029 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards-2:19.0.0-1629.g4f7075d1.el9.noarch 2024-05-09T10:49:17.029 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.029 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.029 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.029 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm-2:19.0.0-1629.g4f7075d1.el9.noarch 2024-05-09T10:49:17.029 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard-2:19.0.0-1629.g4f7075d1.el9.noarch 2024-05-09T10:49:17.029 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1629.g4f7075d1.el9.noarch 2024-05-09T10:49:17.030 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core-2:19.0.0-1629.g4f7075d1.el9.noarch 2024-05-09T10:49:17.030 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook-2:19.0.0-1629.g4f7075d1.el9.noarch 2024-05-09T10:49:17.030 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.030 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.030 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts-2:19.0.0-1629.g4f7075d1.el9.noarch 2024-05-09T10:49:17.030 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.030 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.030 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.030 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume-2:19.0.0-1629.g4f7075d1.el9.noarch 2024-05-09T10:49:17.031 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm-2:19.0.0-1629.g4f7075d1.el9.noarch 2024-05-09T10:49:17.031 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.031 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.031 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top-2:19.0.0-1629.g4f7075d1.el9.noarch 2024-05-09T10:49:17.031 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T10:49:17.031 INFO:teuthology.orchestra.run.smithi105.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T10:49:17.031 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T10:49:17.031 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T10:49:17.031 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T10:49:17.031 INFO:teuthology.orchestra.run.smithi105.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T10:49:17.031 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T10:49:17.031 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T10:49:17.031 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T10:49:17.031 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T10:49:17.031 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T10:49:17.032 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.032 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.032 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.032 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T10:49:17.032 INFO:teuthology.orchestra.run.smithi105.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T10:49:17.032 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T10:49:17.032 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T10:49:17.032 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T10:49:17.032 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.032 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.032 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T10:49:17.032 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.032 INFO:teuthology.orchestra.run.smithi105.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T10:49:17.033 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T10:49:17.033 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T10:49:17.033 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T10:49:17.033 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T10:49:17.033 INFO:teuthology.orchestra.run.smithi105.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T10:49:17.033 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T10:49:17.033 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T10:49:17.033 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T10:49:17.033 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T10:49:17.033 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T10:49:17.033 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T10:49:17.033 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T10:49:17.033 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T10:49:17.033 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T10:49:17.033 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T10:49:17.034 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T10:49:17.034 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T10:49:17.034 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T10:49:17.034 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T10:49:17.034 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T10:49:17.034 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T10:49:17.034 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T10:49:17.034 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.034 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.034 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.034 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T10:49:17.034 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T10:49:17.034 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T10:49:17.035 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T10:49:17.035 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T10:49:17.035 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T10:49:17.035 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T10:49:17.035 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T10:49:17.035 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T10:49:17.035 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T10:49:17.035 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T10:49:17.035 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T10:49:17.035 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T10:49:17.035 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T10:49:17.035 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T10:49:17.035 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T10:49:17.035 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T10:49:17.036 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T10:49:17.036 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T10:49:17.036 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T10:49:17.036 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T10:49:17.036 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T10:49:17.036 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T10:49:17.036 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T10:49:17.036 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T10:49:17.036 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T10:49:17.036 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T10:49:17.036 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T10:49:17.036 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T10:49:17.036 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T10:49:17.036 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T10:49:17.036 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T10:49:17.037 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T10:49:17.037 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T10:49:17.037 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T10:49:17.037 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T10:49:17.037 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T10:49:17.037 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T10:49:17.037 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T10:49:17.037 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T10:49:17.037 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.037 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.037 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T10:49:17.037 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T10:49:17.037 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T10:49:17.037 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.038 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T10:49:17.038 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T10:49:17.038 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T10:49:17.038 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T10:49:17.038 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T10:49:17.038 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T10:49:17.038 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T10:49:17.038 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T10:49:17.038 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T10:49:17.038 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T10:49:17.038 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T10:49:17.038 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T10:49:17.038 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T10:49:17.038 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T10:49:17.039 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T10:49:17.039 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.039 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.039 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd-2:19.0.0-1629.g4f7075d1.el9.x86_64 2024-05-09T10:49:17.039 INFO:teuthology.orchestra.run.smithi105.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T10:49:17.039 INFO:teuthology.orchestra.run.smithi105.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T10:49:17.039 INFO:teuthology.orchestra.run.smithi105.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T10:49:17.039 INFO:teuthology.orchestra.run.smithi105.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T10:49:17.039 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T10:49:17.039 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T10:49:17.040 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T10:49:17.040 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:17.040 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2024-05-09T10:49:17.492 DEBUG:teuthology.parallel:result is None 2024-05-09T10:49:17.492 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20 2024-05-09T10:49:17.632 DEBUG:teuthology.orchestra.run.smithi098:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T10:49:17.660 INFO:teuthology.orchestra.run.smithi098.stdout:19.0.0-1629.g4f7075d1.el9 2024-05-09T10:49:17.660 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1629.g4f7075d1.el9 2024-05-09T10:49:17.660 INFO:teuthology.task.install:The correct ceph version 19.0.0-1629.g4f7075d1 is installed. 2024-05-09T10:49:17.662 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20 2024-05-09T10:49:17.805 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T10:49:17.827 INFO:teuthology.orchestra.run.smithi105.stdout:19.0.0-1629.g4f7075d1.el9 2024-05-09T10:49:17.828 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1629.g4f7075d1.el9 2024-05-09T10:49:17.828 INFO:teuthology.task.install:The correct ceph version 19.0.0-1629.g4f7075d1 is installed. 2024-05-09T10:49:17.829 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-09T10:49:17.830 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:17.830 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T10:49:17.862 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:49:17.862 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T10:49:17.895 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-09T10:49:17.895 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:17.895 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T10:49:17.930 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T10:49:17.996 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:49:17.997 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T10:49:18.022 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T10:49:18.086 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-09T10:49:18.086 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:18.086 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T10:49:18.112 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T10:49:18.177 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:49:18.177 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T10:49:18.201 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T10:49:18.266 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-09T10:49:18.266 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:18.266 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T10:49:18.291 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T10:49:18.355 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:49:18.356 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T10:49:18.380 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T10:49:18.444 INFO:teuthology.run_tasks:Running task ceph... 2024-05-09T10:49:18.538 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-09T10:49:18.538 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 777 /var/log/ceph 2024-05-09T10:49:18.540 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 777 /var/log/ceph 2024-05-09T10:49:18.565 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-09T10:49:18.565 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-09T10:49:18.605 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-09T10:49:18.629 INFO:tasks.ceph:Creating extra log directories... 2024-05-09T10:49:18.630 DEBUG:teuthology.orchestra.run.smithi098:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-09T10:49:18.669 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-09T10:49:18.701 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-09T10:49:18.701 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-09T10:49:18.701 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-09T10:49:18.702 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:18.702 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-09T10:49:18.748 DEBUG:teuthology.orchestra.run.smithi098:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-09T10:49:18.858 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-09T10:49:18.899 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:49:18.899 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-09T10:49:18.914 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-09T10:49:19.025 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-09T10:49:19.056 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-09T10:49:19.056 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-09T10:49:19.056 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2024-05-09_10:23:18-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7699683', 'branch': 'wip-neesingh-testing1-20240507', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7699683', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2024-05-09_10:23:18-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing1-20240507'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-58090', 'sha1': '1836a1f5ed9f69ee7eef7aa7d49d42f0600dc280'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6829, 'sha1': '4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'wip-58090', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_1836a1f5ed9f69ee7eef7aa7d49d42f0600dc280/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': '1836a1f5ed9f69ee7eef7aa7d49d42f0600dc280', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi098.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBU5z6SJ/twu2gk90TtXpSa7p3EIXZVNKxZdCLlaeuFaXfwaRIP6RqKEOLouwyyyQMHNzmoebWboRk+iJDMUGeg=', 'smithi105.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLWdhHoI+mO2K+gFcG8zkd/kk0Xk2KNsMlsL2SxeUB5GE0PwLlokVNRdvL+7C2zbG4Ata0XcRL5tgNl9Mb6iqvY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '4f7075d1ae67d7d50c1b1c72da1cb2a47b619f20', 'cluster': 'ceph'}}, {'kclient': None}, {'check-counter': {'counters': {'mgr': [{'min': 4, 'name': 'finisher-volumes.complete_latency.avgcount'}, {'expected_val': 0, 'name': 'finisher-volumes.queue_len'}]}}}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestPerModuleFinsherThread']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-09_10:23:18', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-09T10:49:19.057 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-09T10:49:19.059 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-09T10:49:19.106 DEBUG:teuthology.orchestra.run.smithi098:> sudo install -d -m0777 -- /var/run/ceph 2024-05-09T10:49:19.114 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/run/ceph 2024-05-09T10:49:19.171 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:19.171 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T10:49:19.193 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T10:49:19.193 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_1 2024-05-09T10:49:19.249 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T10:49:19.249 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T10:49:19.249 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-09T10:49:19.249 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T10:49:19.249 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T10:49:19.249 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-05-09 10:49:03.343169568 +0000 2024-05-09T10:49:19.249 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-05-09 10:47:13.429458239 +0000 2024-05-09T10:49:19.249 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-05-09 10:47:13.429458239 +0000 2024-05-09T10:49:19.249 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-05-09 10:47:13.429458239 +0000 2024-05-09T10:49:19.250 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T10:49:19.312 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-05-09T10:49:19.312 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-05-09T10:49:19.312 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000207112 s, 2.5 MB/s 2024-05-09T10:49:19.313 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T10:49:19.371 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_2 2024-05-09T10:49:19.427 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T10:49:19.427 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T10:49:19.427 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-09T10:49:19.427 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T10:49:19.427 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T10:49:19.427 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-05-09 10:49:03.344169557 +0000 2024-05-09T10:49:19.427 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-05-09 10:47:13.783454326 +0000 2024-05-09T10:49:19.427 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-05-09 10:47:13.783454326 +0000 2024-05-09T10:49:19.427 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-05-09 10:47:13.783454326 +0000 2024-05-09T10:49:19.428 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T10:49:19.490 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-05-09T10:49:19.490 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-05-09T10:49:19.490 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000199848 s, 2.6 MB/s 2024-05-09T10:49:19.491 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T10:49:19.547 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_3 2024-05-09T10:49:19.602 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T10:49:19.602 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T10:49:19.602 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-09T10:49:19.602 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T10:49:19.602 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T10:49:19.602 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-05-09 10:49:03.344169557 +0000 2024-05-09T10:49:19.602 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-05-09 10:47:14.124450556 +0000 2024-05-09T10:49:19.603 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-05-09 10:47:14.124450556 +0000 2024-05-09T10:49:19.603 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-05-09 10:47:14.124450556 +0000 2024-05-09T10:49:19.603 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T10:49:19.665 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-05-09T10:49:19.665 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-05-09T10:49:19.665 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000192679 s, 2.7 MB/s 2024-05-09T10:49:19.666 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T10:49:19.722 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_4 2024-05-09T10:49:19.777 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T10:49:19.778 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T10:49:19.778 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-09T10:49:19.778 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T10:49:19.778 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T10:49:19.778 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-05-09 10:49:03.345169546 +0000 2024-05-09T10:49:19.778 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-05-09 10:47:14.475446675 +0000 2024-05-09T10:49:19.778 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-05-09 10:47:14.475446675 +0000 2024-05-09T10:49:19.778 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-05-09 10:47:14.475446675 +0000 2024-05-09T10:49:19.778 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T10:49:19.841 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-05-09T10:49:19.841 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-05-09T10:49:19.842 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000212484 s, 2.4 MB/s 2024-05-09T10:49:19.843 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T10:49:19.900 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T10:49:19.900 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:49:19.900 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T10:49:19.917 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T10:49:19.917 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_1 2024-05-09T10:49:19.973 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T10:49:19.974 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T10:49:19.974 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-09T10:49:19.974 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T10:49:19.974 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T10:49:19.974 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-05-09 10:49:09.769016948 +0000 2024-05-09T10:49:19.974 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-05-09 10:47:13.620328274 +0000 2024-05-09T10:49:19.974 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-05-09 10:47:13.620328274 +0000 2024-05-09T10:49:19.974 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-05-09 10:47:13.620328274 +0000 2024-05-09T10:49:19.974 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T10:49:20.038 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-05-09T10:49:20.038 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-05-09T10:49:20.038 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000286897 s, 1.8 MB/s 2024-05-09T10:49:20.040 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T10:49:20.097 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_2 2024-05-09T10:49:20.153 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T10:49:20.153 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T10:49:20.153 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-09T10:49:20.153 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T10:49:20.154 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T10:49:20.154 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-05-09 10:49:09.769016948 +0000 2024-05-09T10:49:20.154 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-05-09 10:47:13.973321113 +0000 2024-05-09T10:49:20.154 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-05-09 10:47:13.973321113 +0000 2024-05-09T10:49:20.154 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-05-09 10:47:13.973321113 +0000 2024-05-09T10:49:20.154 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T10:49:20.217 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-05-09T10:49:20.217 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-05-09T10:49:20.217 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000190066 s, 2.7 MB/s 2024-05-09T10:49:20.219 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T10:49:20.276 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_3 2024-05-09T10:49:20.331 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T10:49:20.331 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T10:49:20.331 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-05-09T10:49:20.331 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T10:49:20.332 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T10:49:20.332 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-05-09 10:49:09.770016928 +0000 2024-05-09T10:49:20.332 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-05-09 10:47:14.325313973 +0000 2024-05-09T10:49:20.332 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-05-09 10:47:14.325313973 +0000 2024-05-09T10:49:20.332 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-05-09 10:47:14.325313973 +0000 2024-05-09T10:49:20.332 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T10:49:20.394 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-05-09T10:49:20.394 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-05-09T10:49:20.394 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000223865 s, 2.3 MB/s 2024-05-09T10:49:20.395 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T10:49:20.452 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_4 2024-05-09T10:49:20.507 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T10:49:20.507 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T10:49:20.507 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-09T10:49:20.507 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T10:49:20.507 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T10:49:20.507 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-05-09 10:49:09.770016928 +0000 2024-05-09T10:49:20.507 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-05-09 10:47:14.677306832 +0000 2024-05-09T10:49:20.508 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-05-09 10:47:14.677306832 +0000 2024-05-09T10:49:20.508 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-05-09 10:47:14.677306832 +0000 2024-05-09T10:49:20.508 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T10:49:20.570 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-05-09T10:49:20.570 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-05-09T10:49:20.570 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000204217 s, 2.5 MB/s 2024-05-09T10:49:20.571 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T10:49:20.627 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T10:49:20.627 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi098.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-09T10:49:20.627 INFO:tasks.ceph:Generating config... 2024-05-09T10:49:20.628 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-09T10:49:20.628 INFO:tasks.ceph:[client] debug client = 20 2024-05-09T10:49:20.628 INFO:tasks.ceph:[client] debug ms = 1 2024-05-09T10:49:20.628 INFO:tasks.ceph:[client] pid file = 2024-05-09T10:49:20.629 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-09T10:49:20.629 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-09T10:49:20.629 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-05-09T10:49:20.629 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-09T10:49:20.629 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-09T10:49:20.629 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-09T10:49:20.629 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-09T10:49:20.629 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-09T10:49:20.629 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-09T10:49:20.629 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-09T10:49:20.629 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-09T10:49:20.629 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-09T10:49:20.629 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-09T10:49:20.629 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-09T10:49:20.629 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-05-09T10:49:20.630 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-09T10:49:20.630 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-09T10:49:20.630 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-09T10:49:20.630 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-09T10:49:20.630 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-09T10:49:20.630 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-09T10:49:20.630 INFO:tasks.ceph:[osd] bdev async discard = True 2024-05-09T10:49:20.630 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-05-09T10:49:20.630 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-05-09T10:49:20.630 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-09T10:49:20.630 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-09T10:49:20.630 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-09T10:49:20.630 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-09T10:49:20.630 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-09T10:49:20.630 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-09T10:49:20.631 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-09T10:49:20.631 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-09T10:49:20.631 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-09T10:49:20.631 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-09T10:49:20.631 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-09T10:49:20.631 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-09T10:49:20.631 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-09T10:49:20.631 INFO:tasks.ceph:Setting up mon.a... 2024-05-09T10:49:20.631 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-09T10:49:20.680 INFO:teuthology.orchestra.run.smithi098.stdout:creating /etc/ceph/ceph.keyring 2024-05-09T10:49:20.683 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-09T10:49:20.727 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T10:49:20.753 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.98'), ('mon.b', '172.21.15.105'), ('mon.c', '[v2:172.21.15.105:3301,v1:172.21.15.105:6790]')] 2024-05-09T10:49:20.753 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.98,172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-09T10:49:20.754 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:20.754 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-09T10:49:20.809 DEBUG:teuthology.orchestra.run.smithi098:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.98 --add b 172.21.15.105 --addv c '[v2:172.21.15.105:3301,v1:172.21.15.105:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-09T10:49:20.895 INFO:teuthology.orchestra.run.smithi098.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-09T10:49:20.895 INFO:teuthology.orchestra.run.smithi098.stdout:monmaptool: generated fsid 3f705980-2043-4fe5-8639-5415c9d3c944 2024-05-09T10:49:20.895 INFO:teuthology.orchestra.run.smithi098.stdout:setting min_mon_release = quincy 2024-05-09T10:49:20.895 INFO:teuthology.orchestra.run.smithi098.stdout:epoch 0 2024-05-09T10:49:20.896 INFO:teuthology.orchestra.run.smithi098.stdout:fsid 3f705980-2043-4fe5-8639-5415c9d3c944 2024-05-09T10:49:20.896 INFO:teuthology.orchestra.run.smithi098.stdout:last_changed 2024-05-09T10:49:20.903133+0000 2024-05-09T10:49:20.896 INFO:teuthology.orchestra.run.smithi098.stdout:created 2024-05-09T10:49:20.903133+0000 2024-05-09T10:49:20.896 INFO:teuthology.orchestra.run.smithi098.stdout:min_mon_release 17 (quincy) 2024-05-09T10:49:20.896 INFO:teuthology.orchestra.run.smithi098.stdout:election_strategy: 1 2024-05-09T10:49:20.896 INFO:teuthology.orchestra.run.smithi098.stdout:0: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.a 2024-05-09T10:49:20.896 INFO:teuthology.orchestra.run.smithi098.stdout:1: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2024-05-09T10:49:20.896 INFO:teuthology.orchestra.run.smithi098.stdout:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.c 2024-05-09T10:49:20.896 INFO:teuthology.orchestra.run.smithi098.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-09T10:49:20.897 DEBUG:teuthology.orchestra.run.smithi098:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-09T10:49:20.912 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 3f705980-2043-4fe5-8639-5415c9d3c944... 2024-05-09T10:49:20.913 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T10:49:20.955 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T10:49:20.994 INFO:teuthology.orchestra.run.smithi098.stdout:[global] 2024-05-09T10:49:20.994 INFO:teuthology.orchestra.run.smithi098.stdout: chdir = "" 2024-05-09T10:49:20.994 INFO:teuthology.orchestra.run.smithi098.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T10:49:20.995 INFO:teuthology.orchestra.run.smithi098.stdout: auth supported = cephx 2024-05-09T10:49:20.995 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:20.995 INFO:teuthology.orchestra.run.smithi098.stdout: filestore xattr use omap = true 2024-05-09T10:49:20.995 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:20.995 INFO:teuthology.orchestra.run.smithi098.stdout: mon clock drift allowed = 1.000 2024-05-09T10:49:20.995 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:20.995 INFO:teuthology.orchestra.run.smithi098.stdout: osd crush chooseleaf type = 0 2024-05-09T10:49:20.995 INFO:teuthology.orchestra.run.smithi098.stdout: auth debug = true 2024-05-09T10:49:20.995 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:20.995 INFO:teuthology.orchestra.run.smithi098.stdout: ms die on old message = true 2024-05-09T10:49:20.995 INFO:teuthology.orchestra.run.smithi098.stdout: ms die on bug = true 2024-05-09T10:49:20.995 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:20.995 INFO:teuthology.orchestra.run.smithi098.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T10:49:20.995 INFO:teuthology.orchestra.run.smithi098.stdout: mon pg warn max object skew = 0 2024-05-09T10:49:20.995 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:20.996 INFO:teuthology.orchestra.run.smithi098.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T10:49:20.996 INFO:teuthology.orchestra.run.smithi098.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T10:49:20.996 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:20.996 INFO:teuthology.orchestra.run.smithi098.stdout: osd pool default size = 2 2024-05-09T10:49:20.996 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:20.996 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd allow primary affinity = true 2024-05-09T10:49:20.996 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd allow pg remap = true 2024-05-09T10:49:20.996 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on legacy crush tunables = false 2024-05-09T10:49:20.996 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on crush straw calc version zero = false 2024-05-09T10:49:20.996 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on no sortbitwise = false 2024-05-09T10:49:20.996 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on osd down out interval zero = false 2024-05-09T10:49:20.996 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on too few osds = false 2024-05-09T10:49:20.996 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T10:49:20.996 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T10:49:20.996 INFO:teuthology.orchestra.run.smithi098.stdout: mon_allow_pool_size_one = true 2024-05-09T10:49:20.996 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:20.997 INFO:teuthology.orchestra.run.smithi098.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T10:49:20.997 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:20.997 INFO:teuthology.orchestra.run.smithi098.stdout: osd default data pool replay window = 5 2024-05-09T10:49:20.997 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:20.997 INFO:teuthology.orchestra.run.smithi098.stdout: mon allow pool delete = true 2024-05-09T10:49:20.997 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:20.997 INFO:teuthology.orchestra.run.smithi098.stdout: mon cluster log file level = debug 2024-05-09T10:49:20.997 INFO:teuthology.orchestra.run.smithi098.stdout: debug asserts on shutdown = true 2024-05-09T10:49:20.997 INFO:teuthology.orchestra.run.smithi098.stdout: mon health detail to clog = false 2024-05-09T10:49:20.997 INFO:teuthology.orchestra.run.smithi098.stdout: mon host = "172.21.15.98,172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790]" 2024-05-09T10:49:20.997 INFO:teuthology.orchestra.run.smithi098.stdout: ms die on skipped message = False 2024-05-09T10:49:20.997 INFO:teuthology.orchestra.run.smithi098.stdout: fsid = 3f705980-2043-4fe5-8639-5415c9d3c944 2024-05-09T10:49:20.997 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:20.997 INFO:teuthology.orchestra.run.smithi098.stdout:[osd] 2024-05-09T10:49:20.997 INFO:teuthology.orchestra.run.smithi098.stdout: osd journal size = 100 2024-05-09T10:49:20.997 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:20.998 INFO:teuthology.orchestra.run.smithi098.stdout: osd scrub load threshold = 5.0 2024-05-09T10:49:20.998 INFO:teuthology.orchestra.run.smithi098.stdout: osd scrub max interval = 600 2024-05-09T10:49:20.998 INFO:teuthology.orchestra.run.smithi098.stdout: osd mclock profile = high_recovery_ops 2024-05-09T10:49:20.998 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:20.998 INFO:teuthology.orchestra.run.smithi098.stdout: osd recover clone overlap = true 2024-05-09T10:49:20.998 INFO:teuthology.orchestra.run.smithi098.stdout: osd recovery max chunk = 1048576 2024-05-09T10:49:20.998 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:20.998 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug shutdown = true 2024-05-09T10:49:20.998 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug op order = true 2024-05-09T10:49:20.998 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug verify stray on activate = true 2024-05-09T10:49:20.998 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug trim objects = true 2024-05-09T10:49:20.998 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:20.998 INFO:teuthology.orchestra.run.smithi098.stdout: osd open classes on start = true 2024-05-09T10:49:20.998 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug pg log writeout = true 2024-05-09T10:49:20.998 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:20.998 INFO:teuthology.orchestra.run.smithi098.stdout: osd deep scrub update digest min age = 30 2024-05-09T10:49:20.999 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:20.999 INFO:teuthology.orchestra.run.smithi098.stdout: osd map max advance = 10 2024-05-09T10:49:20.999 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:20.999 INFO:teuthology.orchestra.run.smithi098.stdout: journal zero on create = true 2024-05-09T10:49:20.999 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:20.999 INFO:teuthology.orchestra.run.smithi098.stdout: filestore ondisk finisher threads = 3 2024-05-09T10:49:20.999 INFO:teuthology.orchestra.run.smithi098.stdout: filestore apply finisher threads = 3 2024-05-09T10:49:20.999 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:20.999 INFO:teuthology.orchestra.run.smithi098.stdout: bdev debug aio = true 2024-05-09T10:49:20.999 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug misdirected ops = true 2024-05-09T10:49:20.999 INFO:teuthology.orchestra.run.smithi098.stdout: bdev async discard = True 2024-05-09T10:49:20.999 INFO:teuthology.orchestra.run.smithi098.stdout: bdev enable discard = True 2024-05-09T10:49:20.999 INFO:teuthology.orchestra.run.smithi098.stdout: bluestore allocator = bitmap 2024-05-09T10:49:20.999 INFO:teuthology.orchestra.run.smithi098.stdout: bluestore block size = 96636764160 2024-05-09T10:49:20.999 INFO:teuthology.orchestra.run.smithi098.stdout: bluestore fsck on mount = True 2024-05-09T10:49:20.999 INFO:teuthology.orchestra.run.smithi098.stdout: debug bluefs = 1/20 2024-05-09T10:49:21.000 INFO:teuthology.orchestra.run.smithi098.stdout: debug bluestore = 1/20 2024-05-09T10:49:21.000 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2024-05-09T10:49:21.000 INFO:teuthology.orchestra.run.smithi098.stdout: debug osd = 20 2024-05-09T10:49:21.000 INFO:teuthology.orchestra.run.smithi098.stdout: debug rocksdb = 4/10 2024-05-09T10:49:21.000 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T10:49:21.000 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd full ratio = 0.9 2024-05-09T10:49:21.000 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd nearfull ratio = 0.8 2024-05-09T10:49:21.000 INFO:teuthology.orchestra.run.smithi098.stdout: osd failsafe full ratio = 0.95 2024-05-09T10:49:21.000 INFO:teuthology.orchestra.run.smithi098.stdout: osd objectstore = bluestore 2024-05-09T10:49:21.000 INFO:teuthology.orchestra.run.smithi098.stdout: osd op complaint time = 180 2024-05-09T10:49:21.000 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:21.000 INFO:teuthology.orchestra.run.smithi098.stdout:[mgr] 2024-05-09T10:49:21.000 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2024-05-09T10:49:21.000 INFO:teuthology.orchestra.run.smithi098.stdout: debug mgr = 20 2024-05-09T10:49:21.000 INFO:teuthology.orchestra.run.smithi098.stdout: debug mon = 20 2024-05-09T10:49:21.000 INFO:teuthology.orchestra.run.smithi098.stdout: debug auth = 20 2024-05-09T10:49:21.001 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min pgs per osd = 4 2024-05-09T10:49:21.001 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min bytes per osd = 10 2024-05-09T10:49:21.001 INFO:teuthology.orchestra.run.smithi098.stdout: mgr/telemetry/nag = false 2024-05-09T10:49:21.001 INFO:teuthology.orchestra.run.smithi098.stdout: debug client = 20 2024-05-09T10:49:21.001 INFO:teuthology.orchestra.run.smithi098.stdout: debug finisher = 20 2024-05-09T10:49:21.001 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:21.001 INFO:teuthology.orchestra.run.smithi098.stdout:[mon] 2024-05-09T10:49:21.001 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2024-05-09T10:49:21.001 INFO:teuthology.orchestra.run.smithi098.stdout: debug mon = 20 2024-05-09T10:49:21.001 INFO:teuthology.orchestra.run.smithi098.stdout: debug paxos = 20 2024-05-09T10:49:21.001 INFO:teuthology.orchestra.run.smithi098.stdout: debug auth = 20 2024-05-09T10:49:21.001 INFO:teuthology.orchestra.run.smithi098.stdout: mon data avail warn = 5 2024-05-09T10:49:21.001 INFO:teuthology.orchestra.run.smithi098.stdout: mon mgr mkfs grace = 240 2024-05-09T10:49:21.001 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min pgs per osd = 4 2024-05-09T10:49:21.001 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd reporter subtree level = osd 2024-05-09T10:49:21.002 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd prime pg temp = true 2024-05-09T10:49:21.002 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min bytes per osd = 10 2024-05-09T10:49:21.002 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:21.002 INFO:teuthology.orchestra.run.smithi098.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T10:49:21.002 INFO:teuthology.orchestra.run.smithi098.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T10:49:21.002 INFO:teuthology.orchestra.run.smithi098.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T10:49:21.002 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:21.002 INFO:teuthology.orchestra.run.smithi098.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T10:49:21.002 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T10:49:21.002 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T10:49:21.002 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:21.002 INFO:teuthology.orchestra.run.smithi098.stdout: # 1m isn't quite enough 2024-05-09T10:49:21.002 INFO:teuthology.orchestra.run.smithi098.stdout: mon_down_mkfs_grace = 2m 2024-05-09T10:49:21.002 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:21.002 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_filestore_osds = false 2024-05-09T10:49:21.003 INFO:teuthology.orchestra.run.smithi098.stdout: mon op complaint time = 120 2024-05-09T10:49:21.003 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:21.003 INFO:teuthology.orchestra.run.smithi098.stdout:[client] 2024-05-09T10:49:21.003 INFO:teuthology.orchestra.run.smithi098.stdout: rgw cache enabled = true 2024-05-09T10:49:21.003 INFO:teuthology.orchestra.run.smithi098.stdout: rgw enable ops log = true 2024-05-09T10:49:21.003 INFO:teuthology.orchestra.run.smithi098.stdout: rgw enable usage log = true 2024-05-09T10:49:21.003 INFO:teuthology.orchestra.run.smithi098.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T10:49:21.003 INFO:teuthology.orchestra.run.smithi098.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T10:49:21.003 INFO:teuthology.orchestra.run.smithi098.stdout: client mount timeout = 600 2024-05-09T10:49:21.003 INFO:teuthology.orchestra.run.smithi098.stdout: debug client = 20 2024-05-09T10:49:21.003 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2024-05-09T10:49:21.003 INFO:teuthology.orchestra.run.smithi098.stdout: pid file = "" 2024-05-09T10:49:21.003 INFO:teuthology.orchestra.run.smithi098.stdout: rados mon op timeout = 900 2024-05-09T10:49:21.003 INFO:teuthology.orchestra.run.smithi098.stdout: rados osd op timeout = 900 2024-05-09T10:49:21.003 INFO:teuthology.orchestra.run.smithi098.stdout:[mon.a] 2024-05-09T10:49:21.003 INFO:teuthology.orchestra.run.smithi098.stdout:[mon.b] 2024-05-09T10:49:21.004 INFO:teuthology.orchestra.run.smithi098.stdout:[mon.c] 2024-05-09T10:49:21.004 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.a] 2024-05-09T10:49:21.004 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.c] 2024-05-09T10:49:21.004 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.b] 2024-05-09T10:49:21.004 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.d] 2024-05-09T10:49:21.004 INFO:teuthology.orchestra.run.smithi098.stdout:[mds] 2024-05-09T10:49:21.004 INFO:teuthology.orchestra.run.smithi098.stdout: debug mds = 20 2024-05-09T10:49:21.004 INFO:teuthology.orchestra.run.smithi098.stdout: debug mds balancer = 20 2024-05-09T10:49:21.004 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2024-05-09T10:49:21.004 INFO:teuthology.orchestra.run.smithi098.stdout: mds debug frag = True 2024-05-09T10:49:21.004 INFO:teuthology.orchestra.run.smithi098.stdout: mds debug scatterstat = True 2024-05-09T10:49:21.004 INFO:teuthology.orchestra.run.smithi098.stdout: mds op complaint time = 180 2024-05-09T10:49:21.004 INFO:teuthology.orchestra.run.smithi098.stdout: mds verify scatter = True 2024-05-09T10:49:21.004 INFO:teuthology.orchestra.run.smithi098.stdout: osd op complaint time = 180 2024-05-09T10:49:21.004 INFO:teuthology.orchestra.run.smithi098.stdout: rados mon op timeout = 900 2024-05-09T10:49:21.004 INFO:teuthology.orchestra.run.smithi098.stdout: rados osd op timeout = 900 2024-05-09T10:49:21.005 INFO:teuthology.orchestra.run.smithi105.stdout:[global] 2024-05-09T10:49:21.005 INFO:teuthology.orchestra.run.smithi105.stdout: chdir = "" 2024-05-09T10:49:21.005 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T10:49:21.005 INFO:teuthology.orchestra.run.smithi105.stdout: auth supported = cephx 2024-05-09T10:49:21.005 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.005 INFO:teuthology.orchestra.run.smithi105.stdout: filestore xattr use omap = true 2024-05-09T10:49:21.005 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.006 INFO:teuthology.orchestra.run.smithi105.stdout: mon clock drift allowed = 1.000 2024-05-09T10:49:21.006 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.006 INFO:teuthology.orchestra.run.smithi105.stdout: osd crush chooseleaf type = 0 2024-05-09T10:49:21.006 INFO:teuthology.orchestra.run.smithi105.stdout: auth debug = true 2024-05-09T10:49:21.006 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.006 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on old message = true 2024-05-09T10:49:21.006 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on bug = true 2024-05-09T10:49:21.006 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.006 INFO:teuthology.orchestra.run.smithi105.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T10:49:21.006 INFO:teuthology.orchestra.run.smithi105.stdout: mon pg warn max object skew = 0 2024-05-09T10:49:21.006 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.006 INFO:teuthology.orchestra.run.smithi105.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T10:49:21.006 INFO:teuthology.orchestra.run.smithi105.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T10:49:21.006 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.006 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default size = 2 2024-05-09T10:49:21.007 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.007 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow primary affinity = true 2024-05-09T10:49:21.007 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow pg remap = true 2024-05-09T10:49:21.007 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on legacy crush tunables = false 2024-05-09T10:49:21.007 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on crush straw calc version zero = false 2024-05-09T10:49:21.007 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on no sortbitwise = false 2024-05-09T10:49:21.007 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on osd down out interval zero = false 2024-05-09T10:49:21.007 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on too few osds = false 2024-05-09T10:49:21.007 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T10:49:21.007 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T10:49:21.007 INFO:teuthology.orchestra.run.smithi105.stdout: mon_allow_pool_size_one = true 2024-05-09T10:49:21.007 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.007 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T10:49:21.007 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.007 INFO:teuthology.orchestra.run.smithi105.stdout: osd default data pool replay window = 5 2024-05-09T10:49:21.008 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.008 INFO:teuthology.orchestra.run.smithi105.stdout: mon allow pool delete = true 2024-05-09T10:49:21.008 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.008 INFO:teuthology.orchestra.run.smithi105.stdout: mon cluster log file level = debug 2024-05-09T10:49:21.008 INFO:teuthology.orchestra.run.smithi105.stdout: debug asserts on shutdown = true 2024-05-09T10:49:21.008 INFO:teuthology.orchestra.run.smithi105.stdout: mon health detail to clog = false 2024-05-09T10:49:21.008 INFO:teuthology.orchestra.run.smithi105.stdout: mon host = "172.21.15.98,172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790]" 2024-05-09T10:49:21.008 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on skipped message = False 2024-05-09T10:49:21.008 INFO:teuthology.orchestra.run.smithi105.stdout: fsid = 3f705980-2043-4fe5-8639-5415c9d3c944 2024-05-09T10:49:21.008 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.008 INFO:teuthology.orchestra.run.smithi105.stdout:[osd] 2024-05-09T10:49:21.008 INFO:teuthology.orchestra.run.smithi105.stdout: osd journal size = 100 2024-05-09T10:49:21.008 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.008 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub load threshold = 5.0 2024-05-09T10:49:21.008 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub max interval = 600 2024-05-09T10:49:21.008 INFO:teuthology.orchestra.run.smithi105.stdout: osd mclock profile = high_recovery_ops 2024-05-09T10:49:21.009 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.009 INFO:teuthology.orchestra.run.smithi105.stdout: osd recover clone overlap = true 2024-05-09T10:49:21.009 INFO:teuthology.orchestra.run.smithi105.stdout: osd recovery max chunk = 1048576 2024-05-09T10:49:21.009 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.009 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug shutdown = true 2024-05-09T10:49:21.009 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug op order = true 2024-05-09T10:49:21.009 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug verify stray on activate = true 2024-05-09T10:49:21.009 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug trim objects = true 2024-05-09T10:49:21.009 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.009 INFO:teuthology.orchestra.run.smithi105.stdout: osd open classes on start = true 2024-05-09T10:49:21.009 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug pg log writeout = true 2024-05-09T10:49:21.009 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.010 INFO:teuthology.orchestra.run.smithi105.stdout: osd deep scrub update digest min age = 30 2024-05-09T10:49:21.010 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.010 INFO:teuthology.orchestra.run.smithi105.stdout: osd map max advance = 10 2024-05-09T10:49:21.010 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.010 INFO:teuthology.orchestra.run.smithi105.stdout: journal zero on create = true 2024-05-09T10:49:21.010 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.010 INFO:teuthology.orchestra.run.smithi105.stdout: filestore ondisk finisher threads = 3 2024-05-09T10:49:21.010 INFO:teuthology.orchestra.run.smithi105.stdout: filestore apply finisher threads = 3 2024-05-09T10:49:21.010 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.010 INFO:teuthology.orchestra.run.smithi105.stdout: bdev debug aio = true 2024-05-09T10:49:21.010 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug misdirected ops = true 2024-05-09T10:49:21.010 INFO:teuthology.orchestra.run.smithi105.stdout: bdev async discard = True 2024-05-09T10:49:21.010 INFO:teuthology.orchestra.run.smithi105.stdout: bdev enable discard = True 2024-05-09T10:49:21.010 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore allocator = bitmap 2024-05-09T10:49:21.010 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore block size = 96636764160 2024-05-09T10:49:21.011 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore fsck on mount = True 2024-05-09T10:49:21.011 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluefs = 1/20 2024-05-09T10:49:21.011 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluestore = 1/20 2024-05-09T10:49:21.011 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-05-09T10:49:21.011 INFO:teuthology.orchestra.run.smithi105.stdout: debug osd = 20 2024-05-09T10:49:21.011 INFO:teuthology.orchestra.run.smithi105.stdout: debug rocksdb = 4/10 2024-05-09T10:49:21.011 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T10:49:21.011 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd full ratio = 0.9 2024-05-09T10:49:21.011 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd nearfull ratio = 0.8 2024-05-09T10:49:21.011 INFO:teuthology.orchestra.run.smithi105.stdout: osd failsafe full ratio = 0.95 2024-05-09T10:49:21.011 INFO:teuthology.orchestra.run.smithi105.stdout: osd objectstore = bluestore 2024-05-09T10:49:21.011 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2024-05-09T10:49:21.011 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.011 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr] 2024-05-09T10:49:21.011 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-05-09T10:49:21.011 INFO:teuthology.orchestra.run.smithi105.stdout: debug mgr = 20 2024-05-09T10:49:21.012 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2024-05-09T10:49:21.012 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2024-05-09T10:49:21.012 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2024-05-09T10:49:21.012 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2024-05-09T10:49:21.012 INFO:teuthology.orchestra.run.smithi105.stdout: mgr/telemetry/nag = false 2024-05-09T10:49:21.012 INFO:teuthology.orchestra.run.smithi105.stdout: debug client = 20 2024-05-09T10:49:21.012 INFO:teuthology.orchestra.run.smithi105.stdout: debug finisher = 20 2024-05-09T10:49:21.012 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.012 INFO:teuthology.orchestra.run.smithi105.stdout:[mon] 2024-05-09T10:49:21.012 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-05-09T10:49:21.012 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2024-05-09T10:49:21.012 INFO:teuthology.orchestra.run.smithi105.stdout: debug paxos = 20 2024-05-09T10:49:21.012 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2024-05-09T10:49:21.012 INFO:teuthology.orchestra.run.smithi105.stdout: mon data avail warn = 5 2024-05-09T10:49:21.012 INFO:teuthology.orchestra.run.smithi105.stdout: mon mgr mkfs grace = 240 2024-05-09T10:49:21.012 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2024-05-09T10:49:21.013 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd reporter subtree level = osd 2024-05-09T10:49:21.013 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd prime pg temp = true 2024-05-09T10:49:21.013 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2024-05-09T10:49:21.013 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.013 INFO:teuthology.orchestra.run.smithi105.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T10:49:21.013 INFO:teuthology.orchestra.run.smithi105.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T10:49:21.013 INFO:teuthology.orchestra.run.smithi105.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T10:49:21.013 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.013 INFO:teuthology.orchestra.run.smithi105.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T10:49:21.013 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T10:49:21.013 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T10:49:21.013 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.013 INFO:teuthology.orchestra.run.smithi105.stdout: # 1m isn't quite enough 2024-05-09T10:49:21.013 INFO:teuthology.orchestra.run.smithi105.stdout: mon_down_mkfs_grace = 2m 2024-05-09T10:49:21.013 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.014 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_filestore_osds = false 2024-05-09T10:49:21.014 INFO:teuthology.orchestra.run.smithi105.stdout: mon op complaint time = 120 2024-05-09T10:49:21.014 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-05-09T10:49:21.014 INFO:teuthology.orchestra.run.smithi105.stdout:[client] 2024-05-09T10:49:21.014 INFO:teuthology.orchestra.run.smithi105.stdout: rgw cache enabled = true 2024-05-09T10:49:21.014 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable ops log = true 2024-05-09T10:49:21.014 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable usage log = true 2024-05-09T10:49:21.014 INFO:teuthology.orchestra.run.smithi105.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T10:49:21.014 INFO:teuthology.orchestra.run.smithi105.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T10:49:21.014 INFO:teuthology.orchestra.run.smithi105.stdout: client mount timeout = 600 2024-05-09T10:49:21.014 INFO:teuthology.orchestra.run.smithi105.stdout: debug client = 20 2024-05-09T10:49:21.014 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-05-09T10:49:21.014 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = "" 2024-05-09T10:49:21.014 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2024-05-09T10:49:21.014 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2024-05-09T10:49:21.014 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.a] 2024-05-09T10:49:21.015 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.b] 2024-05-09T10:49:21.015 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.c] 2024-05-09T10:49:21.015 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2024-05-09T10:49:21.015 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2024-05-09T10:49:21.015 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2024-05-09T10:49:21.015 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2024-05-09T10:49:21.015 INFO:teuthology.orchestra.run.smithi105.stdout:[mds] 2024-05-09T10:49:21.015 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds = 20 2024-05-09T10:49:21.015 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds balancer = 20 2024-05-09T10:49:21.015 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-05-09T10:49:21.015 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug frag = True 2024-05-09T10:49:21.015 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug scatterstat = True 2024-05-09T10:49:21.015 INFO:teuthology.orchestra.run.smithi105.stdout: mds op complaint time = 180 2024-05-09T10:49:21.015 INFO:teuthology.orchestra.run.smithi105.stdout: mds verify scatter = True 2024-05-09T10:49:21.015 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2024-05-09T10:49:21.015 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2024-05-09T10:49:21.016 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2024-05-09T10:49:21.017 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-09T10:49:21.017 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-09T10:49:21.111 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-09T10:49:21.111 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:21.112 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-09T10:49:21.127 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:21.127 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-09T10:49:21.182 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi098.front.sepia.ceph.com 2024-05-09T10:49:21.182 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:21.182 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-09T10:49:21.182 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T10:49:21.256 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:21.256 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-09T10:49:21.312 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi105.front.sepia.ceph.com 2024-05-09T10:49:21.312 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:49:21.312 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-09T10:49:21.312 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T10:49:21.347 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:49:21.347 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-09T10:49:21.403 INFO:tasks.ceph:Setting up mon nodes... 2024-05-09T10:49:21.403 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-09T10:49:21.403 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-09T10:49:21.454 INFO:teuthology.orchestra.run.smithi098.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-09T10:49:21.456 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-09T10:49:21.512 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-09T10:49:21.514 INFO:tasks.ceph:Setting up mds nodes... 2024-05-09T10:49:21.514 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-09T10:49:21.564 INFO:teuthology.orchestra.run.smithi098.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-09T10:49:21.567 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-09T10:49:21.591 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-09T10:49:21.681 INFO:teuthology.orchestra.run.smithi098.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-09T10:49:21.684 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-09T10:49:21.708 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-09T10:49:21.759 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-09T10:49:21.761 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-09T10:49:21.786 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-09T10:49:21.876 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-09T10:49:21.879 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-09T10:49:21.903 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-09T10:49:21.903 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-09T10:49:21.945 INFO:teuthology.orchestra.run.smithi098.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-09T10:49:21.958 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-09T10:49:22.039 INFO:teuthology.orchestra.run.smithi098.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-09T10:49:22.051 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-09T10:49:22.092 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-09T10:49:22.104 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-09T10:49:22.184 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-09T10:49:22.197 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-09T10:49:22.197 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi098.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-09T10:49:22.197 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-09T10:49:22.223 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T10:49:22.223 INFO:tasks.ceph:role: osd.0 2024-05-09T10:49:22.223 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi098.front.sepia.ceph.com 2024-05-09T10:49:22.223 DEBUG:teuthology.orchestra.run.smithi098:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-09T10:49:22.295 INFO:teuthology.orchestra.run.smithi098.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T10:49:22.295 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T10:49:22.295 INFO:teuthology.orchestra.run.smithi098.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T10:49:22.295 INFO:teuthology.orchestra.run.smithi098.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T10:49:22.295 INFO:teuthology.orchestra.run.smithi098.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T10:49:22.295 INFO:teuthology.orchestra.run.smithi098.stdout: = sunit=0 swidth=0 blks 2024-05-09T10:49:22.295 INFO:teuthology.orchestra.run.smithi098.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T10:49:22.295 INFO:teuthology.orchestra.run.smithi098.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T10:49:22.295 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T10:49:22.295 INFO:teuthology.orchestra.run.smithi098.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T10:49:22.815 INFO:teuthology.orchestra.run.smithi098.stdout:Discarding blocks...Done. 2024-05-09T10:49:22.816 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi098.front.sepia.ceph.com -o noatime 2024-05-09T10:49:22.816 DEBUG:teuthology.orchestra.run.smithi098:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-09T10:49:22.925 DEBUG:teuthology.orchestra.run.smithi098:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-09T10:49:22.958 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-09T10:49:23.025 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T10:49:23.025 INFO:tasks.ceph:role: osd.1 2024-05-09T10:49:23.025 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi098.front.sepia.ceph.com 2024-05-09T10:49:23.025 DEBUG:teuthology.orchestra.run.smithi098:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-09T10:49:23.098 INFO:teuthology.orchestra.run.smithi098.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T10:49:23.098 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T10:49:23.098 INFO:teuthology.orchestra.run.smithi098.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T10:49:23.098 INFO:teuthology.orchestra.run.smithi098.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T10:49:23.098 INFO:teuthology.orchestra.run.smithi098.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T10:49:23.098 INFO:teuthology.orchestra.run.smithi098.stdout: = sunit=0 swidth=0 blks 2024-05-09T10:49:23.098 INFO:teuthology.orchestra.run.smithi098.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T10:49:23.098 INFO:teuthology.orchestra.run.smithi098.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T10:49:23.098 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T10:49:23.098 INFO:teuthology.orchestra.run.smithi098.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T10:49:23.559 INFO:teuthology.orchestra.run.smithi098.stdout:Discarding blocks...Done. 2024-05-09T10:49:23.561 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi098.front.sepia.ceph.com -o noatime 2024-05-09T10:49:23.561 DEBUG:teuthology.orchestra.run.smithi098:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-09T10:49:23.658 DEBUG:teuthology.orchestra.run.smithi098:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-09T10:49:23.689 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-09T10:49:23.756 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T10:49:23.756 INFO:tasks.ceph:role: osd.2 2024-05-09T10:49:23.756 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi098.front.sepia.ceph.com 2024-05-09T10:49:23.756 DEBUG:teuthology.orchestra.run.smithi098:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-09T10:49:23.829 INFO:teuthology.orchestra.run.smithi098.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T10:49:23.829 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T10:49:23.829 INFO:teuthology.orchestra.run.smithi098.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T10:49:23.829 INFO:teuthology.orchestra.run.smithi098.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T10:49:23.829 INFO:teuthology.orchestra.run.smithi098.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T10:49:23.829 INFO:teuthology.orchestra.run.smithi098.stdout: = sunit=0 swidth=0 blks 2024-05-09T10:49:23.830 INFO:teuthology.orchestra.run.smithi098.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T10:49:23.830 INFO:teuthology.orchestra.run.smithi098.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T10:49:23.830 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T10:49:23.830 INFO:teuthology.orchestra.run.smithi098.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T10:49:24.317 INFO:teuthology.orchestra.run.smithi098.stdout:Discarding blocks...Done. 2024-05-09T10:49:24.319 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi098.front.sepia.ceph.com -o noatime 2024-05-09T10:49:24.319 DEBUG:teuthology.orchestra.run.smithi098:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-09T10:49:24.463 DEBUG:teuthology.orchestra.run.smithi098:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-09T10:49:24.492 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-09T10:49:24.557 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T10:49:24.558 INFO:tasks.ceph:role: osd.3 2024-05-09T10:49:24.558 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi098.front.sepia.ceph.com 2024-05-09T10:49:24.558 DEBUG:teuthology.orchestra.run.smithi098:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-09T10:49:24.630 INFO:teuthology.orchestra.run.smithi098.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T10:49:24.631 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T10:49:24.631 INFO:teuthology.orchestra.run.smithi098.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T10:49:24.631 INFO:teuthology.orchestra.run.smithi098.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T10:49:24.631 INFO:teuthology.orchestra.run.smithi098.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T10:49:24.631 INFO:teuthology.orchestra.run.smithi098.stdout: = sunit=0 swidth=0 blks 2024-05-09T10:49:24.631 INFO:teuthology.orchestra.run.smithi098.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T10:49:24.631 INFO:teuthology.orchestra.run.smithi098.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T10:49:24.631 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T10:49:24.631 INFO:teuthology.orchestra.run.smithi098.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T10:49:25.149 INFO:teuthology.orchestra.run.smithi098.stdout:Discarding blocks...Done. 2024-05-09T10:49:25.151 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi098.front.sepia.ceph.com -o noatime 2024-05-09T10:49:25.151 DEBUG:teuthology.orchestra.run.smithi098:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-09T10:49:25.286 DEBUG:teuthology.orchestra.run.smithi098:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-09T10:49:25.315 DEBUG:teuthology.orchestra.run.smithi098:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T10:49:25.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:25.424+0000 7f390372c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-09T10:49:25.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:25.424+0000 7f390372c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-09T10:49:25.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:25.424+0000 7f390372c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T10:49:25.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:25.424+0000 7f390372c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T10:49:25.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:25.424+0000 7f390372c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T10:49:25.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:25.425+0000 7f390372c640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-09T10:49:28.264 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-09T10:49:28.294 DEBUG:teuthology.orchestra.run.smithi098:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T10:49:28.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:28.391+0000 7f1e28e18640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-09T10:49:28.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:28.391+0000 7f1e28e18640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-09T10:49:28.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:28.391+0000 7f1e28e18640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T10:49:28.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:28.391+0000 7f1e28e18640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T10:49:28.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:28.391+0000 7f1e28e18640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T10:49:28.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:28.392+0000 7f1e28e18640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-09T10:49:31.234 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-09T10:49:31.263 DEBUG:teuthology.orchestra.run.smithi098:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T10:49:31.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:31.359+0000 7f24910af640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-09T10:49:31.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:31.359+0000 7f24910af640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-09T10:49:31.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:31.359+0000 7f24910af640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T10:49:31.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:31.359+0000 7f24910af640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T10:49:31.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:31.359+0000 7f24910af640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T10:49:31.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:31.360+0000 7f24910af640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-09T10:49:34.204 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-09T10:49:34.233 DEBUG:teuthology.orchestra.run.smithi098:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T10:49:34.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:34.334+0000 7f9bbc3c0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-09T10:49:34.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:34.334+0000 7f9bbc3c0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-09T10:49:34.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:34.334+0000 7f9bbc3c0640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T10:49:34.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:34.334+0000 7f9bbc3c0640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T10:49:34.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:34.334+0000 7f9bbc3c0640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T10:49:34.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:34.335+0000 7f9bbc3c0640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-09T10:49:37.188 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-09T10:49:37.217 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-09T10:49:37.248 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T10:49:37.248 INFO:tasks.ceph:role: osd.4 2024-05-09T10:49:37.248 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com 2024-05-09T10:49:37.248 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-09T10:49:37.325 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T10:49:37.325 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T10:49:37.325 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T10:49:37.325 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T10:49:37.325 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T10:49:37.325 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-05-09T10:49:37.325 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T10:49:37.325 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T10:49:37.325 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T10:49:37.325 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T10:49:37.841 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-05-09T10:49:37.844 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-05-09T10:49:37.844 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-09T10:49:37.948 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-09T10:49:37.980 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-09T10:49:38.045 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T10:49:38.045 INFO:tasks.ceph:role: osd.5 2024-05-09T10:49:38.046 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com 2024-05-09T10:49:38.046 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-09T10:49:38.118 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T10:49:38.118 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T10:49:38.118 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T10:49:38.118 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T10:49:38.118 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T10:49:38.119 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-05-09T10:49:38.119 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T10:49:38.119 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T10:49:38.119 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T10:49:38.119 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T10:49:38.589 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-05-09T10:49:38.592 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-05-09T10:49:38.592 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-09T10:49:38.685 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-09T10:49:38.713 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-09T10:49:38.778 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T10:49:38.778 INFO:tasks.ceph:role: osd.6 2024-05-09T10:49:38.778 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com 2024-05-09T10:49:38.779 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-09T10:49:38.849 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T10:49:38.850 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T10:49:38.850 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T10:49:38.850 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T10:49:38.850 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T10:49:38.850 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-05-09T10:49:38.850 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T10:49:38.850 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T10:49:38.850 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T10:49:38.850 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T10:49:39.352 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-05-09T10:49:39.354 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-05-09T10:49:39.354 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-09T10:49:39.493 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-09T10:49:39.525 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-09T10:49:39.591 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T10:49:39.592 INFO:tasks.ceph:role: osd.7 2024-05-09T10:49:39.592 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi105.front.sepia.ceph.com 2024-05-09T10:49:39.592 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-09T10:49:39.665 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T10:49:39.665 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T10:49:39.665 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T10:49:39.665 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T10:49:39.665 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T10:49:39.665 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-05-09T10:49:39.665 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T10:49:39.665 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T10:49:39.666 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T10:49:39.666 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T10:49:40.186 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-05-09T10:49:40.188 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-05-09T10:49:40.188 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-09T10:49:40.330 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-09T10:49:40.366 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T10:49:40.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:40.473+0000 7f4c39901640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-09T10:49:40.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:40.473+0000 7f4c39901640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-09T10:49:40.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:40.473+0000 7f4c39901640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T10:49:40.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:40.473+0000 7f4c39901640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T10:49:40.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:40.473+0000 7f4c39901640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T10:49:40.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:40.474+0000 7f4c39901640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-09T10:49:43.324 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-09T10:49:43.353 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T10:49:43.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:43.452+0000 7f064cbd4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-09T10:49:43.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:43.452+0000 7f064cbd4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-09T10:49:43.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:43.452+0000 7f064cbd4640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T10:49:43.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:43.452+0000 7f064cbd4640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T10:49:43.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:43.452+0000 7f064cbd4640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T10:49:43.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:43.453+0000 7f064cbd4640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-09T10:49:46.293 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-09T10:49:46.323 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T10:49:46.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:46.419+0000 7f524b016640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-09T10:49:46.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:46.419+0000 7f524b016640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-09T10:49:46.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:46.419+0000 7f524b016640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T10:49:46.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:46.419+0000 7f524b016640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T10:49:46.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:46.419+0000 7f524b016640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T10:49:46.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:46.420+0000 7f524b016640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-09T10:49:49.058 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T10:49:49.061 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T10:49:49.268 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-09T10:49:49.294 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T10:49:49.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:49.387+0000 7f85c44ad640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-09T10:49:49.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:49.387+0000 7f85c44ad640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-09T10:49:49.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:49.387+0000 7f85c44ad640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T10:49:49.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:49.387+0000 7f85c44ad640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T10:49:49.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:49.387+0000 7f85c44ad640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T10:49:49.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:49.388+0000 7f85c44ad640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-09T10:49:52.230 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-09T10:49:52.258 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-09T10:49:52.258 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:52.258 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-09T10:49:52.288 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:52.289 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-09T10:49:52.357 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:52.357 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-09T10:49:52.425 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:52.425 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-09T10:49:52.493 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:52.493 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-09T10:49:52.560 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:52.560 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-09T10:49:52.626 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:52.626 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-09T10:49:52.692 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:49:52.692 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-09T10:49:52.722 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:49:52.722 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-09T10:49:52.790 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:49:52.791 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-09T10:49:52.858 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:49:52.858 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-09T10:49:52.924 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:49:52.924 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-09T10:49:52.990 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:49:52.990 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-09T10:49:53.057 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:49:53.057 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-09T10:49:53.123 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:53.123 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-09T10:49:53.139 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:53.140 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-09T10:49:53.197 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:49:53.197 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-09T10:49:53.213 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:49:53.213 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-09T10:49:53.268 INFO:tasks.ceph:Adding keys to all mons... 2024-05-09T10:49:53.268 DEBUG:teuthology.orchestra.run.smithi098:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-09T10:49:53.270 DEBUG:teuthology.orchestra.run.smithi105:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-09T10:49:53.311 INFO:teuthology.orchestra.run.smithi098.stdout:[mgr.y] 2024-05-09T10:49:53.311 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQAxqjxmiTuEGxAAc7n/z9PkjoC3HrAV8BKplw== 2024-05-09T10:49:53.311 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.a] 2024-05-09T10:49:53.312 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQAxqjxmKekgIhAAWaiiDMySfjchssEwwBC7OQ== 2024-05-09T10:49:53.312 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.c] 2024-05-09T10:49:53.312 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQAxqjxmM7wWKRAA/BXGwXIJscgJYJjaSg+BqA== 2024-05-09T10:49:53.312 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.0] 2024-05-09T10:49:53.312 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQA1qjxmUPtjGRAAfaVYD/ym9Ih1QbbgDvDJWg== 2024-05-09T10:49:53.312 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.1] 2024-05-09T10:49:53.312 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQA4qjxmbM9hFxAACR1PQvRZOQ4nRXzL+3FGog== 2024-05-09T10:49:53.312 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.2] 2024-05-09T10:49:53.312 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQA7qjxm/Q93FRAAKgbFv84TwLs6cwe91IN0Rg== 2024-05-09T10:49:53.312 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.3] 2024-05-09T10:49:53.312 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQA+qjxmACv9ExAACD1053vaJzS13heB2SrByQ== 2024-05-09T10:49:53.312 INFO:teuthology.orchestra.run.smithi098.stdout:[mgr.x] 2024-05-09T10:49:53.312 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQAxqjxmDF77HhAAzZTxVNfOjYpxuVCXDJ/Xzg== 2024-05-09T10:49:53.312 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.b] 2024-05-09T10:49:53.313 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQAxqjxmesW1LRAA4vCFcVQ7pP4hB32YyDeMKw== 2024-05-09T10:49:53.313 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.d] 2024-05-09T10:49:53.313 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQAxqjxmRHGyNBAAvcEknZ4L7zuRE3KOo8fpnA== 2024-05-09T10:49:53.313 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.4] 2024-05-09T10:49:53.313 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQBEqjxmaLBEHBAAfQJfAs+6TcDM1KOfzdIlKA== 2024-05-09T10:49:53.313 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.5] 2024-05-09T10:49:53.313 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQBHqjxmAE0BGxAA0LIjdnt3rsNCXKHtr4400Q== 2024-05-09T10:49:53.313 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.6] 2024-05-09T10:49:53.313 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQBKqjxmNjsRGRAAsX+aMYgavwhrMhLBL6bMFw== 2024-05-09T10:49:53.313 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.7] 2024-05-09T10:49:53.313 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQBNqjxmc+MjFxAAtI4Tx9jbZgy8hHOAFbJSAA== 2024-05-09T10:49:53.313 INFO:teuthology.orchestra.run.smithi098.stdout:[client.0] 2024-05-09T10:49:53.313 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQAxqjxmT+bPOBAA4L/A/EK0lI7oydvPJ27lMA== 2024-05-09T10:49:53.313 INFO:teuthology.orchestra.run.smithi098.stdout:[client.2] 2024-05-09T10:49:53.313 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQAyqjxmbyLKAhAAQpXTBuUvHOibVF0c9c0pOw== 2024-05-09T10:49:53.313 INFO:teuthology.orchestra.run.smithi098.stdout:[client.1] 2024-05-09T10:49:53.314 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQAyqjxmhz/zBRAAH8iWmULG5e1d7ZuQoCtIZg== 2024-05-09T10:49:53.314 INFO:teuthology.orchestra.run.smithi098.stdout:[client.3] 2024-05-09T10:49:53.314 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQAyqjxmq0h4CxAAAaZzvfbsvhfYfs01IBWbKw== 2024-05-09T10:49:53.331 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr.y] 2024-05-09T10:49:53.331 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQAxqjxmiTuEGxAAc7n/z9PkjoC3HrAV8BKplw== 2024-05-09T10:49:53.331 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2024-05-09T10:49:53.331 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQAxqjxmKekgIhAAWaiiDMySfjchssEwwBC7OQ== 2024-05-09T10:49:53.331 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2024-05-09T10:49:53.331 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQAxqjxmM7wWKRAA/BXGwXIJscgJYJjaSg+BqA== 2024-05-09T10:49:53.331 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.0] 2024-05-09T10:49:53.331 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQA1qjxmUPtjGRAAfaVYD/ym9Ih1QbbgDvDJWg== 2024-05-09T10:49:53.331 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.1] 2024-05-09T10:49:53.331 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQA4qjxmbM9hFxAACR1PQvRZOQ4nRXzL+3FGog== 2024-05-09T10:49:53.331 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.2] 2024-05-09T10:49:53.332 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQA7qjxm/Q93FRAAKgbFv84TwLs6cwe91IN0Rg== 2024-05-09T10:49:53.332 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.3] 2024-05-09T10:49:53.332 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQA+qjxmACv9ExAACD1053vaJzS13heB2SrByQ== 2024-05-09T10:49:53.332 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr.x] 2024-05-09T10:49:53.332 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQAxqjxmDF77HhAAzZTxVNfOjYpxuVCXDJ/Xzg== 2024-05-09T10:49:53.332 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2024-05-09T10:49:53.332 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQAxqjxmesW1LRAA4vCFcVQ7pP4hB32YyDeMKw== 2024-05-09T10:49:53.332 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2024-05-09T10:49:53.332 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQAxqjxmRHGyNBAAvcEknZ4L7zuRE3KOo8fpnA== 2024-05-09T10:49:53.332 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.4] 2024-05-09T10:49:53.332 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBEqjxmaLBEHBAAfQJfAs+6TcDM1KOfzdIlKA== 2024-05-09T10:49:53.332 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.5] 2024-05-09T10:49:53.332 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBHqjxmAE0BGxAA0LIjdnt3rsNCXKHtr4400Q== 2024-05-09T10:49:53.332 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.6] 2024-05-09T10:49:53.332 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBKqjxmNjsRGRAAsX+aMYgavwhrMhLBL6bMFw== 2024-05-09T10:49:53.332 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.7] 2024-05-09T10:49:53.333 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBNqjxmc+MjFxAAtI4Tx9jbZgy8hHOAFbJSAA== 2024-05-09T10:49:53.333 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2024-05-09T10:49:53.333 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQAxqjxmT+bPOBAA4L/A/EK0lI7oydvPJ27lMA== 2024-05-09T10:49:53.333 INFO:teuthology.orchestra.run.smithi105.stdout:[client.2] 2024-05-09T10:49:53.333 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQAyqjxmbyLKAhAAQpXTBuUvHOibVF0c9c0pOw== 2024-05-09T10:49:53.333 INFO:teuthology.orchestra.run.smithi105.stdout:[client.1] 2024-05-09T10:49:53.333 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQAyqjxmhz/zBRAAH8iWmULG5e1d7ZuQoCtIZg== 2024-05-09T10:49:53.333 INFO:teuthology.orchestra.run.smithi105.stdout:[client.3] 2024-05-09T10:49:53.333 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQAyqjxmq0h4CxAAAaZzvfbsvhfYfs01IBWbKw== 2024-05-09T10:49:53.334 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T10:49:53.357 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T10:49:53.424 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T10:49:53.426 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T10:49:53.479 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T10:49:53.481 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T10:49:53.534 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T10:49:53.537 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T10:49:53.587 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T10:49:53.589 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T10:49:53.640 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T10:49:53.643 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T10:49:53.693 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T10:49:53.695 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T10:49:53.745 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T10:49:53.747 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T10:49:53.798 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T10:49:53.800 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T10:49:53.853 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T10:49:53.858 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T10:49:53.911 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T10:49:53.913 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T10:49:53.964 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T10:49:53.966 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T10:49:54.016 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T10:49:54.019 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T10:49:54.069 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T10:49:54.071 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T10:49:54.122 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T10:49:54.124 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T10:49:54.177 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T10:49:54.180 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T10:49:54.233 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T10:49:54.235 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T10:49:54.288 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T10:49:54.290 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T10:49:54.343 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-09T10:49:54.344 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-09T10:49:54.368 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T10:49:54.463 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-09T10:49:54.488 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-09T10:49:54.513 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T10:49:54.609 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-09T10:49:54.634 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-09T10:49:54.699 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T10:49:54.794 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-09T10:49:54.818 DEBUG:teuthology.orchestra.run.smithi098:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-09T10:49:54.821 DEBUG:teuthology.orchestra.run.smithi105:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-09T10:49:54.873 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-09T10:49:54.873 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-09T10:49:54.873 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-09T10:49:54.876 INFO:tasks.ceph.mon.a:Started 2024-05-09T10:49:54.877 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-09T10:49:54.877 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-09T10:49:54.915 INFO:tasks.ceph.mon.b:Started 2024-05-09T10:49:54.916 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-09T10:49:54.916 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-09T10:49:54.918 INFO:tasks.ceph.mon.c:Started 2024-05-09T10:49:54.918 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-09T10:49:54.918 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-09T10:49:54.918 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-09T10:49:54.921 INFO:tasks.ceph.mgr.y:Started 2024-05-09T10:49:54.921 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-09T10:49:54.921 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-09T10:49:54.923 INFO:tasks.ceph.mgr.x:Started 2024-05-09T10:49:54.923 INFO:tasks.ceph:Setting crush tunables to default 2024-05-09T10:49:54.923 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph osd crush tunables default 2024-05-09T10:49:55.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.029+0000 7fb475322640 1 Processor -- start 2024-05-09T10:49:55.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.029+0000 7fb475322640 1 -- start start 2024-05-09T10:49:55.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.030+0000 7fb475322640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb47015e5d0 0x7fb4700b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:55.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.030+0000 7fb475322640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb47015afe0 0x7fb4700b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:55.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.030+0000 7fb475322640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb47015b4e0 con 0x7fb47015e9b0 2024-05-09T10:49:55.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.030+0000 7fb475322640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb47006abe0 con 0x7fb47015e5d0 2024-05-09T10:49:55.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.030+0000 7fb475322640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb47015b7c0 con 0x7fb47015afe0 2024-05-09T10:49:55.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.030+0000 7fb46effd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb47015e5d0 0x7fb4700b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:55.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.030+0000 7fb46f7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb47015e9b0 0x7fb47015ee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53094/0 (socket says 172.21.15.98:53094) 2024-05-09T10:49:55.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.030+0000 7fb46f7fe640 1 -- 172.21.15.98:0/3222637716 learned_addr learned my addr 172.21.15.98:0/3222637716 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:49:55.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.030+0000 7fb46effd640 1 -- 172.21.15.98:0/3222637716 >> v2:172.21.15.98:3300/0 conn(0x7fb47015e5d0 msgr2=0x7fb4700b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-09T10:49:55.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.030+0000 7fb46effd640 1 -- 172.21.15.98:0/3222637716 >> v2:172.21.15.98:3300/0 conn(0x7fb47015e5d0 msgr2=0x7fb4700b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T10:49:55.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.030+0000 7fb46effd640 1 --2- 172.21.15.98:0/3222637716 >> v2:172.21.15.98:3300/0 conn(0x7fb47015e5d0 0x7fb4700b2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T10:49:55.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.030+0000 7fb46effd640 1 --2- 172.21.15.98:0/3222637716 >> v2:172.21.15.98:3300/0 conn(0x7fb47015e5d0 0x7fb4700b2eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T10:49:55.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.030+0000 7fb46e7fc640 1 --2- 172.21.15.98:0/3222637716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb47015afe0 0x7fb4700b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:55.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.030+0000 7fb46e7fc640 1 -- 172.21.15.98:0/3222637716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb47015afe0 msgr2=0x7fb4700b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T10:49:55.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.030+0000 7fb46e7fc640 1 -- 172.21.15.98:0/3222637716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb47015afe0 msgr2=0x7fb4700b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T10:49:55.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.030+0000 7fb46e7fc640 1 --2- 172.21.15.98:0/3222637716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb47015afe0 0x7fb4700b33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T10:49:55.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.030+0000 7fb46e7fc640 1 --2- 172.21.15.98:0/3222637716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb47015afe0 0x7fb4700b33f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T10:49:55.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.200+0000 7fb46dffb640 1 -- 172.21.15.98:0/3222637716 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3671099671 0 0) 0x7fb47015b4e0 con 0x7fb47015e9b0 2024-05-09T10:49:55.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.200+0000 7fb46dffb640 1 -- 172.21.15.98:0/3222637716 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb45c003520 con 0x7fb47015e9b0 2024-05-09T10:49:55.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.200+0000 7fb46dffb640 1 -- 172.21.15.98:0/3222637716 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 664418472 0 0) 0x7fb45c003520 con 0x7fb47015e9b0 2024-05-09T10:49:55.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.200+0000 7fb46dffb640 1 -- 172.21.15.98:0/3222637716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb47015afe0 msgr2=0x7fb4700b33f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T10:49:55.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.200+0000 7fb46dffb640 1 --2- 172.21.15.98:0/3222637716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb47015afe0 0x7fb4700b33f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:55.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.200+0000 7fb46dffb640 1 -- 172.21.15.98:0/3222637716 >> v2:172.21.15.98:3300/0 conn(0x7fb47015e5d0 msgr2=0x7fb4700b2eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T10:49:55.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.200+0000 7fb46dffb640 1 --2- 172.21.15.98:0/3222637716 >> v2:172.21.15.98:3300/0 conn(0x7fb47015e5d0 0x7fb4700b2eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:55.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.200+0000 7fb46dffb640 1 -- 172.21.15.98:0/3222637716 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47015baa0 con 0x7fb47015e9b0 2024-05-09T10:49:55.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.200+0000 7fb46dffb640 1 -- 172.21.15.98:0/3222637716 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7fb4680019f0 con 0x7fb47015e9b0 2024-05-09T10:49:55.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.200+0000 7fb46dffb640 1 -- 172.21.15.98:0/3222637716 >> v1:172.21.15.98:6789/0 conn(0x7fb47015e9b0 legacy=0x7fb47015ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:55.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.200+0000 7fb46dffb640 1 --2- 172.21.15.98:0/3222637716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb47015afe0 0x7fb45c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:55.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.201+0000 7fb46dffb640 1 --2- 172.21.15.98:0/3222637716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb45c003fe0 0x7fb45c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:55.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.201+0000 7fb46dffb640 1 --2- 172.21.15.98:0/3222637716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb45c004d40 0x7fb45c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:55.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.201+0000 7fb46dffb640 1 -- 172.21.15.98:0/3222637716 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47006abe0 con 0x7fb45c003fe0 2024-05-09T10:49:55.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.201+0000 7fb46dffb640 1 -- 172.21.15.98:0/3222637716 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47015b7c0 con 0x7fb45c004d40 2024-05-09T10:49:55.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.201+0000 7fb46dffb640 1 -- 172.21.15.98:0/3222637716 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c00bac0 con 0x7fb47015afe0 2024-05-09T10:49:55.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.201+0000 7fb46effd640 1 --2- 172.21.15.98:0/3222637716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb47015afe0 0x7fb45c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:55.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.201+0000 7fb46e7fc640 1 --2- 172.21.15.98:0/3222637716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb45c003fe0 0x7fb45c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:55.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.201+0000 7fb46f7fe640 1 --2- 172.21.15.98:0/3222637716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb45c004d40 0x7fb45c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:55.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.201+0000 7fb46e7fc640 1 -- 172.21.15.98:0/3222637716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb47015afe0 msgr2=0x7fb45c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:49:55.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.201+0000 7fb46e7fc640 1 --2- 172.21.15.98:0/3222637716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb47015afe0 0x7fb45c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:55.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.201+0000 7fb46e7fc640 1 -- 172.21.15.98:0/3222637716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb45c004d40 msgr2=0x7fb45c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:49:55.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.201+0000 7fb46e7fc640 1 --2- 172.21.15.98:0/3222637716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb45c004d40 0x7fb45c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:55.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.201+0000 7fb46e7fc640 1 -- 172.21.15.98:0/3222637716 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45c003520 con 0x7fb45c003fe0 2024-05-09T10:49:55.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.202+0000 7fb46e7fc640 1 --2- 172.21.15.98:0/3222637716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb45c003fe0 0x7fb45c004800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb46000b9e0 tx=0x7fb45c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5daaf4ca8eb00a5c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:49:55.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.202+0000 7fb46dffb640 1 -- 172.21.15.98:0/3222637716 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460030070 con 0x7fb45c003fe0 2024-05-09T10:49:55.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.202+0000 7fb46dffb640 1 -- 172.21.15.98:0/3222637716 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46002ada0 con 0x7fb45c003fe0 2024-05-09T10:49:55.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.202+0000 7fb46dffb640 1 -- 172.21.15.98:0/3222637716 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46000b5e0 con 0x7fb45c003fe0 2024-05-09T10:49:55.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.203+0000 7fb475322640 1 -- 172.21.15.98:0/3222637716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb45c003fe0 msgr2=0x7fb45c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:55.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.203+0000 7fb475322640 1 --2- 172.21.15.98:0/3222637716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb45c003fe0 0x7fb45c004800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb46000b9e0 tx=0x7fb45c005830 comp rx=0 tx=0).stop 2024-05-09T10:49:55.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.203+0000 7fb46f7fe640 1 -- 172.21.15.98:0/3222637716 reap_dead start 2024-05-09T10:49:55.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.203+0000 7fb475322640 1 -- 172.21.15.98:0/3222637716 shutdown_connections 2024-05-09T10:49:55.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.203+0000 7fb475322640 1 -- 172.21.15.98:0/3222637716 >> 172.21.15.98:0/3222637716 conn(0x7fb4700b1bb0 msgr2=0x7fb4700b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:49:55.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.203+0000 7fb475322640 1 -- 172.21.15.98:0/3222637716 shutdown_connections 2024-05-09T10:49:55.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.203+0000 7fb475322640 1 -- 172.21.15.98:0/3222637716 wait complete. 2024-05-09T10:49:55.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.204+0000 7fb475322640 1 Processor -- start 2024-05-09T10:49:55.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.204+0000 7fb475322640 1 -- start start 2024-05-09T10:49:55.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.205+0000 7fb475322640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb45c003fe0 0x7fb47014d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:55.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.205+0000 7fb475322640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4701e6830 0x7fb47009cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:55.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.205+0000 7fb475322640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4701e6c90 0x7fb47009d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:55.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.205+0000 7fb475322640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4700b33f0 con 0x7fb4701e6c90 2024-05-09T10:49:55.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.205+0000 7fb475322640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47006e860 con 0x7fb4701e6830 2024-05-09T10:49:55.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.205+0000 7fb475322640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4700b3570 con 0x7fb45c003fe0 2024-05-09T10:49:55.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.205+0000 7fb46f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4701e6c90 0x7fb47009d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:55.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.205+0000 7fb46f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4701e6c90 0x7fb47009d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38486/0 (socket says 172.21.15.98:38486) 2024-05-09T10:49:55.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.205+0000 7fb46e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4701e6830 0x7fb47009cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:55.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.205+0000 7fb46f7fe640 1 -- 172.21.15.98:0/295432507 learned_addr learned my addr 172.21.15.98:0/295432507 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:49:55.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.205+0000 7fb46effd640 1 --2- 172.21.15.98:0/295432507 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb45c003fe0 0x7fb47014d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:55.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.205+0000 7fb46f7fe640 1 -- 172.21.15.98:0/295432507 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb45c003fe0 msgr2=0x7fb47014d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:55.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.205+0000 7fb46f7fe640 1 --2- 172.21.15.98:0/295432507 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb45c003fe0 0x7fb47014d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:55.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.205+0000 7fb46f7fe640 1 -- 172.21.15.98:0/295432507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4701e6830 msgr2=0x7fb47009cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:55.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.205+0000 7fb46f7fe640 1 --2- 172.21.15.98:0/295432507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4701e6830 0x7fb47009cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:55.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.205+0000 7fb46f7fe640 1 -- 172.21.15.98:0/295432507 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47009b6b0 con 0x7fb4701e6c90 2024-05-09T10:49:55.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.206+0000 7fb46f7fe640 1 --2- 172.21.15.98:0/295432507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4701e6c90 0x7fb47009d530 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb4700cc570 tx=0x7fb468007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:49:55.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.206+0000 7fb4677fe640 1 -- 172.21.15.98:0/295432507 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46800ab90 con 0x7fb4701e6c90 2024-05-09T10:49:55.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.206+0000 7fb4677fe640 1 -- 172.21.15.98:0/295432507 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb468018070 con 0x7fb4701e6c90 2024-05-09T10:49:55.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.206+0000 7fb4677fe640 1 -- 172.21.15.98:0/295432507 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4680133f0 con 0x7fb4701e6c90 2024-05-09T10:49:55.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.206+0000 7fb475322640 1 -- 172.21.15.98:0/295432507 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4700ca4f0 con 0x7fb4701e6c90 2024-05-09T10:49:55.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.206+0000 7fb475322640 1 -- 172.21.15.98:0/295432507 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4700caab0 con 0x7fb4701e6c90 2024-05-09T10:49:55.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.206+0000 7fb4677fe640 1 -- 172.21.15.98:0/295432507 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb468013590 con 0x7fb4701e6c90 2024-05-09T10:49:55.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.206+0000 7fb4677fe640 1 -- 172.21.15.98:0/295432507 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb46801da90 con 0x7fb4701e6c90 2024-05-09T10:49:55.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.207+0000 7fb475322640 1 -- 172.21.15.98:0/295432507 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4700b4d00 con 0x7fb4701e6c90 2024-05-09T10:49:55.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.209+0000 7fb4677fe640 1 -- 172.21.15.98:0/295432507 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb4700b4d00 con 0x7fb4701e6c90 2024-05-09T10:49:55.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.275+0000 7fb475322640 1 -- 172.21.15.98:0/295432507 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fb47009e210 con 0x7fb4701e6c90 2024-05-09T10:49:55.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.276+0000 7fb4677fe640 1 -- 172.21.15.98:0/295432507 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fb46801dc90 con 0x7fb4701e6c90 2024-05-09T10:49:55.270 INFO:teuthology.orchestra.run.smithi098.stderr:adjusted tunables profile to default 2024-05-09T10:49:55.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.277+0000 7fb475322640 1 -- 172.21.15.98:0/295432507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4701e6c90 msgr2=0x7fb47009d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:55.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.277+0000 7fb475322640 1 --2- 172.21.15.98:0/295432507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4701e6c90 0x7fb47009d530 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb4700cc570 tx=0x7fb468007b00 comp rx=0 tx=0).stop 2024-05-09T10:49:55.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.277+0000 7fb475322640 1 -- 172.21.15.98:0/295432507 shutdown_connections 2024-05-09T10:49:55.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.277+0000 7fb475322640 1 --2- 172.21.15.98:0/295432507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4701e6c90 0x7fb47009d530 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:55.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.277+0000 7fb475322640 1 --2- 172.21.15.98:0/295432507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4701e6830 0x7fb47009cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:55.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.277+0000 7fb475322640 1 --2- 172.21.15.98:0/295432507 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb45c003fe0 0x7fb47014d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:55.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.277+0000 7fb475322640 1 -- 172.21.15.98:0/295432507 >> 172.21.15.98:0/295432507 conn(0x7fb4700b1bb0 msgr2=0x7fb4700b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:49:55.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.277+0000 7fb475322640 1 -- 172.21.15.98:0/295432507 shutdown_connections 2024-05-09T10:49:55.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:55.277+0000 7fb475322640 1 -- 172.21.15.98:0/295432507 wait complete. 2024-05-09T10:49:55.282 INFO:tasks.ceph:check_enable_crimson: False 2024-05-09T10:49:55.282 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-09T10:49:55.282 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:55.282 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-09T10:49:55.304 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:55.310+0000 7f476571b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T10:49:55.307 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:55.307 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-09T10:49:55.316 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:55.322+0000 7fb2d1723040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T10:49:55.342 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:55.342 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-09T10:49:55.408 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:49:55.408 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-09T10:49:55.473 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:49:55.473 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-09T10:49:55.479 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:55.486+0000 7f476571b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T10:49:55.496 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:55.503+0000 7fb2d1723040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T10:49:55.498 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:49:55.498 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-09T10:49:55.534 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:55.541+0000 7f476571b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T10:49:55.549 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:55.556+0000 7fb2d1723040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T10:49:55.558 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:49:55.558 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-09T10:49:55.623 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:49:55.623 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-09T10:49:55.688 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee 0 2024-05-09T10:49:55.785 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:55.792+0000 7f476571b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T10:49:55.805 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:55.812+0000 7fb2d1723040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T10:49:55.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.835+0000 7fc7a0e93640 1 Processor -- start 2024-05-09T10:49:55.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.836+0000 7fc7a0e93640 1 -- start start 2024-05-09T10:49:55.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.836+0000 7fc7a0e93640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc79c15f2d0 0x7fc79c0d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:55.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.836+0000 7fc7a0e93640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79c0ce0c0 con 0x7fc79c0b40e0 2024-05-09T10:49:55.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.836+0000 7fc7a0e93640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79c0b32a0 con 0x7fc79c0b2610 2024-05-09T10:49:55.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.836+0000 7fc7a0e93640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c0cf200 con 0x7fc79c15f2d0 2024-05-09T10:49:55.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.837+0000 7fc79a59c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc79c15f2d0 0x7fc79c0d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:55.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.837+0000 7fc793fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc79c0b2610 0x7fc79c0ce3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43680/0 (socket says 172.21.15.105:43680) 2024-05-09T10:49:55.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.837+0000 7fc793fff640 1 -- 172.21.15.105:0/1284442601 learned_addr learned my addr 172.21.15.105:0/1284442601 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-05-09T10:49:55.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.837+0000 7fc799d9b640 1 -- 172.21.15.105:0/1284442601 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3502443937 0 0) 0x7fc79c0b32a0 con 0x7fc79c0b2610 2024-05-09T10:49:55.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.837+0000 7fc79a59c640 1 -- 172.21.15.105:0/1284442601 >> v1:172.21.15.105:6789/0 conn(0x7fc79c0b2610 legacy=0x7fc79c0ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:55.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.837+0000 7fc79a59c640 1 -- 172.21.15.105:0/1284442601 >> v1:172.21.15.98:6789/0 conn(0x7fc79c0b40e0 legacy=0x7fc79c0ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:55.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.837+0000 7fc79a59c640 1 -- 172.21.15.105:0/1284442601 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c0b2f70 con 0x7fc79c15f2d0 2024-05-09T10:49:55.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.838+0000 7fc79a59c640 1 --2- 172.21.15.105:0/1284442601 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc79c15f2d0 0x7fc79c0d07c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0097d0 tx=0x7fc78c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d28352a60d67a1f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:49:55.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.838+0000 7fc799d9b640 1 -- 172.21.15.105:0/1284442601 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c037070 con 0x7fc79c15f2d0 2024-05-09T10:49:55.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.838+0000 7fc799d9b640 1 -- 172.21.15.105:0/1284442601 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78c032440 con 0x7fc79c15f2d0 2024-05-09T10:49:55.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.838+0000 7fc799d9b640 1 -- 172.21.15.105:0/1284442601 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c032810 con 0x7fc79c15f2d0 2024-05-09T10:49:55.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.838+0000 7fc7a0e93640 1 -- 172.21.15.105:0/1284442601 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc79c15f2d0 msgr2=0x7fc79c0d07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:55.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.838+0000 7fc7a0e93640 1 --2- 172.21.15.105:0/1284442601 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc79c15f2d0 0x7fc79c0d07c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0097d0 tx=0x7fc78c02a210 comp rx=0 tx=0).stop 2024-05-09T10:49:55.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.838+0000 7fc7a0e93640 1 -- 172.21.15.105:0/1284442601 shutdown_connections 2024-05-09T10:49:55.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.838+0000 7fc7a0e93640 1 --2- 172.21.15.105:0/1284442601 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc79c15f2d0 0x7fc79c0d07c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:55.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.838+0000 7fc7a0e93640 1 -- 172.21.15.105:0/1284442601 >> 172.21.15.105:0/1284442601 conn(0x7fc79c0b35c0 msgr2=0x7fc79c0b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:49:55.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.838+0000 7fc7a0e93640 1 -- 172.21.15.105:0/1284442601 shutdown_connections 2024-05-09T10:49:55.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.838+0000 7fc7a0e93640 1 -- 172.21.15.105:0/1284442601 wait complete. 2024-05-09T10:49:55.834 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:55.840+0000 7f476571b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T10:49:55.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.840+0000 7fc7a0e93640 1 Processor -- start 2024-05-09T10:49:55.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.840+0000 7fc7a0e93640 1 -- start start 2024-05-09T10:49:55.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.840+0000 7fc7a0e93640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc79c0b2610 0x7fc79c15b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:55.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.840+0000 7fc7a0e93640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc79c0b40e0 0x7fc79c0b1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:55.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.840+0000 7fc7a0e93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc79c15f2d0 0x7fc79c147c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:55.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.840+0000 7fc7a0e93640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c166b00 con 0x7fc79c0b40e0 2024-05-09T10:49:55.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.840+0000 7fc7a0e93640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c0cde70 con 0x7fc79c15f2d0 2024-05-09T10:49:55.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.840+0000 7fc7a0e93640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c166c80 con 0x7fc79c0b2610 2024-05-09T10:49:55.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.841+0000 7fc79a59c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc79c0b2610 0x7fc79c15b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:55.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.841+0000 7fc79a59c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc79c0b2610 0x7fc79c15b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:56784/0 (socket says 172.21.15.105:56784) 2024-05-09T10:49:55.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.841+0000 7fc79a59c640 1 -- 172.21.15.105:0/3990928416 learned_addr learned my addr 172.21.15.105:0/3990928416 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-05-09T10:49:55.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.841+0000 7fc79ad9d640 1 --2- 172.21.15.105:0/3990928416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc79c15f2d0 0x7fc79c147c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:55.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.841+0000 7fc793fff640 1 --2- 172.21.15.105:0/3990928416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc79c0b40e0 0x7fc79c0b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:55.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.841+0000 7fc79a59c640 1 -- 172.21.15.105:0/3990928416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc79c15f2d0 msgr2=0x7fc79c147c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:55.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.841+0000 7fc79a59c640 1 --2- 172.21.15.105:0/3990928416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc79c15f2d0 0x7fc79c147c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:55.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.841+0000 7fc79a59c640 1 -- 172.21.15.105:0/3990928416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc79c0b40e0 msgr2=0x7fc79c0b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:55.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.841+0000 7fc79a59c640 1 --2- 172.21.15.105:0/3990928416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc79c0b40e0 0x7fc79c0b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:55.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.841+0000 7fc79a59c640 1 -- 172.21.15.105:0/3990928416 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c06be70 con 0x7fc79c0b2610 2024-05-09T10:49:55.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.841+0000 7fc79a59c640 1 --2- 172.21.15.105:0/3990928416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc79c0b2610 0x7fc79c15b570 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc78c035040 tx=0x7fc78c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:49:55.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.841+0000 7fc792ffd640 1 -- 172.21.15.105:0/3990928416 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c040020 con 0x7fc79c0b2610 2024-05-09T10:49:55.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.841+0000 7fc792ffd640 1 -- 172.21.15.105:0/3990928416 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78c038040 con 0x7fc79c0b2610 2024-05-09T10:49:55.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.841+0000 7fc7a0e93640 1 -- 172.21.15.105:0/3990928416 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc79c06c150 con 0x7fc79c0b2610 2024-05-09T10:49:55.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.841+0000 7fc792ffd640 1 -- 172.21.15.105:0/3990928416 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c037070 con 0x7fc79c0b2610 2024-05-09T10:49:55.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.841+0000 7fc7a0e93640 1 -- 172.21.15.105:0/3990928416 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc79c06c710 con 0x7fc79c0b2610 2024-05-09T10:49:55.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.841+0000 7fc792ffd640 1 -- 172.21.15.105:0/3990928416 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc78c043020 con 0x7fc79c0b2610 2024-05-09T10:49:55.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.841+0000 7fc792ffd640 1 -- 172.21.15.105:0/3990928416 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc78c03c400 con 0x7fc79c0b2610 2024-05-09T10:49:55.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.842+0000 7fc7a0e93640 1 -- 172.21.15.105:0/3990928416 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc79c09fc20 con 0x7fc79c0b2610 2024-05-09T10:49:55.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.844+0000 7fc792ffd640 1 -- 172.21.15.105:0/3990928416 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc78c04a030 con 0x7fc79c0b2610 2024-05-09T10:49:55.852 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:55.858+0000 7fb2d1723040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T10:49:55.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.915+0000 7fc7a0e93640 1 -- 172.21.15.105:0/3990928416 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee", "id": 0} v 0) v1 -- 0x7fc79c04f000 con 0x7fc79c0b2610 2024-05-09T10:49:55.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.921+0000 7fc792ffd640 1 -- 172.21.15.105:0/3990928416 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc78c030d40 con 0x7fc79c0b2610 2024-05-09T10:49:55.914 INFO:teuthology.orchestra.run.smithi105.stdout:0 2024-05-09T10:49:55.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.922+0000 7fc7a0e93640 1 -- 172.21.15.105:0/3990928416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc79c0b2610 msgr2=0x7fc79c15b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:55.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.922+0000 7fc7a0e93640 1 --2- 172.21.15.105:0/3990928416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc79c0b2610 0x7fc79c15b570 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc78c035040 tx=0x7fc78c0046c0 comp rx=0 tx=0).stop 2024-05-09T10:49:55.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.922+0000 7fc7a0e93640 1 -- 172.21.15.105:0/3990928416 shutdown_connections 2024-05-09T10:49:55.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.922+0000 7fc7a0e93640 1 --2- 172.21.15.105:0/3990928416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc79c15f2d0 0x7fc79c147c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:55.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.922+0000 7fc7a0e93640 1 --2- 172.21.15.105:0/3990928416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc79c0b40e0 0x7fc79c0b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:55.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.922+0000 7fc7a0e93640 1 --2- 172.21.15.105:0/3990928416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc79c0b2610 0x7fc79c15b570 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:55.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.922+0000 7fc7a0e93640 1 -- 172.21.15.105:0/3990928416 >> 172.21.15.105:0/3990928416 conn(0x7fc79c0b35c0 msgr2=0x7fc79c0b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:49:55.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.922+0000 7fc7a0e93640 1 -- 172.21.15.105:0/3990928416 shutdown_connections 2024-05-09T10:49:55.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:55.922+0000 7fc7a0e93640 1 -- 172.21.15.105:0/3990928416 wait complete. 2024-05-09T10:49:55.926 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new 7ab030de-0196-40aa-a8f4-88b7ec5c6bd0 1 2024-05-09T10:49:56.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.031+0000 7f2d47e9c640 1 Processor -- start 2024-05-09T10:49:56.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.032+0000 7f2d47e9c640 1 -- start start 2024-05-09T10:49:56.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.032+0000 7f2d47e9c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2d4015f270 0x7f2d400cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.032+0000 7f2d47e9c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2d4006b2f0 0x7f2d4006d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.033+0000 7f2d47e9c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d400afc00 con 0x7f2d400af770 2024-05-09T10:49:56.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.033+0000 7f2d47e9c640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d400cd830 con 0x7f2d4015f270 2024-05-09T10:49:56.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.033+0000 7f2d47e9c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d400cf720 con 0x7f2d4006b2f0 2024-05-09T10:49:56.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.033+0000 7f2d46439640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2d4006b2f0 0x7f2d4006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.033+0000 7f2d46439640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2d4006b2f0 0x7f2d4006d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45454/0 (socket says 172.21.15.105:45454) 2024-05-09T10:49:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.033+0000 7f2d46439640 1 -- 172.21.15.105:0/2682429670 learned_addr learned my addr 172.21.15.105:0/2682429670 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-05-09T10:49:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.033+0000 7f2d45c38640 1 --2- 172.21.15.105:0/2682429670 >> v2:172.21.15.98:3300/0 conn(0x7f2d4015f270 0x7f2d400cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.033+0000 7f2d46439640 1 -- 172.21.15.105:0/2682429670 >> v2:172.21.15.98:3300/0 conn(0x7f2d4015f270 msgr2=0x7f2d400cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:49:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.033+0000 7f2d46439640 1 --2- 172.21.15.105:0/2682429670 >> v2:172.21.15.98:3300/0 conn(0x7f2d4015f270 0x7f2d400cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.033+0000 7f2d46439640 1 -- 172.21.15.105:0/2682429670 >> v1:172.21.15.98:6789/0 conn(0x7f2d400af770 legacy=0x7f2d4006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.033+0000 7f2d46439640 1 -- 172.21.15.105:0/2682429670 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d400afef0 con 0x7f2d4006b2f0 2024-05-09T10:49:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.033+0000 7f2d45c38640 1 --2- 172.21.15.105:0/2682429670 >> v2:172.21.15.98:3300/0 conn(0x7f2d4015f270 0x7f2d400cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:49:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.034+0000 7f2d46439640 1 --2- 172.21.15.105:0/2682429670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d4006b2f0 0x7f2d4006d390 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2d40071010 tx=0x7f2d3802a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ea902f67960b882 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:49:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.034+0000 7f2d44c36640 1 -- 172.21.15.105:0/2682429670 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d38036070 con 0x7f2d4006b2f0 2024-05-09T10:49:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.034+0000 7f2d44c36640 1 -- 172.21.15.105:0/2682429670 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d38031600 con 0x7f2d4006b2f0 2024-05-09T10:49:56.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.034+0000 7f2d44c36640 1 -- 172.21.15.105:0/2682429670 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d380319d0 con 0x7f2d4006b2f0 2024-05-09T10:49:56.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.034+0000 7f2d47e9c640 1 -- 172.21.15.105:0/2682429670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d4006b2f0 msgr2=0x7f2d4006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.034+0000 7f2d47e9c640 1 --2- 172.21.15.105:0/2682429670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d4006b2f0 0x7f2d4006d390 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2d40071010 tx=0x7f2d3802a730 comp rx=0 tx=0).stop 2024-05-09T10:49:56.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.034+0000 7f2d47e9c640 1 -- 172.21.15.105:0/2682429670 shutdown_connections 2024-05-09T10:49:56.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.034+0000 7f2d47e9c640 1 --2- 172.21.15.105:0/2682429670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d4006b2f0 0x7f2d4006d390 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.034+0000 7f2d47e9c640 1 --2- 172.21.15.105:0/2682429670 >> v2:172.21.15.98:3300/0 conn(0x7f2d4015f270 0x7f2d400cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.034+0000 7f2d47e9c640 1 -- 172.21.15.105:0/2682429670 >> 172.21.15.105:0/2682429670 conn(0x7f2d400d0860 msgr2=0x7f2d400d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:49:56.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.034+0000 7f2d47e9c640 1 -- 172.21.15.105:0/2682429670 shutdown_connections 2024-05-09T10:49:56.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.035+0000 7f2d47e9c640 1 -- 172.21.15.105:0/2682429670 wait complete. 2024-05-09T10:49:56.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.036+0000 7f2d47e9c640 1 Processor -- start 2024-05-09T10:49:56.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.036+0000 7f2d47e9c640 1 -- start start 2024-05-09T10:49:56.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.036+0000 7f2d47e9c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d4006b2f0 0x7f2d401590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.036+0000 7f2d47e9c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d4015f270 0x7f2d400b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.036+0000 7f2d47e9c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d400b4b00 0x7f2d401f7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.036+0000 7f2d47e9c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d400d1610 con 0x7f2d400b4b00 2024-05-09T10:49:56.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.036+0000 7f2d47e9c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d400cf970 con 0x7f2d4006b2f0 2024-05-09T10:49:56.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.036+0000 7f2d47e9c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d400cfe90 con 0x7f2d4015f270 2024-05-09T10:49:56.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.037+0000 7f2d45c38640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d4006b2f0 0x7f2d401590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.037+0000 7f2d45437640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d4015f270 0x7f2d400b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.037+0000 7f2d45c38640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d4006b2f0 0x7f2d401590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45466/0 (socket says 172.21.15.105:45466) 2024-05-09T10:49:56.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.037+0000 7f2d45c38640 1 -- 172.21.15.105:0/3718983888 learned_addr learned my addr 172.21.15.105:0/3718983888 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-05-09T10:49:56.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.037+0000 7f2d46439640 1 --2- 172.21.15.105:0/3718983888 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d400b4b00 0x7f2d401f7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.037+0000 7f2d45c38640 1 -- 172.21.15.105:0/3718983888 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d4015f270 msgr2=0x7f2d400b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.037+0000 7f2d45c38640 1 --2- 172.21.15.105:0/3718983888 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d4015f270 0x7f2d400b45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.037+0000 7f2d45c38640 1 -- 172.21.15.105:0/3718983888 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d400b4b00 msgr2=0x7f2d401f7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.037+0000 7f2d45c38640 1 --2- 172.21.15.105:0/3718983888 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d400b4b00 0x7f2d401f7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.037+0000 7f2d45c38640 1 -- 172.21.15.105:0/3718983888 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4014ef00 con 0x7f2d4006b2f0 2024-05-09T10:49:56.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.037+0000 7f2d45437640 1 --2- 172.21.15.105:0/3718983888 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d4015f270 0x7f2d400b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:49:56.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.037+0000 7f2d45c38640 1 --2- 172.21.15.105:0/3718983888 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d4006b2f0 0x7f2d401590e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00d800 tx=0x7f2d3c00dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:49:56.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.037+0000 7f2d367fc640 1 -- 172.21.15.105:0/3718983888 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c004030 con 0x7f2d4006b2f0 2024-05-09T10:49:56.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.037+0000 7f2d367fc640 1 -- 172.21.15.105:0/3718983888 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3c0041d0 con 0x7f2d4006b2f0 2024-05-09T10:49:56.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.037+0000 7f2d47e9c640 1 -- 172.21.15.105:0/3718983888 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d40158040 con 0x7f2d4006b2f0 2024-05-09T10:49:56.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.037+0000 7f2d367fc640 1 -- 172.21.15.105:0/3718983888 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c0096a0 con 0x7f2d4006b2f0 2024-05-09T10:49:56.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.037+0000 7f2d47e9c640 1 -- 172.21.15.105:0/3718983888 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d40158600 con 0x7f2d4006b2f0 2024-05-09T10:49:56.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.037+0000 7f2d367fc640 1 -- 172.21.15.105:0/3718983888 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f2d3c009840 con 0x7f2d4006b2f0 2024-05-09T10:49:56.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.037+0000 7f2d367fc640 1 -- 172.21.15.105:0/3718983888 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f2d3c0147e0 con 0x7f2d4006b2f0 2024-05-09T10:49:56.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.038+0000 7f2d47e9c640 1 -- 172.21.15.105:0/3718983888 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d400673a0 con 0x7f2d4006b2f0 2024-05-09T10:49:56.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.040+0000 7f2d367fc640 1 -- 172.21.15.105:0/3718983888 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f2d3c009c20 con 0x7f2d4006b2f0 2024-05-09T10:49:56.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.112+0000 7f2d47e9c640 1 -- 172.21.15.105:0/3718983888 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7ab030de-0196-40aa-a8f4-88b7ec5c6bd0", "id": 1} v 0) v1 -- 0x7f2d40072f30 con 0x7f2d4006b2f0 2024-05-09T10:49:56.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.121+0000 7f2d367fc640 1 -- 172.21.15.105:0/3718983888 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7ab030de-0196-40aa-a8f4-88b7ec5c6bd0", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2d3c009e00 con 0x7f2d4006b2f0 2024-05-09T10:49:56.114 INFO:teuthology.orchestra.run.smithi105.stdout:1 2024-05-09T10:49:56.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.122+0000 7f2d47e9c640 1 -- 172.21.15.105:0/3718983888 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d4006b2f0 msgr2=0x7f2d401590e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.122+0000 7f2d47e9c640 1 --2- 172.21.15.105:0/3718983888 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d4006b2f0 0x7f2d401590e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00d800 tx=0x7f2d3c00dcc0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.122+0000 7f2d47e9c640 1 -- 172.21.15.105:0/3718983888 shutdown_connections 2024-05-09T10:49:56.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.122+0000 7f2d47e9c640 1 --2- 172.21.15.105:0/3718983888 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d400b4b00 0x7f2d401f7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.122+0000 7f2d47e9c640 1 --2- 172.21.15.105:0/3718983888 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d4015f270 0x7f2d400b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.122+0000 7f2d47e9c640 1 --2- 172.21.15.105:0/3718983888 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d4006b2f0 0x7f2d401590e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.122+0000 7f2d47e9c640 1 -- 172.21.15.105:0/3718983888 >> 172.21.15.105:0/3718983888 conn(0x7f2d400d0860 msgr2=0x7f2d4006c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:49:56.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.122+0000 7f2d47e9c640 1 -- 172.21.15.105:0/3718983888 shutdown_connections 2024-05-09T10:49:56.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.122+0000 7f2d47e9c640 1 -- 172.21.15.105:0/3718983888 wait complete. 2024-05-09T10:49:56.126 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new a086d01b-45f0-4681-8691-143b9be38503 2 2024-05-09T10:49:56.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.234+0000 7faacbc6a640 1 Processor -- start 2024-05-09T10:49:56.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.234+0000 7faacbc6a640 1 -- start start 2024-05-09T10:49:56.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.235+0000 7faacbc6a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7faac406f8a0 0x7faac40cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.235+0000 7faacbc6a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faac4071b00 0x7faac40b4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.235+0000 7faacbc6a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac40b3e60 con 0x7faac40afcb0 2024-05-09T10:49:56.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.235+0000 7faacbc6a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac40b3500 con 0x7faac406f8a0 2024-05-09T10:49:56.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.235+0000 7faacbc6a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac40d09a0 con 0x7faac4071b00 2024-05-09T10:49:56.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.235+0000 7faaca207640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faac4071b00 0x7faac40b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.235+0000 7faac9205640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7faac40afcb0 0x7faac40713f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43688/0 (socket says 172.21.15.105:43688) 2024-05-09T10:49:56.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.235+0000 7faac9205640 1 -- 172.21.15.105:0/1417697260 learned_addr learned my addr 172.21.15.105:0/1417697260 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-05-09T10:49:56.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.235+0000 7faac9a06640 1 --2- 172.21.15.105:0/1417697260 >> v2:172.21.15.98:3300/0 conn(0x7faac406f8a0 0x7faac40cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.235+0000 7faac8a04640 1 -- 172.21.15.105:0/1417697260 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2605357649 0 0) 0x7faac40b3e60 con 0x7faac40afcb0 2024-05-09T10:49:56.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.235+0000 7faaca207640 1 -- 172.21.15.105:0/1417697260 >> v2:172.21.15.98:3300/0 conn(0x7faac406f8a0 msgr2=0x7faac40cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:49:56.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.235+0000 7faaca207640 1 --2- 172.21.15.105:0/1417697260 >> v2:172.21.15.98:3300/0 conn(0x7faac406f8a0 0x7faac40cbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.235+0000 7faaca207640 1 -- 172.21.15.105:0/1417697260 >> v1:172.21.15.105:6789/0 conn(0x7faac40afcb0 legacy=0x7faac40713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.235+0000 7faaca207640 1 -- 172.21.15.105:0/1417697260 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac40d0c80 con 0x7faac4071b00 2024-05-09T10:49:56.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.235+0000 7faac9a06640 1 --2- 172.21.15.105:0/1417697260 >> v2:172.21.15.98:3300/0 conn(0x7faac406f8a0 0x7faac40cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:49:56.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.236+0000 7faaca207640 1 --2- 172.21.15.105:0/1417697260 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faac4071b00 0x7faac40b4340 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faabc009d90 tx=0x7faabc02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e9a8b57edfbb713 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:49:56.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.236+0000 7faac8a04640 1 -- 172.21.15.105:0/1417697260 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faabc007740 con 0x7faac4071b00 2024-05-09T10:49:56.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.236+0000 7faac8a04640 1 -- 172.21.15.105:0/1417697260 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faabc0078e0 con 0x7faac4071b00 2024-05-09T10:49:56.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.236+0000 7faac8a04640 1 -- 172.21.15.105:0/1417697260 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faabc007cb0 con 0x7faac4071b00 2024-05-09T10:49:56.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.237+0000 7faacbc6a640 1 -- 172.21.15.105:0/1417697260 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faac4071b00 msgr2=0x7faac40b4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.237+0000 7faacbc6a640 1 --2- 172.21.15.105:0/1417697260 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faac4071b00 0x7faac40b4340 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faabc009d90 tx=0x7faabc02a660 comp rx=0 tx=0).stop 2024-05-09T10:49:56.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.237+0000 7faacbc6a640 1 -- 172.21.15.105:0/1417697260 shutdown_connections 2024-05-09T10:49:56.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.237+0000 7faacbc6a640 1 --2- 172.21.15.105:0/1417697260 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faac4071b00 0x7faac40b4340 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.237+0000 7faacbc6a640 1 --2- 172.21.15.105:0/1417697260 >> v2:172.21.15.98:3300/0 conn(0x7faac406f8a0 0x7faac40cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.237+0000 7faacbc6a640 1 -- 172.21.15.105:0/1417697260 >> 172.21.15.105:0/1417697260 conn(0x7faac40b25a0 msgr2=0x7faac40b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:49:56.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.237+0000 7faacbc6a640 1 -- 172.21.15.105:0/1417697260 shutdown_connections 2024-05-09T10:49:56.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.237+0000 7faacbc6a640 1 -- 172.21.15.105:0/1417697260 wait complete. 2024-05-09T10:49:56.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.238+0000 7faacbc6a640 1 Processor -- start 2024-05-09T10:49:56.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.238+0000 7faacbc6a640 1 -- start start 2024-05-09T10:49:56.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.239+0000 7faacbc6a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faac406f8a0 0x7faac4150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.239+0000 7faacbc6a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac4071b00 0x7faac41531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.239+0000 7faacbc6a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faac40afcb0 0x7faac41536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.239+0000 7faacbc6a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac406fc80 con 0x7faac4071b00 2024-05-09T10:49:56.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.239+0000 7faacbc6a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac40cc270 con 0x7faac406f8a0 2024-05-09T10:49:56.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.239+0000 7faacbc6a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac406fe00 con 0x7faac40afcb0 2024-05-09T10:49:56.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.239+0000 7faac9a06640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faac406f8a0 0x7faac4150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.239+0000 7faaca207640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faac40afcb0 0x7faac41536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.239+0000 7faac9a06640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faac406f8a0 0x7faac4150330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45478/0 (socket says 172.21.15.105:45478) 2024-05-09T10:49:56.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.239+0000 7faaca207640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faac40afcb0 0x7faac41536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:56808/0 (socket says 172.21.15.105:56808) 2024-05-09T10:49:56.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.239+0000 7faac9a06640 1 -- 172.21.15.105:0/2923289119 learned_addr learned my addr 172.21.15.105:0/2923289119 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-05-09T10:49:56.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.239+0000 7faac9205640 1 --2- 172.21.15.105:0/2923289119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac4071b00 0x7faac41531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.239+0000 7faac9a06640 1 -- 172.21.15.105:0/2923289119 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faac40afcb0 msgr2=0x7faac41536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.239+0000 7faac9a06640 1 --2- 172.21.15.105:0/2923289119 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faac40afcb0 0x7faac41536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.239+0000 7faac9a06640 1 -- 172.21.15.105:0/2923289119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac4071b00 msgr2=0x7faac41531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.239+0000 7faac9a06640 1 --2- 172.21.15.105:0/2923289119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac4071b00 0x7faac41531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.239+0000 7faac9a06640 1 -- 172.21.15.105:0/2923289119 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac406e220 con 0x7faac406f8a0 2024-05-09T10:49:56.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.239+0000 7faaca207640 1 --2- 172.21.15.105:0/2923289119 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faac40afcb0 0x7faac41536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:49:56.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.239+0000 7faac9a06640 1 --2- 172.21.15.105:0/2923289119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faac406f8a0 0x7faac4150330 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faac000d7d0 tx=0x7faac000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:49:56.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.239+0000 7faaba7fc640 1 -- 172.21.15.105:0/2923289119 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac0004030 con 0x7faac406f8a0 2024-05-09T10:49:56.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.239+0000 7faaba7fc640 1 -- 172.21.15.105:0/2923289119 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac00041d0 con 0x7faac406f8a0 2024-05-09T10:49:56.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.239+0000 7faacbc6a640 1 -- 172.21.15.105:0/2923289119 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac406e4c0 con 0x7faac406f8a0 2024-05-09T10:49:56.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.240+0000 7faaba7fc640 1 -- 172.21.15.105:0/2923289119 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac000b6c0 con 0x7faac406f8a0 2024-05-09T10:49:56.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.240+0000 7faacbc6a640 1 -- 172.21.15.105:0/2923289119 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac406ea80 con 0x7faac406f8a0 2024-05-09T10:49:56.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.240+0000 7faaba7fc640 1 -- 172.21.15.105:0/2923289119 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7faac000b860 con 0x7faac406f8a0 2024-05-09T10:49:56.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.240+0000 7faaba7fc640 1 -- 172.21.15.105:0/2923289119 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7faac0003dc0 con 0x7faac406f8a0 2024-05-09T10:49:56.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.240+0000 7faacbc6a640 1 -- 172.21.15.105:0/2923289119 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac404f000 con 0x7faac406f8a0 2024-05-09T10:49:56.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.243+0000 7faaba7fc640 1 -- 172.21.15.105:0/2923289119 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7faac000bce0 con 0x7faac406f8a0 2024-05-09T10:49:56.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.315+0000 7faacbc6a640 1 -- 172.21.15.105:0/2923289119 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a086d01b-45f0-4681-8691-143b9be38503", "id": 2} v 0) v1 -- 0x7faac406bab0 con 0x7faac406f8a0 2024-05-09T10:49:56.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.320+0000 7faaba7fc640 1 -- 172.21.15.105:0/2923289119 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a086d01b-45f0-4681-8691-143b9be38503", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7faac0003610 con 0x7faac406f8a0 2024-05-09T10:49:56.313 INFO:teuthology.orchestra.run.smithi105.stdout:2 2024-05-09T10:49:56.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.321+0000 7faacbc6a640 1 -- 172.21.15.105:0/2923289119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faac406f8a0 msgr2=0x7faac4150330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.321+0000 7faacbc6a640 1 --2- 172.21.15.105:0/2923289119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faac406f8a0 0x7faac4150330 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faac000d7d0 tx=0x7faac000dc90 comp rx=0 tx=0).stop 2024-05-09T10:49:56.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.321+0000 7faacbc6a640 1 -- 172.21.15.105:0/2923289119 shutdown_connections 2024-05-09T10:49:56.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.321+0000 7faacbc6a640 1 --2- 172.21.15.105:0/2923289119 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faac40afcb0 0x7faac41536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.321+0000 7faacbc6a640 1 --2- 172.21.15.105:0/2923289119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac4071b00 0x7faac41531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.321+0000 7faacbc6a640 1 --2- 172.21.15.105:0/2923289119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faac406f8a0 0x7faac4150330 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.321+0000 7faacbc6a640 1 -- 172.21.15.105:0/2923289119 >> 172.21.15.105:0/2923289119 conn(0x7faac40b25a0 msgr2=0x7faac40cd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:49:56.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.321+0000 7faacbc6a640 1 -- 172.21.15.105:0/2923289119 shutdown_connections 2024-05-09T10:49:56.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.321+0000 7faacbc6a640 1 -- 172.21.15.105:0/2923289119 wait complete. 2024-05-09T10:49:56.325 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new fbaae71d-5472-4203-9f21-bf9affa127f5 3 2024-05-09T10:49:56.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.432+0000 7fd91eeac640 1 Processor -- start 2024-05-09T10:49:56.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.432+0000 7fd91eeac640 1 -- start start 2024-05-09T10:49:56.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.433+0000 7fd91eeac640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd91815e000 0x7fd9180ca430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.433+0000 7fd91eeac640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd91806f430 0x7fd91806f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.433+0000 7fd91eeac640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd918161b10 con 0x7fd9180ce750 2024-05-09T10:49:56.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.433+0000 7fd91eeac640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd91815e3e0 con 0x7fd91815e000 2024-05-09T10:49:56.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.433+0000 7fd91eeac640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd918161e40 con 0x7fd91806f430 2024-05-09T10:49:56.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.433+0000 7fd91d449640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd91806f430 0x7fd91806f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.433+0000 7fd917fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd9180ce750 0x7fd9181622d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43694/0 (socket says 172.21.15.105:43694) 2024-05-09T10:49:56.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.433+0000 7fd917fff640 1 -- 172.21.15.105:0/751962626 learned_addr learned my addr 172.21.15.105:0/751962626 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-05-09T10:49:56.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.433+0000 7fd91cc48640 1 --2- 172.21.15.105:0/751962626 >> v2:172.21.15.98:3300/0 conn(0x7fd91815e000 0x7fd9180ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.434+0000 7fd9177fe640 1 -- 172.21.15.105:0/751962626 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3276889683 0 0) 0x7fd918161b10 con 0x7fd9180ce750 2024-05-09T10:49:56.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.434+0000 7fd9177fe640 1 -- 172.21.15.105:0/751962626 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd908003520 con 0x7fd9180ce750 2024-05-09T10:49:56.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.434+0000 7fd91d449640 1 -- 172.21.15.105:0/751962626 >> v2:172.21.15.98:3300/0 conn(0x7fd91815e000 msgr2=0x7fd9180ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:49:56.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.434+0000 7fd91d449640 1 --2- 172.21.15.105:0/751962626 >> v2:172.21.15.98:3300/0 conn(0x7fd91815e000 0x7fd9180ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.434+0000 7fd91d449640 1 -- 172.21.15.105:0/751962626 >> v1:172.21.15.105:6789/0 conn(0x7fd9180ce750 legacy=0x7fd9181622d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.434+0000 7fd91d449640 1 -- 172.21.15.105:0/751962626 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9180cec50 con 0x7fd91806f430 2024-05-09T10:49:56.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.435+0000 7fd91d449640 1 --2- 172.21.15.105:0/751962626 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd91806f430 0x7fd91806f8c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd910009d90 tx=0x7fd91002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37851a52a8f2d13e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:49:56.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.435+0000 7fd9177fe640 1 -- 172.21.15.105:0/751962626 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910007740 con 0x7fd91806f430 2024-05-09T10:49:56.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.435+0000 7fd9177fe640 1 -- 172.21.15.105:0/751962626 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9100078e0 con 0x7fd91806f430 2024-05-09T10:49:56.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.435+0000 7fd9177fe640 1 -- 172.21.15.105:0/751962626 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910007cb0 con 0x7fd91806f430 2024-05-09T10:49:56.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.436+0000 7fd91eeac640 1 -- 172.21.15.105:0/751962626 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd91806f430 msgr2=0x7fd91806f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.436+0000 7fd91eeac640 1 --2- 172.21.15.105:0/751962626 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd91806f430 0x7fd91806f8c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd910009d90 tx=0x7fd91002a660 comp rx=0 tx=0).stop 2024-05-09T10:49:56.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.436+0000 7fd91eeac640 1 -- 172.21.15.105:0/751962626 shutdown_connections 2024-05-09T10:49:56.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.436+0000 7fd91eeac640 1 --2- 172.21.15.105:0/751962626 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd91806f430 0x7fd91806f8c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.436+0000 7fd91eeac640 1 --2- 172.21.15.105:0/751962626 >> v2:172.21.15.98:3300/0 conn(0x7fd91815e000 0x7fd9180ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.436+0000 7fd91eeac640 1 -- 172.21.15.105:0/751962626 >> 172.21.15.105:0/751962626 conn(0x7fd9180b25a0 msgr2=0x7fd9180b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:49:56.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.436+0000 7fd91eeac640 1 -- 172.21.15.105:0/751962626 shutdown_connections 2024-05-09T10:49:56.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.436+0000 7fd91eeac640 1 -- 172.21.15.105:0/751962626 wait complete. 2024-05-09T10:49:56.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.437+0000 7fd91eeac640 1 Processor -- start 2024-05-09T10:49:56.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.437+0000 7fd91eeac640 1 -- start start 2024-05-09T10:49:56.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.438+0000 7fd91eeac640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd91806f430 0x7fd918158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.438+0000 7fd91eeac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9180ce750 0x7fd9180b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.438+0000 7fd91eeac640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd91815e000 0x7fd9180b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.438+0000 7fd91eeac640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180cf390 con 0x7fd91815e000 2024-05-09T10:49:56.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.438+0000 7fd91eeac640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180cf210 con 0x7fd9180ce750 2024-05-09T10:49:56.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.438+0000 7fd91eeac640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180b0190 con 0x7fd91806f430 2024-05-09T10:49:56.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.438+0000 7fd91cc48640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd91806f430 0x7fd918158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.438+0000 7fd917fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9180ce750 0x7fd9180b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.438+0000 7fd91cc48640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd91806f430 0x7fd918158710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:56814/0 (socket says 172.21.15.105:56814) 2024-05-09T10:49:56.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.438+0000 7fd91cc48640 1 -- 172.21.15.105:0/65578373 learned_addr learned my addr 172.21.15.105:0/65578373 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-05-09T10:49:56.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.438+0000 7fd91d449640 1 --2- 172.21.15.105:0/65578373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd91815e000 0x7fd9180b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.438+0000 7fd91cc48640 1 -- 172.21.15.105:0/65578373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9180ce750 msgr2=0x7fd9180b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.438+0000 7fd91cc48640 1 --2- 172.21.15.105:0/65578373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9180ce750 0x7fd9180b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.438+0000 7fd91cc48640 1 -- 172.21.15.105:0/65578373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd91815e000 msgr2=0x7fd9180b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.438+0000 7fd91cc48640 1 --2- 172.21.15.105:0/65578373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd91815e000 0x7fd9180b1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.438+0000 7fd91cc48640 1 -- 172.21.15.105:0/65578373 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd918158c50 con 0x7fd91806f430 2024-05-09T10:49:56.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.438+0000 7fd917fff640 1 --2- 172.21.15.105:0/65578373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9180ce750 0x7fd9180b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:49:56.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.438+0000 7fd91cc48640 1 --2- 172.21.15.105:0/65578373 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd91806f430 0x7fd918158710 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00e870 tx=0x7fd90c00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:49:56.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.438+0000 7fd9157fa640 1 -- 172.21.15.105:0/65578373 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c017070 con 0x7fd91806f430 2024-05-09T10:49:56.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.438+0000 7fd9157fa640 1 -- 172.21.15.105:0/65578373 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c00c940 con 0x7fd91806f430 2024-05-09T10:49:56.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.438+0000 7fd9157fa640 1 -- 172.21.15.105:0/65578373 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00cd30 con 0x7fd91806f430 2024-05-09T10:49:56.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.439+0000 7fd91eeac640 1 -- 172.21.15.105:0/65578373 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9181f7770 con 0x7fd91806f430 2024-05-09T10:49:56.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.439+0000 7fd91eeac640 1 -- 172.21.15.105:0/65578373 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9181f7d30 con 0x7fd91806f430 2024-05-09T10:49:56.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.439+0000 7fd9157fa640 1 -- 172.21.15.105:0/65578373 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd90c01a070 con 0x7fd91806f430 2024-05-09T10:49:56.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.439+0000 7fd9157fa640 1 -- 172.21.15.105:0/65578373 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fd90c013470 con 0x7fd91806f430 2024-05-09T10:49:56.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.440+0000 7fd91eeac640 1 -- 172.21.15.105:0/65578373 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd91804f000 con 0x7fd91806f430 2024-05-09T10:49:56.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.442+0000 7fd9157fa640 1 -- 172.21.15.105:0/65578373 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd90c006020 con 0x7fd91806f430 2024-05-09T10:49:56.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.512+0000 7fd91eeac640 1 -- 172.21.15.105:0/65578373 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fbaae71d-5472-4203-9f21-bf9affa127f5", "id": 3} v 0) v1 -- 0x7fd9180c9dd0 con 0x7fd91806f430 2024-05-09T10:49:56.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.518+0000 7fd9157fa640 1 -- 172.21.15.105:0/65578373 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fbaae71d-5472-4203-9f21-bf9affa127f5", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd90c006560 con 0x7fd91806f430 2024-05-09T10:49:56.511 INFO:teuthology.orchestra.run.smithi105.stdout:3 2024-05-09T10:49:56.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.519+0000 7fd91eeac640 1 -- 172.21.15.105:0/65578373 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd91806f430 msgr2=0x7fd918158710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.519+0000 7fd91eeac640 1 --2- 172.21.15.105:0/65578373 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd91806f430 0x7fd918158710 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00e870 tx=0x7fd90c00ed30 comp rx=0 tx=0).stop 2024-05-09T10:49:56.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.519+0000 7fd91eeac640 1 -- 172.21.15.105:0/65578373 shutdown_connections 2024-05-09T10:49:56.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.519+0000 7fd91eeac640 1 --2- 172.21.15.105:0/65578373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd91815e000 0x7fd9180b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.519+0000 7fd91eeac640 1 --2- 172.21.15.105:0/65578373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9180ce750 0x7fd9180b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.519+0000 7fd91eeac640 1 --2- 172.21.15.105:0/65578373 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd91806f430 0x7fd918158710 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.519+0000 7fd91eeac640 1 -- 172.21.15.105:0/65578373 >> 172.21.15.105:0/65578373 conn(0x7fd9180b25a0 msgr2=0x7fd9180ca430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:49:56.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.519+0000 7fd91eeac640 1 -- 172.21.15.105:0/65578373 shutdown_connections 2024-05-09T10:49:56.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.519+0000 7fd91eeac640 1 -- 172.21.15.105:0/65578373 wait complete. 2024-05-09T10:49:56.523 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new b996590f-1eac-4b34-9843-10c52c7bae73 4 2024-05-09T10:49:56.565 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:56.571+0000 7fb2d1723040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T10:49:56.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.629+0000 7f0991190640 1 Processor -- start 2024-05-09T10:49:56.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.629+0000 7f0991190640 1 -- start start 2024-05-09T10:49:56.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.630+0000 7f0991190640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f098c06f8a0 0x7f098c0cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.630+0000 7f0991190640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f098c0afcb0 0x7f098c0713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.630+0000 7f0991190640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f098c071930 0x7f098c0cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.630+0000 7f0991190640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f098c0b3500 con 0x7f098c071930 2024-05-09T10:49:56.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.630+0000 7f0991190640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f098c071d80 con 0x7f098c06f8a0 2024-05-09T10:49:56.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.630+0000 7f0991190640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098c0b3e60 con 0x7f098c0afcb0 2024-05-09T10:49:56.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.630+0000 7f098a59c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f098c0afcb0 0x7f098c0713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.630+0000 7f098ad9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f098c06f8a0 0x7f098c0cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.630+0000 7f098a59c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f098c0afcb0 0x7f098c0713f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:56842/0 (socket says 172.21.15.105:56842) 2024-05-09T10:49:56.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.630+0000 7f098a59c640 1 -- 172.21.15.105:0/4087984302 learned_addr learned my addr 172.21.15.105:0/4087984302 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-05-09T10:49:56.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.630+0000 7f098b59e640 1 --2- 172.21.15.105:0/4087984302 >> v2:172.21.15.98:3300/0 conn(0x7f098c071930 0x7f098c0cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.630+0000 7f098a59c640 1 -- 172.21.15.105:0/4087984302 >> v2:172.21.15.105:3300/0 conn(0x7f098c06f8a0 msgr2=0x7f098c0cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:49:56.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.630+0000 7f098a59c640 1 --2- 172.21.15.105:0/4087984302 >> v2:172.21.15.105:3300/0 conn(0x7f098c06f8a0 0x7f098c0cbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.630+0000 7f098a59c640 1 -- 172.21.15.105:0/4087984302 >> v2:172.21.15.98:3300/0 conn(0x7f098c071930 msgr2=0x7f098c0cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:49:56.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.630+0000 7f098a59c640 1 --2- 172.21.15.105:0/4087984302 >> v2:172.21.15.98:3300/0 conn(0x7f098c071930 0x7f098c0cec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.630+0000 7f098a59c640 1 -- 172.21.15.105:0/4087984302 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098c0d09a0 con 0x7f098c0afcb0 2024-05-09T10:49:56.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.630+0000 7f098b59e640 1 --2- 172.21.15.105:0/4087984302 >> v2:172.21.15.98:3300/0 conn(0x7f098c071930 0x7f098c0cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:49:56.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.631+0000 7f098a59c640 1 --2- 172.21.15.105:0/4087984302 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f098c0afcb0 0x7f098c0713f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f098c0685c0 tx=0x7f097402a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=176c65423d26a667 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:49:56.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.631+0000 7f0989d9b640 1 -- 172.21.15.105:0/4087984302 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974037070 con 0x7f098c0afcb0 2024-05-09T10:49:56.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.631+0000 7f0989d9b640 1 -- 172.21.15.105:0/4087984302 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0974032440 con 0x7f098c0afcb0 2024-05-09T10:49:56.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.631+0000 7f0989d9b640 1 -- 172.21.15.105:0/4087984302 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974032810 con 0x7f098c0afcb0 2024-05-09T10:49:56.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.632+0000 7f0991190640 1 -- 172.21.15.105:0/4087984302 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f098c0afcb0 msgr2=0x7f098c0713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.632+0000 7f0991190640 1 --2- 172.21.15.105:0/4087984302 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f098c0afcb0 0x7f098c0713f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f098c0685c0 tx=0x7f097402a1e0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.632+0000 7f0991190640 1 -- 172.21.15.105:0/4087984302 shutdown_connections 2024-05-09T10:49:56.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.632+0000 7f0991190640 1 --2- 172.21.15.105:0/4087984302 >> v2:172.21.15.98:3300/0 conn(0x7f098c071930 0x7f098c0cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.632+0000 7f0991190640 1 --2- 172.21.15.105:0/4087984302 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f098c0afcb0 0x7f098c0713f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.632+0000 7f0991190640 1 --2- 172.21.15.105:0/4087984302 >> v2:172.21.15.105:3300/0 conn(0x7f098c06f8a0 0x7f098c0cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.632+0000 7f0991190640 1 -- 172.21.15.105:0/4087984302 >> 172.21.15.105:0/4087984302 conn(0x7f098c0b25a0 msgr2=0x7f098c0b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:49:56.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.632+0000 7f0991190640 1 -- 172.21.15.105:0/4087984302 shutdown_connections 2024-05-09T10:49:56.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.632+0000 7f0991190640 1 -- 172.21.15.105:0/4087984302 wait complete. 2024-05-09T10:49:56.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.633+0000 7f0991190640 1 Processor -- start 2024-05-09T10:49:56.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.633+0000 7f0991190640 1 -- start start 2024-05-09T10:49:56.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.634+0000 7f0991190640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f098c06f8a0 0x7f098c0ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.634+0000 7f0991190640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f098c071930 0x7f098c15cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.634+0000 7f0991190640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f098c0afcb0 0x7f098c15d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.634+0000 7f0991190640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c0cc270 con 0x7f098c071930 2024-05-09T10:49:56.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.634+0000 7f0991190640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c0d0f60 con 0x7f098c06f8a0 2024-05-09T10:49:56.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.634+0000 7f0991190640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098c06fc80 con 0x7f098c0afcb0 2024-05-09T10:49:56.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.634+0000 7f098ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f098c06f8a0 0x7f098c0ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.634+0000 7f098b59e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f098c0afcb0 0x7f098c15d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.634+0000 7f098ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f098c06f8a0 0x7f098c0ca1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60130/0 (socket says 172.21.15.105:60130) 2024-05-09T10:49:56.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.634+0000 7f098ad9d640 1 -- 172.21.15.105:0/755293119 learned_addr learned my addr 172.21.15.105:0/755293119 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-05-09T10:49:56.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.634+0000 7f098b59e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f098c0afcb0 0x7f098c15d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:56850/0 (socket says 172.21.15.105:56850) 2024-05-09T10:49:56.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.634+0000 7f098a59c640 1 --2- 172.21.15.105:0/755293119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f098c071930 0x7f098c15cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.634+0000 7f098b59e640 1 -- 172.21.15.105:0/755293119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f098c06f8a0 msgr2=0x7f098c0ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.634+0000 7f098b59e640 1 --2- 172.21.15.105:0/755293119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f098c06f8a0 0x7f098c0ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.634+0000 7f098b59e640 1 -- 172.21.15.105:0/755293119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f098c071930 msgr2=0x7f098c15cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.634+0000 7f098b59e640 1 --2- 172.21.15.105:0/755293119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f098c071930 0x7f098c15cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.634+0000 7f098b59e640 1 -- 172.21.15.105:0/755293119 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098c1e40a0 con 0x7f098c0afcb0 2024-05-09T10:49:56.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.635+0000 7f098b59e640 1 --2- 172.21.15.105:0/755293119 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f098c0afcb0 0x7f098c15d050 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f098400ff80 tx=0x7f09840077c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:49:56.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.635+0000 7f09837fe640 1 -- 172.21.15.105:0/755293119 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0984003990 con 0x7f098c0afcb0 2024-05-09T10:49:56.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.635+0000 7f09837fe640 1 -- 172.21.15.105:0/755293119 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0984003b30 con 0x7f098c0afcb0 2024-05-09T10:49:56.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.635+0000 7f0991190640 1 -- 172.21.15.105:0/755293119 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f098c14a070 con 0x7f098c0afcb0 2024-05-09T10:49:56.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.635+0000 7f09837fe640 1 -- 172.21.15.105:0/755293119 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0984004030 con 0x7f098c0afcb0 2024-05-09T10:49:56.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.635+0000 7f0991190640 1 -- 172.21.15.105:0/755293119 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f098c14a630 con 0x7f098c0afcb0 2024-05-09T10:49:56.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.635+0000 7f09837fe640 1 -- 172.21.15.105:0/755293119 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f09840041d0 con 0x7f098c0afcb0 2024-05-09T10:49:56.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.635+0000 7f09837fe640 1 -- 172.21.15.105:0/755293119 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f098401ad10 con 0x7f098c0afcb0 2024-05-09T10:49:56.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.636+0000 7f0991190640 1 -- 172.21.15.105:0/755293119 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f098c04f000 con 0x7f098c0afcb0 2024-05-09T10:49:56.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.638+0000 7f09837fe640 1 -- 172.21.15.105:0/755293119 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0984004650 con 0x7f098c0afcb0 2024-05-09T10:49:56.649 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:56.655+0000 7f476571b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T10:49:56.652 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:56.659+0000 7fb2d1723040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T10:49:56.694 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:56.701+0000 7fb2d1723040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T10:49:56.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.710+0000 7f0991190640 1 -- 172.21.15.105:0/755293119 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b996590f-1eac-4b34-9843-10c52c7bae73", "id": 4} v 0) v1 -- 0x7f098c0b4450 con 0x7f098c0afcb0 2024-05-09T10:49:56.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.716+0000 7f09837fe640 1 -- 172.21.15.105:0/755293119 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b996590f-1eac-4b34-9843-10c52c7bae73", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f098401a410 con 0x7f098c0afcb0 2024-05-09T10:49:56.709 INFO:teuthology.orchestra.run.smithi105.stdout:4 2024-05-09T10:49:56.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.716+0000 7f0991190640 1 -- 172.21.15.105:0/755293119 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f098c0afcb0 msgr2=0x7f098c15d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.716+0000 7f0991190640 1 --2- 172.21.15.105:0/755293119 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f098c0afcb0 0x7f098c15d050 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f098400ff80 tx=0x7f09840077c0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.716+0000 7f0991190640 1 -- 172.21.15.105:0/755293119 shutdown_connections 2024-05-09T10:49:56.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.716+0000 7f0991190640 1 --2- 172.21.15.105:0/755293119 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f098c0afcb0 0x7f098c15d050 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.716+0000 7f0991190640 1 --2- 172.21.15.105:0/755293119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f098c071930 0x7f098c15cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.716+0000 7f0991190640 1 --2- 172.21.15.105:0/755293119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f098c06f8a0 0x7f098c0ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.717+0000 7f0991190640 1 -- 172.21.15.105:0/755293119 >> 172.21.15.105:0/755293119 conn(0x7f098c0b25a0 msgr2=0x7f098c0b53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:49:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.717+0000 7f0991190640 1 -- 172.21.15.105:0/755293119 shutdown_connections 2024-05-09T10:49:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.717+0000 7f0991190640 1 -- 172.21.15.105:0/755293119 wait complete. 2024-05-09T10:49:56.720 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new 2a4fad49-a064-48cf-a77d-04105bb5dca5 5 2024-05-09T10:49:56.740 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:56.747+0000 7f476571b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T10:49:56.785 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:56.792+0000 7f476571b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T10:49:56.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.827+0000 7f5f57607640 1 Processor -- start 2024-05-09T10:49:56.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.827+0000 7f5f57607640 1 -- start start 2024-05-09T10:49:56.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.828+0000 7f5f57607640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5f5006f8a0 0x7f5f500cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.828+0000 7f5f57607640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f50071b00 0x7f5f500b4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.828+0000 7f5f57607640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f500b3e60 con 0x7f5f500afcb0 2024-05-09T10:49:56.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.828+0000 7f5f57607640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f500b3500 con 0x7f5f5006f8a0 2024-05-09T10:49:56.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.828+0000 7f5f57607640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f500d09a0 con 0x7f5f50071b00 2024-05-09T10:49:56.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.828+0000 7f5f553a3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5f5006f8a0 0x7f5f500cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.828+0000 7f5f55ba4640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f50071b00 0x7f5f500b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.828+0000 7f5f553a3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5f5006f8a0 0x7f5f500cbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60132/0 (socket says 172.21.15.105:60132) 2024-05-09T10:49:56.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.828+0000 7f5f553a3640 1 -- 172.21.15.105:0/1950186632 learned_addr learned my addr 172.21.15.105:0/1950186632 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-05-09T10:49:56.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.828+0000 7f5f55ba4640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f50071b00 0x7f5f500b4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:56858/0 (socket says 172.21.15.105:56858) 2024-05-09T10:49:56.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.828+0000 7f5f55ba4640 1 -- 172.21.15.105:0/1950186632 >> v2:172.21.15.105:3300/0 conn(0x7f5f5006f8a0 msgr2=0x7f5f500cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:49:56.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.828+0000 7f5f55ba4640 1 --2- 172.21.15.105:0/1950186632 >> v2:172.21.15.105:3300/0 conn(0x7f5f5006f8a0 0x7f5f500cbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.828+0000 7f5f55ba4640 1 -- 172.21.15.105:0/1950186632 >> v1:172.21.15.98:6789/0 conn(0x7f5f500afcb0 legacy=0x7f5f500713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.829+0000 7f5f55ba4640 1 -- 172.21.15.105:0/1950186632 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f500d0c80 con 0x7f5f50071b00 2024-05-09T10:49:56.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.829+0000 7f5f55ba4640 1 --2- 172.21.15.105:0/1950186632 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f50071b00 0x7f5f500b4340 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5f48009d90 tx=0x7f5f4802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffcdfe2ada9f51f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:49:56.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.829+0000 7f5f47fff640 1 -- 172.21.15.105:0/1950186632 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f48007740 con 0x7f5f50071b00 2024-05-09T10:49:56.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.829+0000 7f5f47fff640 1 -- 172.21.15.105:0/1950186632 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f480078e0 con 0x7f5f50071b00 2024-05-09T10:49:56.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.829+0000 7f5f47fff640 1 -- 172.21.15.105:0/1950186632 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f48007cb0 con 0x7f5f50071b00 2024-05-09T10:49:56.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.830+0000 7f5f57607640 1 -- 172.21.15.105:0/1950186632 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f50071b00 msgr2=0x7f5f500b4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.830+0000 7f5f57607640 1 --2- 172.21.15.105:0/1950186632 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f50071b00 0x7f5f500b4340 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5f48009d90 tx=0x7f5f4802a660 comp rx=0 tx=0).stop 2024-05-09T10:49:56.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.830+0000 7f5f57607640 1 -- 172.21.15.105:0/1950186632 shutdown_connections 2024-05-09T10:49:56.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.830+0000 7f5f57607640 1 --2- 172.21.15.105:0/1950186632 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f50071b00 0x7f5f500b4340 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.830+0000 7f5f57607640 1 --2- 172.21.15.105:0/1950186632 >> v2:172.21.15.105:3300/0 conn(0x7f5f5006f8a0 0x7f5f500cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.830+0000 7f5f57607640 1 -- 172.21.15.105:0/1950186632 >> 172.21.15.105:0/1950186632 conn(0x7f5f500b25a0 msgr2=0x7f5f500b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:49:56.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.830+0000 7f5f57607640 1 -- 172.21.15.105:0/1950186632 shutdown_connections 2024-05-09T10:49:56.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.830+0000 7f5f57607640 1 -- 172.21.15.105:0/1950186632 wait complete. 2024-05-09T10:49:56.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.831+0000 7f5f57607640 1 Processor -- start 2024-05-09T10:49:56.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.831+0000 7f5f57607640 1 -- start start 2024-05-09T10:49:56.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.832+0000 7f5f57607640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f5006f8a0 0x7f5f50156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.832+0000 7f5f57607640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f50071b00 0x7f5f501552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.832+0000 7f5f57607640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f500afcb0 0x7f5f50155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.832+0000 7f5f57607640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5006fc80 con 0x7f5f500afcb0 2024-05-09T10:49:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.832+0000 7f5f57607640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f500cc270 con 0x7f5f5006f8a0 2024-05-09T10:49:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.832+0000 7f5f57607640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5006fe00 con 0x7f5f50071b00 2024-05-09T10:49:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.832+0000 7f5f553a3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f5006f8a0 0x7f5f50156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.832+0000 7f5f54ba2640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f50071b00 0x7f5f501552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.832+0000 7f5f553a3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f5006f8a0 0x7f5f50156e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60136/0 (socket says 172.21.15.105:60136) 2024-05-09T10:49:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.832+0000 7f5f553a3640 1 -- 172.21.15.105:0/313512707 learned_addr learned my addr 172.21.15.105:0/313512707 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-05-09T10:49:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.832+0000 7f5f55ba4640 1 --2- 172.21.15.105:0/313512707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f500afcb0 0x7f5f50155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.832+0000 7f5f553a3640 1 -- 172.21.15.105:0/313512707 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f50071b00 msgr2=0x7f5f501552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.832+0000 7f5f553a3640 1 --2- 172.21.15.105:0/313512707 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f50071b00 0x7f5f501552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.832+0000 7f5f553a3640 1 -- 172.21.15.105:0/313512707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f500afcb0 msgr2=0x7f5f50155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.832+0000 7f5f553a3640 1 --2- 172.21.15.105:0/313512707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f500afcb0 0x7f5f50155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.832+0000 7f5f553a3640 1 -- 172.21.15.105:0/313512707 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f501577d0 con 0x7f5f5006f8a0 2024-05-09T10:49:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.832+0000 7f5f553a3640 1 --2- 172.21.15.105:0/313512707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f5006f8a0 0x7f5f50156e10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000e840 tx=0x7f5f4000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:49:56.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.833+0000 7f5f45ffb640 1 -- 172.21.15.105:0/313512707 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f40017070 con 0x7f5f5006f8a0 2024-05-09T10:49:56.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.833+0000 7f5f45ffb640 1 -- 172.21.15.105:0/313512707 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4000c8e0 con 0x7f5f5006f8a0 2024-05-09T10:49:56.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.833+0000 7f5f57607640 1 -- 172.21.15.105:0/313512707 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f50157ab0 con 0x7f5f5006f8a0 2024-05-09T10:49:56.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.833+0000 7f5f45ffb640 1 -- 172.21.15.105:0/313512707 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4000ccd0 con 0x7f5f5006f8a0 2024-05-09T10:49:56.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.833+0000 7f5f57607640 1 -- 172.21.15.105:0/313512707 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f50158070 con 0x7f5f5006f8a0 2024-05-09T10:49:56.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.833+0000 7f5f45ffb640 1 -- 172.21.15.105:0/313512707 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f5f4001a070 con 0x7f5f5006f8a0 2024-05-09T10:49:56.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.833+0000 7f5f45ffb640 1 -- 172.21.15.105:0/313512707 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f5f40013470 con 0x7f5f5006f8a0 2024-05-09T10:49:56.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.834+0000 7f5f57607640 1 -- 172.21.15.105:0/313512707 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f5014db40 con 0x7f5f5006f8a0 2024-05-09T10:49:56.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.836+0000 7f5f45ffb640 1 -- 172.21.15.105:0/313512707 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f5f40006480 con 0x7f5f5006f8a0 2024-05-09T10:49:56.859 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:56.866+0000 7fb2d1723040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T10:49:56.906 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:56.913+0000 7fb2d1723040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T10:49:56.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.926+0000 7f5f57607640 1 -- 172.21.15.105:0/313512707 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2a4fad49-a064-48cf-a77d-04105bb5dca5", "id": 5} v 0) v1 -- 0x7f5f5006b7b0 con 0x7f5f5006f8a0 2024-05-09T10:49:56.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.932+0000 7f5f45ffb640 1 -- 172.21.15.105:0/313512707 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2a4fad49-a064-48cf-a77d-04105bb5dca5", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5f40006660 con 0x7f5f5006f8a0 2024-05-09T10:49:56.925 INFO:teuthology.orchestra.run.smithi105.stdout:5 2024-05-09T10:49:56.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.932+0000 7f5f57607640 1 -- 172.21.15.105:0/313512707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f5006f8a0 msgr2=0x7f5f50156e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:56.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.932+0000 7f5f57607640 1 --2- 172.21.15.105:0/313512707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f5006f8a0 0x7f5f50156e10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000e840 tx=0x7f5f4000ed00 comp rx=0 tx=0).stop 2024-05-09T10:49:56.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.933+0000 7f5f57607640 1 -- 172.21.15.105:0/313512707 shutdown_connections 2024-05-09T10:49:56.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.933+0000 7f5f57607640 1 --2- 172.21.15.105:0/313512707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f500afcb0 0x7f5f50155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.933+0000 7f5f57607640 1 --2- 172.21.15.105:0/313512707 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f50071b00 0x7f5f501552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.933+0000 7f5f57607640 1 --2- 172.21.15.105:0/313512707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f5006f8a0 0x7f5f50156e10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:56.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.933+0000 7f5f57607640 1 -- 172.21.15.105:0/313512707 >> 172.21.15.105:0/313512707 conn(0x7f5f500b25a0 msgr2=0x7f5f500713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:49:56.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.933+0000 7f5f57607640 1 -- 172.21.15.105:0/313512707 shutdown_connections 2024-05-09T10:49:56.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:56.933+0000 7f5f57607640 1 -- 172.21.15.105:0/313512707 wait complete. 2024-05-09T10:49:56.936 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new d4e69c47-342a-4f33-94fa-bad0fba92863 6 2024-05-09T10:49:56.959 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:56.966+0000 7fb2d1723040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T10:49:56.986 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:56.992+0000 7f476571b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T10:49:57.038 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:57.045+0000 7f476571b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T10:49:57.070 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:57.077+0000 7fb2d1723040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T10:49:57.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.083+0000 7f928c8f7640 1 Processor -- start 2024-05-09T10:49:57.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.083+0000 7f928c8f7640 1 -- start start 2024-05-09T10:49:57.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.084+0000 7f928c8f7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f928815de90 0x7f92880b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:57.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.084+0000 7f928c8f7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f92880b38a0 0x7f928806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:57.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.084+0000 7f928c8f7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f928806b120 0x7f92881618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:57.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.084+0000 7f928c8f7640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92880b3df0 con 0x7f92880b38a0 2024-05-09T10:49:57.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.084+0000 7f928c8f7640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f928815e640 con 0x7f928806b120 2024-05-09T10:49:57.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.084+0000 7f928c8f7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f928815e920 con 0x7f928815de90 2024-05-09T10:49:57.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.084+0000 7f928659c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f928815de90 0x7f92880b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:57.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.084+0000 7f9286d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f928806b120 0x7f92881618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:57.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.084+0000 7f928659c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f928815de90 0x7f92880b55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:56862/0 (socket says 172.21.15.105:56862) 2024-05-09T10:49:57.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.084+0000 7f9286d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f928806b120 0x7f92881618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60138/0 (socket says 172.21.15.105:60138) 2024-05-09T10:49:57.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.084+0000 7f928659c640 1 -- 172.21.15.105:0/1008311994 learned_addr learned my addr 172.21.15.105:0/1008311994 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-05-09T10:49:57.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.084+0000 7f9285d9b640 1 --2- 172.21.15.105:0/1008311994 >> v2:172.21.15.98:3300/0 conn(0x7f92880b38a0 0x7f928806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:57.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.084+0000 7f928659c640 1 -- 172.21.15.105:0/1008311994 >> v2:172.21.15.105:3300/0 conn(0x7f928806b120 msgr2=0x7f92881618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:49:57.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.084+0000 7f928659c640 1 --2- 172.21.15.105:0/1008311994 >> v2:172.21.15.105:3300/0 conn(0x7f928806b120 0x7f92881618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.084+0000 7f928659c640 1 -- 172.21.15.105:0/1008311994 >> v2:172.21.15.98:3300/0 conn(0x7f92880b38a0 msgr2=0x7f928806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:49:57.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.084+0000 7f928659c640 1 --2- 172.21.15.105:0/1008311994 >> v2:172.21.15.98:3300/0 conn(0x7f92880b38a0 0x7f928806abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.084+0000 7f928659c640 1 -- 172.21.15.105:0/1008311994 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92880b5b00 con 0x7f928815de90 2024-05-09T10:49:57.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.084+0000 7f9286d9d640 1 --2- 172.21.15.105:0/1008311994 >> v2:172.21.15.105:3300/0 conn(0x7f928806b120 0x7f92881618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:49:57.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.085+0000 7f928659c640 1 --2- 172.21.15.105:0/1008311994 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f928815de90 0x7f92880b55c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9288162980 tx=0x7f927802a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bb281b679fb98f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:49:57.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.085+0000 7f928559a640 1 -- 172.21.15.105:0/1008311994 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9278037070 con 0x7f928815de90 2024-05-09T10:49:57.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.085+0000 7f928559a640 1 -- 172.21.15.105:0/1008311994 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9278032440 con 0x7f928815de90 2024-05-09T10:49:57.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.085+0000 7f928559a640 1 -- 172.21.15.105:0/1008311994 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9278032810 con 0x7f928815de90 2024-05-09T10:49:57.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.086+0000 7f928c8f7640 1 -- 172.21.15.105:0/1008311994 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f928815de90 msgr2=0x7f92880b55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:57.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.086+0000 7f928c8f7640 1 --2- 172.21.15.105:0/1008311994 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f928815de90 0x7f92880b55c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9288162980 tx=0x7f927802a1e0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.086+0000 7f928c8f7640 1 -- 172.21.15.105:0/1008311994 shutdown_connections 2024-05-09T10:49:57.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.086+0000 7f928c8f7640 1 --2- 172.21.15.105:0/1008311994 >> v2:172.21.15.105:3300/0 conn(0x7f928806b120 0x7f92881618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.086+0000 7f928c8f7640 1 --2- 172.21.15.105:0/1008311994 >> v2:172.21.15.98:3300/0 conn(0x7f92880b38a0 0x7f928806abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.086+0000 7f928c8f7640 1 --2- 172.21.15.105:0/1008311994 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f928815de90 0x7f92880b55c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.086+0000 7f928c8f7640 1 -- 172.21.15.105:0/1008311994 >> 172.21.15.105:0/1008311994 conn(0x7f92880b25a0 msgr2=0x7f92880b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:49:57.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.086+0000 7f928c8f7640 1 -- 172.21.15.105:0/1008311994 shutdown_connections 2024-05-09T10:49:57.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.086+0000 7f928c8f7640 1 -- 172.21.15.105:0/1008311994 wait complete. 2024-05-09T10:49:57.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.087+0000 7f928c8f7640 1 Processor -- start 2024-05-09T10:49:57.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.087+0000 7f928c8f7640 1 -- start start 2024-05-09T10:49:57.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.088+0000 7f928c8f7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f928806b120 0x7f9288145d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:57.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.088+0000 7f928c8f7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92880b38a0 0x7f9288148ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:57.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.088+0000 7f928c8f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f928815de90 0x7f92880cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:57.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.088+0000 7f928c8f7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928815e3f0 con 0x7f928806b120 2024-05-09T10:49:57.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.088+0000 7f928c8f7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928815e270 con 0x7f928815de90 2024-05-09T10:49:57.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.088+0000 7f928c8f7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9288161e20 con 0x7f92880b38a0 2024-05-09T10:49:57.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.088+0000 7f9285d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92880b38a0 0x7f9288148ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:57.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.088+0000 7f9286d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f928815de90 0x7f92880cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:57.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.088+0000 7f9285d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92880b38a0 0x7f9288148ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:56874/0 (socket says 172.21.15.105:56874) 2024-05-09T10:49:57.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.088+0000 7f9285d9b640 1 -- 172.21.15.105:0/3314497547 learned_addr learned my addr 172.21.15.105:0/3314497547 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-05-09T10:49:57.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.088+0000 7f928659c640 1 --2- 172.21.15.105:0/3314497547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f928806b120 0x7f9288145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:57.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.088+0000 7f9285d9b640 1 -- 172.21.15.105:0/3314497547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f928815de90 msgr2=0x7f92880cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:57.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.088+0000 7f9285d9b640 1 --2- 172.21.15.105:0/3314497547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f928815de90 0x7f92880cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.088+0000 7f9285d9b640 1 -- 172.21.15.105:0/3314497547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f928806b120 msgr2=0x7f9288145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:57.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.088+0000 7f9285d9b640 1 --2- 172.21.15.105:0/3314497547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f928806b120 0x7f9288145d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.088+0000 7f9285d9b640 1 -- 172.21.15.105:0/3314497547 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9288149400 con 0x7f92880b38a0 2024-05-09T10:49:57.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.088+0000 7f9286d9d640 1 --2- 172.21.15.105:0/3314497547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f928815de90 0x7f92880cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:49:57.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.088+0000 7f9285d9b640 1 --2- 172.21.15.105:0/3314497547 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92880b38a0 0x7f9288148ec0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f927c00e9b0 tx=0x7f927c00edb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:49:57.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.089+0000 7f9276ffd640 1 -- 172.21.15.105:0/3314497547 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c017070 con 0x7f92880b38a0 2024-05-09T10:49:57.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.089+0000 7f9276ffd640 1 -- 172.21.15.105:0/3314497547 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c00c940 con 0x7f92880b38a0 2024-05-09T10:49:57.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.089+0000 7f9276ffd640 1 -- 172.21.15.105:0/3314497547 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c00cd10 con 0x7f92880b38a0 2024-05-09T10:49:57.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.089+0000 7f928c8f7640 1 -- 172.21.15.105:0/3314497547 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92881467a0 con 0x7f92880b38a0 2024-05-09T10:49:57.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.089+0000 7f928c8f7640 1 -- 172.21.15.105:0/3314497547 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9288146d60 con 0x7f92880b38a0 2024-05-09T10:49:57.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.089+0000 7f9276ffd640 1 -- 172.21.15.105:0/3314497547 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f927c01a070 con 0x7f92880b38a0 2024-05-09T10:49:57.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.089+0000 7f9276ffd640 1 -- 172.21.15.105:0/3314497547 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f927c0064d0 con 0x7f92880b38a0 2024-05-09T10:49:57.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.090+0000 7f928c8f7640 1 -- 172.21.15.105:0/3314497547 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9288072ea0 con 0x7f92880b38a0 2024-05-09T10:49:57.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.092+0000 7f9276ffd640 1 -- 172.21.15.105:0/3314497547 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9288149400 con 0x7f92880b38a0 2024-05-09T10:49:57.094 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:57.101+0000 7f476571b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T10:49:57.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.164+0000 7f928c8f7640 1 -- 172.21.15.105:0/3314497547 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d4e69c47-342a-4f33-94fa-bad0fba92863", "id": 6} v 0) v1 -- 0x7f92880afda0 con 0x7f92880b38a0 2024-05-09T10:49:57.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.169+0000 7f9276ffd640 1 -- 172.21.15.105:0/3314497547 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d4e69c47-342a-4f33-94fa-bad0fba92863", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f927c0067f0 con 0x7f92880b38a0 2024-05-09T10:49:57.163 INFO:teuthology.orchestra.run.smithi105.stdout:6 2024-05-09T10:49:57.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.170+0000 7f928c8f7640 1 -- 172.21.15.105:0/3314497547 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92880b38a0 msgr2=0x7f9288148ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:57.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.170+0000 7f928c8f7640 1 --2- 172.21.15.105:0/3314497547 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92880b38a0 0x7f9288148ec0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f927c00e9b0 tx=0x7f927c00edb0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.170+0000 7f928c8f7640 1 -- 172.21.15.105:0/3314497547 shutdown_connections 2024-05-09T10:49:57.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.170+0000 7f928c8f7640 1 --2- 172.21.15.105:0/3314497547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f928815de90 0x7f92880cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.170+0000 7f928c8f7640 1 --2- 172.21.15.105:0/3314497547 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92880b38a0 0x7f9288148ec0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.170+0000 7f928c8f7640 1 --2- 172.21.15.105:0/3314497547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f928806b120 0x7f9288145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.170+0000 7f928c8f7640 1 -- 172.21.15.105:0/3314497547 >> 172.21.15.105:0/3314497547 conn(0x7f92880b25a0 msgr2=0x7f92880b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:49:57.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.170+0000 7f928c8f7640 1 -- 172.21.15.105:0/3314497547 shutdown_connections 2024-05-09T10:49:57.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.170+0000 7f928c8f7640 1 -- 172.21.15.105:0/3314497547 wait complete. 2024-05-09T10:49:57.174 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new 5d0f0551-460f-4131-89b0-df7b1630a4e8 7 2024-05-09T10:49:57.211 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:57.218+0000 7f476571b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T10:49:57.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.318+0000 7fcd11d42640 1 Processor -- start 2024-05-09T10:49:57.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.318+0000 7fcd11d42640 1 -- start start 2024-05-09T10:49:57.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.319+0000 7fcd11d42640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcd0c0afcb0 0x7fcd0c0cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:57.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.319+0000 7fcd11d42640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd0c0b3e60 con 0x7fcd0c071b00 2024-05-09T10:49:57.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.319+0000 7fcd11d42640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd0c0d09a0 con 0x7fcd0c06f8a0 2024-05-09T10:49:57.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.319+0000 7fcd11d42640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd0c0b3500 con 0x7fcd0c0afcb0 2024-05-09T10:49:57.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.319+0000 7fcd0b7fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fcd0c06f8a0 0x7fcd0c0713f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:56784/0 (socket says 172.21.15.105:56784) 2024-05-09T10:49:57.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.319+0000 7fcd0b7fe640 1 -- 172.21.15.105:0/1204875072 learned_addr learned my addr 172.21.15.105:0/1204875072 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-05-09T10:49:57.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.320+0000 7fcd0bfff640 1 --2- 172.21.15.105:0/1204875072 >> v2:172.21.15.98:3300/0 conn(0x7fcd0c0afcb0 0x7fcd0c0cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:57.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.320+0000 7fcd0a7fc640 1 -- 172.21.15.105:0/1204875072 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1241555215 0 0) 0x7fcd0c0d09a0 con 0x7fcd0c06f8a0 2024-05-09T10:49:57.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.320+0000 7fcd0a7fc640 1 -- 172.21.15.105:0/1204875072 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccf8003520 con 0x7fcd0c06f8a0 2024-05-09T10:49:57.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.320+0000 7fcd0bfff640 1 -- 172.21.15.105:0/1204875072 >> v1:172.21.15.105:6789/0 conn(0x7fcd0c06f8a0 legacy=0x7fcd0c0713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:57.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.320+0000 7fcd0bfff640 1 -- 172.21.15.105:0/1204875072 >> v1:172.21.15.98:6789/0 conn(0x7fcd0c071b00 legacy=0x7fcd0c0b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:57.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.320+0000 7fcd0a7fc640 1 -- 172.21.15.105:0/1204875072 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141478144 0 0) 0x7fcd0c0b3e60 con 0x7fcd0c071b00 2024-05-09T10:49:57.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.320+0000 7fcd0bfff640 1 -- 172.21.15.105:0/1204875072 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c0d0c80 con 0x7fcd0c0afcb0 2024-05-09T10:49:57.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.321+0000 7fcd0bfff640 1 --2- 172.21.15.105:0/1204875072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd0c0afcb0 0x7fcd0c0cbb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009d90 tx=0x7fcd0402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604790f973e0fdfa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:49:57.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.321+0000 7fcd0a7fc640 1 -- 172.21.15.105:0/1204875072 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd04007740 con 0x7fcd0c0afcb0 2024-05-09T10:49:57.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.321+0000 7fcd0a7fc640 1 -- 172.21.15.105:0/1204875072 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd040078e0 con 0x7fcd0c0afcb0 2024-05-09T10:49:57.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.321+0000 7fcd0a7fc640 1 -- 172.21.15.105:0/1204875072 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd04007cb0 con 0x7fcd0c0afcb0 2024-05-09T10:49:57.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.321+0000 7fcd11d42640 1 -- 172.21.15.105:0/1204875072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd0c0afcb0 msgr2=0x7fcd0c0cbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:57.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.321+0000 7fcd11d42640 1 --2- 172.21.15.105:0/1204875072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd0c0afcb0 0x7fcd0c0cbb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009d90 tx=0x7fcd0402a6f0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.321+0000 7fcd11d42640 1 -- 172.21.15.105:0/1204875072 shutdown_connections 2024-05-09T10:49:57.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.321+0000 7fcd11d42640 1 --2- 172.21.15.105:0/1204875072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd0c0afcb0 0x7fcd0c0cbb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.321+0000 7fcd11d42640 1 -- 172.21.15.105:0/1204875072 >> 172.21.15.105:0/1204875072 conn(0x7fcd0c0b25a0 msgr2=0x7fcd0c0b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:49:57.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.321+0000 7fcd11d42640 1 -- 172.21.15.105:0/1204875072 shutdown_connections 2024-05-09T10:49:57.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.321+0000 7fcd11d42640 1 -- 172.21.15.105:0/1204875072 wait complete. 2024-05-09T10:49:57.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.322+0000 7fcd11d42640 1 Processor -- start 2024-05-09T10:49:57.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.323+0000 7fcd11d42640 1 -- start start 2024-05-09T10:49:57.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.323+0000 7fcd11d42640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd0c06f8a0 0x7fcd0c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:57.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.323+0000 7fcd11d42640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd0c071b00 0x7fcd0c0c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:57.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.323+0000 7fcd11d42640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd0c0afcb0 0x7fcd0c1f7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:57.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.323+0000 7fcd11d42640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c06fc80 con 0x7fcd0c06f8a0 2024-05-09T10:49:57.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.323+0000 7fcd11d42640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c0cc270 con 0x7fcd0c071b00 2024-05-09T10:49:57.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.323+0000 7fcd11d42640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c06fe00 con 0x7fcd0c0afcb0 2024-05-09T10:49:57.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.323+0000 7fcd0bfff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd0c0afcb0 0x7fcd0c1f7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:57.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.323+0000 7fcd0affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd0c071b00 0x7fcd0c0c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:57.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.324+0000 7fcd0bfff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd0c0afcb0 0x7fcd0c1f7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:56888/0 (socket says 172.21.15.105:56888) 2024-05-09T10:49:57.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.324+0000 7fcd0affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd0c071b00 0x7fcd0c0c9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60158/0 (socket says 172.21.15.105:60158) 2024-05-09T10:49:57.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.324+0000 7fcd0bfff640 1 -- 172.21.15.105:0/2492057435 learned_addr learned my addr 172.21.15.105:0/2492057435 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-05-09T10:49:57.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.324+0000 7fcd0b7fe640 1 --2- 172.21.15.105:0/2492057435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd0c06f8a0 0x7fcd0c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:57.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.324+0000 7fcd0affd640 1 -- 172.21.15.105:0/2492057435 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd0c0afcb0 msgr2=0x7fcd0c1f7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:57.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.324+0000 7fcd0affd640 1 --2- 172.21.15.105:0/2492057435 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd0c0afcb0 0x7fcd0c1f7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.324+0000 7fcd0affd640 1 -- 172.21.15.105:0/2492057435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd0c06f8a0 msgr2=0x7fcd0c06ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:57.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.324+0000 7fcd0affd640 1 --2- 172.21.15.105:0/2492057435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd0c06f8a0 0x7fcd0c06ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.324+0000 7fcd0affd640 1 -- 172.21.15.105:0/2492057435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c0ac520 con 0x7fcd0c071b00 2024-05-09T10:49:57.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.324+0000 7fcd0bfff640 1 --2- 172.21.15.105:0/2492057435 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd0c0afcb0 0x7fcd0c1f7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:49:57.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.324+0000 7fcd0affd640 1 --2- 172.21.15.105:0/2492057435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd0c071b00 0x7fcd0c0c9e90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fccf400c450 tx=0x7fccf400c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:49:57.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.324+0000 7fcd03fff640 1 -- 172.21.15.105:0/2492057435 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf4017070 con 0x7fcd0c071b00 2024-05-09T10:49:57.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.324+0000 7fcd03fff640 1 -- 172.21.15.105:0/2492057435 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf40035f0 con 0x7fcd0c071b00 2024-05-09T10:49:57.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.324+0000 7fcd11d42640 1 -- 172.21.15.105:0/2492057435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd0c06f320 con 0x7fcd0c071b00 2024-05-09T10:49:57.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.324+0000 7fcd03fff640 1 -- 172.21.15.105:0/2492057435 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf40039e0 con 0x7fcd0c071b00 2024-05-09T10:49:57.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.324+0000 7fcd11d42640 1 -- 172.21.15.105:0/2492057435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd0c15dc50 con 0x7fcd0c071b00 2024-05-09T10:49:57.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.324+0000 7fcd03fff640 1 -- 172.21.15.105:0/2492057435 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fccf401a070 con 0x7fcd0c071b00 2024-05-09T10:49:57.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.324+0000 7fcd03fff640 1 -- 172.21.15.105:0/2492057435 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fccf4012a60 con 0x7fcd0c071b00 2024-05-09T10:49:57.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.325+0000 7fcd11d42640 1 -- 172.21.15.105:0/2492057435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd0c04f000 con 0x7fcd0c071b00 2024-05-09T10:49:57.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.327+0000 7fcd03fff640 1 -- 172.21.15.105:0/2492057435 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fccf4021020 con 0x7fcd0c071b00 2024-05-09T10:49:57.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.399+0000 7fcd11d42640 1 -- 172.21.15.105:0/2492057435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5d0f0551-460f-4131-89b0-df7b1630a4e8", "id": 7} v 0) v1 -- 0x7fcd0c06ac80 con 0x7fcd0c071b00 2024-05-09T10:49:57.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.405+0000 7fcd03fff640 1 -- 172.21.15.105:0/2492057435 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5d0f0551-460f-4131-89b0-df7b1630a4e8", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcd0c06f320 con 0x7fcd0c071b00 2024-05-09T10:49:57.398 INFO:teuthology.orchestra.run.smithi105.stdout:7 2024-05-09T10:49:57.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.406+0000 7fcd11d42640 1 -- 172.21.15.105:0/2492057435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd0c071b00 msgr2=0x7fcd0c0c9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:57.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.406+0000 7fcd11d42640 1 --2- 172.21.15.105:0/2492057435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd0c071b00 0x7fcd0c0c9e90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fccf400c450 tx=0x7fccf400c910 comp rx=0 tx=0).stop 2024-05-09T10:49:57.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.406+0000 7fcd11d42640 1 -- 172.21.15.105:0/2492057435 shutdown_connections 2024-05-09T10:49:57.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.406+0000 7fcd11d42640 1 --2- 172.21.15.105:0/2492057435 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd0c0afcb0 0x7fcd0c1f7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.406+0000 7fcd11d42640 1 --2- 172.21.15.105:0/2492057435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd0c071b00 0x7fcd0c0c9e90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.406+0000 7fcd11d42640 1 --2- 172.21.15.105:0/2492057435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd0c06f8a0 0x7fcd0c06ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.406+0000 7fcd11d42640 1 -- 172.21.15.105:0/2492057435 >> 172.21.15.105:0/2492057435 conn(0x7fcd0c0b25a0 msgr2=0x7fcd0c06b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:49:57.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.406+0000 7fcd11d42640 1 -- 172.21.15.105:0/2492057435 shutdown_connections 2024-05-09T10:49:57.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-05-09T10:49:57.406+0000 7fcd11d42640 1 -- 172.21.15.105:0/2492057435 wait complete. 2024-05-09T10:49:57.409 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-09T10:49:57.410 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-09T10:49:57.412 INFO:tasks.ceph.osd.0:Started 2024-05-09T10:49:57.412 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-09T10:49:57.412 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-09T10:49:57.416 INFO:tasks.ceph.osd.1:Started 2024-05-09T10:49:57.416 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-09T10:49:57.416 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-09T10:49:57.418 INFO:tasks.ceph.osd.2:Started 2024-05-09T10:49:57.418 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-09T10:49:57.418 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-09T10:49:57.421 INFO:tasks.ceph.osd.3:Started 2024-05-09T10:49:57.421 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-09T10:49:57.421 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-09T10:49:57.423 INFO:tasks.ceph.osd.4:Started 2024-05-09T10:49:57.423 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-09T10:49:57.423 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-09T10:49:57.425 INFO:tasks.ceph.osd.5:Started 2024-05-09T10:49:57.425 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-09T10:49:57.426 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-09T10:49:57.428 INFO:tasks.ceph.osd.6:Started 2024-05-09T10:49:57.428 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-09T10:49:57.428 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-09T10:49:57.430 INFO:tasks.ceph.osd.7:Started 2024-05-09T10:49:57.430 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:49:57.468 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:57.474+0000 7fb2d1723040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T10:49:57.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.580+0000 7fc0501a5640 1 Processor -- start 2024-05-09T10:49:57.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.581+0000 7fc0501a5640 1 -- start start 2024-05-09T10:49:57.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.581+0000 7fc0501a5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc04806eb30 0x7fc0480b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:57.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.581+0000 7fc0501a5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc04806d570 0x7fc04806ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:57.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.581+0000 7fc0501a5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc04806f450 con 0x7fc04815ddc0 2024-05-09T10:49:57.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.582+0000 7fc0501a5640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0480d0db0 con 0x7fc04806d570 2024-05-09T10:49:57.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.582+0000 7fc0501a5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0480d07a0 con 0x7fc04806eb30 2024-05-09T10:49:57.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.582+0000 7fc04d740640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc04806d570 0x7fc04806ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:57.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.582+0000 7fc04d740640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc04806d570 0x7fc04806ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38582/0 (socket says 172.21.15.98:38582) 2024-05-09T10:49:57.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.582+0000 7fc04e742640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc04815ddc0 0x7fc04815e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53130/0 (socket says 172.21.15.98:53130) 2024-05-09T10:49:57.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.582+0000 7fc04d740640 1 -- 172.21.15.98:0/394025069 learned_addr learned my addr 172.21.15.98:0/394025069 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:49:57.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.582+0000 7fc04df41640 1 --2- 172.21.15.98:0/394025069 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc04806eb30 0x7fc0480b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:57.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.582+0000 7fc04d740640 1 -- 172.21.15.98:0/394025069 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc04806eb30 msgr2=0x7fc0480b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:49:57.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.582+0000 7fc04d740640 1 --2- 172.21.15.98:0/394025069 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc04806eb30 0x7fc0480b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.582+0000 7fc04d740640 1 -- 172.21.15.98:0/394025069 >> v1:172.21.15.98:6789/0 conn(0x7fc04815ddc0 legacy=0x7fc04815e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:57.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.582+0000 7fc04d740640 1 -- 172.21.15.98:0/394025069 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04806f9b0 con 0x7fc04806d570 2024-05-09T10:49:57.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.582+0000 7fc04cf3f640 1 -- 172.21.15.98:0/394025069 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1113772985 0 0) 0x7fc040001490 con 0x7fc04815ddc0 2024-05-09T10:49:57.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.582+0000 7fc04df41640 1 --2- 172.21.15.98:0/394025069 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc04806eb30 0x7fc0480b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:49:57.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.583+0000 7fc04d740640 1 --2- 172.21.15.98:0/394025069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc04806d570 0x7fc04806ef10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc03c009870 tx=0x7fc03c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997c0d3458a017a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:49:57.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.583+0000 7fc04cf3f640 1 -- 172.21.15.98:0/394025069 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c037070 con 0x7fc04806d570 2024-05-09T10:49:57.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.583+0000 7fc04cf3f640 1 -- 172.21.15.98:0/394025069 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03c032440 con 0x7fc04806d570 2024-05-09T10:49:57.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.583+0000 7fc04cf3f640 1 -- 172.21.15.98:0/394025069 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c032810 con 0x7fc04806d570 2024-05-09T10:49:57.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.583+0000 7fc0501a5640 1 -- 172.21.15.98:0/394025069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc04806d570 msgr2=0x7fc04806ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:57.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.583+0000 7fc0501a5640 1 --2- 172.21.15.98:0/394025069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc04806d570 0x7fc04806ef10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc03c009870 tx=0x7fc03c02a1c0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.583+0000 7fc0501a5640 1 -- 172.21.15.98:0/394025069 shutdown_connections 2024-05-09T10:49:57.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.583+0000 7fc0501a5640 1 --2- 172.21.15.98:0/394025069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc04806d570 0x7fc04806ef10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.584+0000 7fc0501a5640 1 --2- 172.21.15.98:0/394025069 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc04806eb30 0x7fc0480b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.584+0000 7fc0501a5640 1 -- 172.21.15.98:0/394025069 >> 172.21.15.98:0/394025069 conn(0x7fc0480b34e0 msgr2=0x7fc0480b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:49:57.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.584+0000 7fc0501a5640 1 -- 172.21.15.98:0/394025069 shutdown_connections 2024-05-09T10:49:57.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.584+0000 7fc0501a5640 1 -- 172.21.15.98:0/394025069 wait complete. 2024-05-09T10:49:57.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.585+0000 7fc0501a5640 1 Processor -- start 2024-05-09T10:49:57.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.585+0000 7fc0501a5640 1 -- start start 2024-05-09T10:49:57.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.585+0000 7fc0501a5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc04806d570 0x7fc04815e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:57.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.585+0000 7fc0501a5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc04806eb30 0x7fc048159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:57.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.585+0000 7fc0501a5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc04815ddc0 0x7fc0480b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:57.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.586+0000 7fc0501a5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04806bea0 con 0x7fc04806eb30 2024-05-09T10:49:57.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.586+0000 7fc0501a5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0480d0a40 con 0x7fc04806d570 2024-05-09T10:49:57.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.586+0000 7fc0501a5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc04806c020 con 0x7fc04815ddc0 2024-05-09T10:49:57.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.586+0000 7fc04d740640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc04806eb30 0x7fc048159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:57.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.586+0000 7fc04d740640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc04806eb30 0x7fc048159220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38584/0 (socket says 172.21.15.98:38584) 2024-05-09T10:49:57.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.586+0000 7fc04d740640 1 -- 172.21.15.98:0/1053900798 learned_addr learned my addr 172.21.15.98:0/1053900798 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:49:57.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.586+0000 7fc04df41640 1 --2- 172.21.15.98:0/1053900798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc04806d570 0x7fc04815e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:57.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.586+0000 7fc04e742640 1 --2- 172.21.15.98:0/1053900798 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc04815ddc0 0x7fc0480b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:57.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.586+0000 7fc04d740640 1 -- 172.21.15.98:0/1053900798 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc04815ddc0 msgr2=0x7fc0480b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:57.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.586+0000 7fc04d740640 1 --2- 172.21.15.98:0/1053900798 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc04815ddc0 0x7fc0480b38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.586+0000 7fc04d740640 1 -- 172.21.15.98:0/1053900798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc04806d570 msgr2=0x7fc04815e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:57.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.586+0000 7fc04d740640 1 --2- 172.21.15.98:0/1053900798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc04806d570 0x7fc04815e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.586+0000 7fc04d740640 1 -- 172.21.15.98:0/1053900798 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc048159be0 con 0x7fc04806eb30 2024-05-09T10:49:57.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.586+0000 7fc04d740640 1 --2- 172.21.15.98:0/1053900798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc04806eb30 0x7fc048159220 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc03c000c00 tx=0x7fc03c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:49:57.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.586+0000 7fc03a7fc640 1 -- 172.21.15.98:0/1053900798 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c041020 con 0x7fc04806eb30 2024-05-09T10:49:57.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.586+0000 7fc03a7fc640 1 -- 172.21.15.98:0/1053900798 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03c03d620 con 0x7fc04806eb30 2024-05-09T10:49:57.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.586+0000 7fc0501a5640 1 -- 172.21.15.98:0/1053900798 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc048159ec0 con 0x7fc04806eb30 2024-05-09T10:49:57.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.586+0000 7fc03a7fc640 1 -- 172.21.15.98:0/1053900798 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c037040 con 0x7fc04806eb30 2024-05-09T10:49:57.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.586+0000 7fc0501a5640 1 -- 172.21.15.98:0/1053900798 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0480c8090 con 0x7fc04806eb30 2024-05-09T10:49:57.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.586+0000 7fc03a7fc640 1 -- 172.21.15.98:0/1053900798 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc03c044020 con 0x7fc04806eb30 2024-05-09T10:49:57.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.586+0000 7fc03a7fc640 1 -- 172.21.15.98:0/1053900798 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fc03c046b80 con 0x7fc04806eb30 2024-05-09T10:49:57.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.587+0000 7fc0501a5640 1 -- 172.21.15.98:0/1053900798 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0480c8260 con 0x7fc04806eb30 2024-05-09T10:49:57.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.589+0000 7fc03a7fc640 1 -- 172.21.15.98:0/1053900798 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc0480c8260 con 0x7fc04806eb30 2024-05-09T10:49:57.622 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:57.629+0000 7f476571b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T10:49:57.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.650+0000 7fc0501a5640 1 -- 172.21.15.98:0/1053900798 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0481612d0 con 0x7fc04806eb30 2024-05-09T10:49:57.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.651+0000 7fc03a7fc640 1 -- 172.21.15.98:0/1053900798 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fc03c04b030 con 0x7fc04806eb30 2024-05-09T10:49:57.644 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:49:57.644 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":10,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:49:57.400886+0000","last_up_change":"0.000000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:49:57.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.652+0000 7fc0501a5640 1 -- 172.21.15.98:0/1053900798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc04806eb30 msgr2=0x7fc048159220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:57.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.652+0000 7fc0501a5640 1 --2- 172.21.15.98:0/1053900798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc04806eb30 0x7fc048159220 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc03c000c00 tx=0x7fc03c0046c0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.652+0000 7fc0501a5640 1 -- 172.21.15.98:0/1053900798 shutdown_connections 2024-05-09T10:49:57.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.652+0000 7fc0501a5640 1 --2- 172.21.15.98:0/1053900798 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc04815ddc0 0x7fc0480b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.652+0000 7fc0501a5640 1 --2- 172.21.15.98:0/1053900798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc04806eb30 0x7fc048159220 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.652+0000 7fc0501a5640 1 --2- 172.21.15.98:0/1053900798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc04806d570 0x7fc04815e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:57.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.652+0000 7fc0501a5640 1 -- 172.21.15.98:0/1053900798 >> 172.21.15.98:0/1053900798 conn(0x7fc0480b34e0 msgr2=0x7fc04815c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:49:57.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.652+0000 7fc0501a5640 1 -- 172.21.15.98:0/1053900798 shutdown_connections 2024-05-09T10:49:57.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:49:57.652+0000 7fc0501a5640 1 -- 172.21.15.98:0/1053900798 wait complete. 2024-05-09T10:49:57.656 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-09T10:49:57.656 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-09T10:49:57.765 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:57.772+0000 7fb2d1723040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T10:49:57.819 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:57.826+0000 7fb2d1723040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T10:49:57.877 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:57.883+0000 7f476571b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T10:49:57.928 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:57.935+0000 7f476571b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T10:49:57.934 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:57.941+0000 7fb2d1723040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T10:49:58.026 INFO:tasks.ceph.osd.0.smithi098.stderr:2024-05-09T10:49:58.032+0000 7f9970388640 -1 Falling back to public interface 2024-05-09T10:49:58.029 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-05-09T10:49:58.036+0000 7f41ad4b5640 -1 Falling back to public interface 2024-05-09T10:49:58.034 INFO:tasks.ceph.osd.5.smithi105.stderr:2024-05-09T10:49:58.041+0000 7f6af5227640 -1 Falling back to public interface 2024-05-09T10:49:58.035 INFO:tasks.ceph.osd.4.smithi105.stderr:2024-05-09T10:49:58.042+0000 7f4b21e1b640 -1 Falling back to public interface 2024-05-09T10:49:58.037 INFO:tasks.ceph.osd.1.smithi098.stderr:2024-05-09T10:49:58.043+0000 7f5595484640 -1 Falling back to public interface 2024-05-09T10:49:58.038 INFO:tasks.ceph.osd.6.smithi105.stderr:2024-05-09T10:49:58.045+0000 7fc6b653e640 -1 Falling back to public interface 2024-05-09T10:49:58.040 INFO:tasks.ceph.osd.2.smithi098.stderr:2024-05-09T10:49:58.047+0000 7f85c981e640 -1 Falling back to public interface 2024-05-09T10:49:58.042 INFO:tasks.ceph.osd.7.smithi105.stderr:2024-05-09T10:49:58.049+0000 7fe98c91a640 -1 Falling back to public interface 2024-05-09T10:49:58.044 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:58.051+0000 7f476571b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T10:49:58.237 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:58.243+0000 7fb2d1723040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T10:49:58.284 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:58.291+0000 7fb2d1723040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T10:49:58.344 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:58.351+0000 7f476571b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T10:49:58.392 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:58.399+0000 7f476571b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T10:49:58.441 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:58.448+0000 7fb2d1723040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T10:49:58.461 DEBUG:teuthology.orchestra.run.smithi098:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-09T10:49:58.547 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:58.554+0000 7f476571b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T10:49:58.553 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.560+0000 7f58c2370640 1 Processor -- start 2024-05-09T10:49:58.553 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.560+0000 7f58c2370640 1 -- start start 2024-05-09T10:49:58.555 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.561+0000 7f58c2370640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f58bc074500 0x7f58bc0ace60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:58.555 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.561+0000 7f58c2370640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f58bc0d0c40 0x7f58bc06ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:58.555 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.561+0000 7f58c2370640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58bc0af400 con 0x7f58bc0b3f90 2024-05-09T10:49:58.555 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.561+0000 7f58c2370640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58bc06e5f0 con 0x7f58bc074500 2024-05-09T10:49:58.555 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.561+0000 7f58c2370640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc15ce40 con 0x7f58bc0d0c40 2024-05-09T10:49:58.555 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.561+0000 7f58c090d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f58bc0d0c40 0x7f58bc06ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:58.555 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.561+0000 7f58bb7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f58bc074500 0x7f58bc0ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:58.555 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.561+0000 7f58c090d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f58bc0d0c40 0x7f58bc06ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:49530/0 (socket says 172.21.15.98:49530) 2024-05-09T10:49:58.555 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.561+0000 7f58bbfff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f58bc0b3f90 0x7f58bc0b4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.98:50142/0 (socket says 172.21.15.98:50142) 2024-05-09T10:49:58.556 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.561+0000 7f58c090d640 1 -- 172.21.15.98:0/219973442 learned_addr learned my addr 172.21.15.98:0/219973442 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:49:58.556 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.561+0000 7f58c090d640 1 -- 172.21.15.98:0/219973442 >> v2:172.21.15.105:3300/0 conn(0x7f58bc074500 msgr2=0x7f58bc0ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:49:58.556 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.561+0000 7f58c090d640 1 --2- 172.21.15.98:0/219973442 >> v2:172.21.15.105:3300/0 conn(0x7f58bc074500 0x7f58bc0ace60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:58.556 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.561+0000 7f58c090d640 1 -- 172.21.15.98:0/219973442 >> v1:172.21.15.105:6789/0 conn(0x7f58bc0b3f90 legacy=0x7f58bc0b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:58.556 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.561+0000 7f58c090d640 1 -- 172.21.15.98:0/219973442 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc15cb60 con 0x7f58bc0d0c40 2024-05-09T10:49:58.556 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.561+0000 7f58baffd640 1 -- 172.21.15.98:0/219973442 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 665125853 0 0) 0x7f58bc0af400 con 0x7f58bc0b3f90 2024-05-09T10:49:58.556 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.561+0000 7f58bb7fe640 1 --2- 172.21.15.98:0/219973442 >> v2:172.21.15.105:3300/0 conn(0x7f58bc074500 0x7f58bc0ace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:49:58.557 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.562+0000 7f58c090d640 1 --2- 172.21.15.98:0/219973442 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f58bc0d0c40 0x7f58bc06ea80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f58bc06e770 tx=0x7f58b402a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b825636adcb14802 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:49:58.557 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.562+0000 7f58baffd640 1 -- 172.21.15.98:0/219973442 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b4036070 con 0x7f58bc0d0c40 2024-05-09T10:49:58.557 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.562+0000 7f58baffd640 1 -- 172.21.15.98:0/219973442 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b4031600 con 0x7f58bc0d0c40 2024-05-09T10:49:58.557 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.562+0000 7f58baffd640 1 -- 172.21.15.98:0/219973442 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b40319d0 con 0x7f58bc0d0c40 2024-05-09T10:49:58.558 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.563+0000 7f58c2370640 1 -- 172.21.15.98:0/219973442 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f58bc0d0c40 msgr2=0x7f58bc06ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:58.558 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.563+0000 7f58c2370640 1 --2- 172.21.15.98:0/219973442 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f58bc0d0c40 0x7f58bc06ea80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f58bc06e770 tx=0x7f58b402a7a0 comp rx=0 tx=0).stop 2024-05-09T10:49:58.558 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.563+0000 7f58c2370640 1 -- 172.21.15.98:0/219973442 shutdown_connections 2024-05-09T10:49:58.558 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.563+0000 7f58c2370640 1 --2- 172.21.15.98:0/219973442 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f58bc0d0c40 0x7f58bc06ea80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:58.558 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.563+0000 7f58c2370640 1 --2- 172.21.15.98:0/219973442 >> v2:172.21.15.105:3300/0 conn(0x7f58bc074500 0x7f58bc0ace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:58.558 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.563+0000 7f58c2370640 1 -- 172.21.15.98:0/219973442 >> 172.21.15.98:0/219973442 conn(0x7f58bc0cff90 msgr2=0x7f58bc0d03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:49:58.558 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.563+0000 7f58c2370640 1 -- 172.21.15.98:0/219973442 shutdown_connections 2024-05-09T10:49:58.558 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.563+0000 7f58c2370640 1 -- 172.21.15.98:0/219973442 wait complete. 2024-05-09T10:49:58.558 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.564+0000 7f58c2370640 1 Processor -- start 2024-05-09T10:49:58.561 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.564+0000 7f58c2370640 1 -- start start 2024-05-09T10:49:58.561 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.565+0000 7f58c2370640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58bc074500 0x7f58bc09d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:58.561 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.565+0000 7f58c2370640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58bc0b3f90 0x7f58bc150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:58.561 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.565+0000 7f58c2370640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f58bc0d0c40 0x7f58bc0cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:49:58.561 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.565+0000 7f58c2370640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc06ccf0 con 0x7f58bc074500 2024-05-09T10:49:58.561 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.565+0000 7f58c2370640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc06cb70 con 0x7f58bc0b3f90 2024-05-09T10:49:58.561 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.565+0000 7f58c2370640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc06ce70 con 0x7f58bc0d0c40 2024-05-09T10:49:58.561 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.565+0000 7f58bbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58bc074500 0x7f58bc09d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:58.561 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.565+0000 7f58bbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58bc074500 0x7f58bc09d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38594/0 (socket says 172.21.15.98:38594) 2024-05-09T10:49:58.561 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.565+0000 7f58bbfff640 1 -- 172.21.15.98:0/2684235361 learned_addr learned my addr 172.21.15.98:0/2684235361 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:49:58.561 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.565+0000 7f58c090d640 1 --2- 172.21.15.98:0/2684235361 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f58bc0d0c40 0x7f58bc0cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:58.562 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.565+0000 7f58bb7fe640 1 --2- 172.21.15.98:0/2684235361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58bc0b3f90 0x7f58bc150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:49:58.562 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.566+0000 7f58bbfff640 1 -- 172.21.15.98:0/2684235361 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f58bc0d0c40 msgr2=0x7f58bc0cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:58.562 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.566+0000 7f58bbfff640 1 --2- 172.21.15.98:0/2684235361 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f58bc0d0c40 0x7f58bc0cdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:58.562 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.566+0000 7f58bbfff640 1 -- 172.21.15.98:0/2684235361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58bc0b3f90 msgr2=0x7f58bc150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:58.562 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.566+0000 7f58bbfff640 1 --2- 172.21.15.98:0/2684235361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58bc0b3f90 0x7f58bc150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:58.562 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.566+0000 7f58bbfff640 1 -- 172.21.15.98:0/2684235361 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc09b960 con 0x7f58bc074500 2024-05-09T10:49:58.562 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.566+0000 7f58bbfff640 1 --2- 172.21.15.98:0/2684235361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58bc074500 0x7f58bc09d060 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00c4d0 tx=0x7f58ac00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:49:58.562 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.566+0000 7f58b8ff9640 1 -- 172.21.15.98:0/2684235361 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac018070 con 0x7f58bc074500 2024-05-09T10:49:58.562 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.566+0000 7f58b8ff9640 1 -- 172.21.15.98:0/2684235361 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac007500 con 0x7f58bc074500 2024-05-09T10:49:58.562 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.566+0000 7f58c2370640 1 -- 172.21.15.98:0/2684235361 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58bc09bc40 con 0x7f58bc074500 2024-05-09T10:49:58.562 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.566+0000 7f58b8ff9640 1 -- 172.21.15.98:0/2684235361 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac004a20 con 0x7f58bc074500 2024-05-09T10:49:58.562 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.566+0000 7f58c2370640 1 -- 172.21.15.98:0/2684235361 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58bc09c200 con 0x7f58bc074500 2024-05-09T10:49:58.562 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.566+0000 7f58b8ff9640 1 -- 172.21.15.98:0/2684235361 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f58ac01b070 con 0x7f58bc074500 2024-05-09T10:49:58.562 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.566+0000 7f58b8ff9640 1 -- 172.21.15.98:0/2684235361 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f58ac012ad0 con 0x7f58bc074500 2024-05-09T10:49:58.563 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.567+0000 7f58c2370640 1 -- 172.21.15.98:0/2684235361 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58bc152ab0 con 0x7f58bc074500 2024-05-09T10:49:58.563 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.569+0000 7f58b8ff9640 1 -- 172.21.15.98:0/2684235361 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f58ac004bc0 con 0x7f58bc074500 2024-05-09T10:49:58.584 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:58.591+0000 7fb2d1723040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T10:49:58.623 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.630+0000 7f58c2370640 1 -- 172.21.15.98:0/2684235361 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58bc15e7a0 con 0x7f58bc074500 2024-05-09T10:49:58.624 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.631+0000 7f58b8ff9640 1 -- 172.21.15.98:0/2684235361 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f58ac004da0 con 0x7f58bc074500 2024-05-09T10:49:58.624 INFO:teuthology.misc.health.smithi098.stdout: 2024-05-09T10:49:58.624 INFO:teuthology.misc.health.smithi098.stdout:{"epoch":10,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:49:57.400886+0000","last_up_change":"0.000000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:49:58.625 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.632+0000 7f58c2370640 1 -- 172.21.15.98:0/2684235361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58bc074500 msgr2=0x7f58bc09d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:49:58.625 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.632+0000 7f58c2370640 1 --2- 172.21.15.98:0/2684235361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58bc074500 0x7f58bc09d060 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00c4d0 tx=0x7f58ac00c990 comp rx=0 tx=0).stop 2024-05-09T10:49:58.625 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.632+0000 7f58c2370640 1 -- 172.21.15.98:0/2684235361 shutdown_connections 2024-05-09T10:49:58.625 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.632+0000 7f58c2370640 1 --2- 172.21.15.98:0/2684235361 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f58bc0d0c40 0x7f58bc0cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:58.626 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.632+0000 7f58c2370640 1 --2- 172.21.15.98:0/2684235361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58bc0b3f90 0x7f58bc150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:58.626 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.632+0000 7f58c2370640 1 --2- 172.21.15.98:0/2684235361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58bc074500 0x7f58bc09d060 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:49:58.626 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.632+0000 7f58c2370640 1 -- 172.21.15.98:0/2684235361 >> 172.21.15.98:0/2684235361 conn(0x7f58bc0cff90 msgr2=0x7f58bc15cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:49:58.626 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.632+0000 7f58c2370640 1 -- 172.21.15.98:0/2684235361 shutdown_connections 2024-05-09T10:49:58.626 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:49:58.632+0000 7f58c2370640 1 -- 172.21.15.98:0/2684235361 wait complete. 2024-05-09T10:49:58.635 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-09T10:49:58.636 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:58.643+0000 7fb2d1723040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T10:49:58.685 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:58.691+0000 7f476571b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T10:49:58.707 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:58.714+0000 7fb2d1723040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T10:49:58.734 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:58.740+0000 7f476571b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T10:49:58.752 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:58.759+0000 7fb2d1723040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T10:49:58.803 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:58.809+0000 7f476571b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T10:49:58.848 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:58.855+0000 7f476571b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T10:49:59.233 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:59.239+0000 7fb2d1723040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T10:49:59.320 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:59.327+0000 7f476571b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T10:49:59.346 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:49:59.353+0000 7fb2d1723040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T10:49:59.440 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-05-09T10:49:59.447+0000 7f476571b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T10:50:00.362 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:50:00.368+0000 7fb2722d6640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-09T10:50:00.362 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:50:00.368+0000 7fb2722d6640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-09T10:50:01.502 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-05-09T10:50:01.509+0000 7f41ad4b5640 -1 osd.3 0 log_to_monitors true 2024-05-09T10:50:01.513 INFO:tasks.ceph.osd.0.smithi098.stderr:2024-05-09T10:50:01.520+0000 7f9970388640 -1 osd.0 0 log_to_monitors true 2024-05-09T10:50:01.525 INFO:tasks.ceph.osd.5.smithi105.stderr:2024-05-09T10:50:01.530+0000 7f6af5227640 -1 osd.5 0 log_to_monitors true 2024-05-09T10:50:01.530 INFO:tasks.ceph.osd.4.smithi105.stderr:2024-05-09T10:50:01.536+0000 7f4b21e1b640 -1 osd.4 0 log_to_monitors true 2024-05-09T10:50:01.551 INFO:tasks.ceph.osd.1.smithi098.stderr:2024-05-09T10:50:01.558+0000 7f5595484640 -1 osd.1 0 log_to_monitors true 2024-05-09T10:50:01.551 INFO:tasks.ceph.osd.2.smithi098.stderr:2024-05-09T10:50:01.558+0000 7f85c981e640 -1 osd.2 0 log_to_monitors true 2024-05-09T10:50:01.575 INFO:tasks.ceph.osd.7.smithi105.stderr:2024-05-09T10:50:01.580+0000 7fe98c91a640 -1 osd.7 0 log_to_monitors true 2024-05-09T10:50:01.602 INFO:tasks.ceph.osd.6.smithi105.stderr:2024-05-09T10:50:01.607+0000 7fc6b653e640 -1 osd.6 0 log_to_monitors true 2024-05-09T10:50:05.391 INFO:tasks.ceph.osd.1.smithi098.stderr:2024-05-09T10:50:05.398+0000 7f5590bf7640 -1 osd.1 0 waiting for initial osdmap 2024-05-09T10:50:05.395 INFO:tasks.ceph.osd.1.smithi098.stderr:2024-05-09T10:50:05.402+0000 7f5589229640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T10:50:05.413 INFO:tasks.ceph.osd.6.smithi105.stderr:2024-05-09T10:50:05.420+0000 7fc6b1cb1640 -1 osd.6 0 waiting for initial osdmap 2024-05-09T10:50:05.421 INFO:tasks.ceph.osd.5.smithi105.stderr:2024-05-09T10:50:05.428+0000 7f6af099a640 -1 osd.5 0 waiting for initial osdmap 2024-05-09T10:50:05.432 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-05-09T10:50:05.439+0000 7f41a8c28640 -1 osd.3 0 waiting for initial osdmap 2024-05-09T10:50:05.433 INFO:tasks.ceph.osd.6.smithi105.stderr:2024-05-09T10:50:05.440+0000 7fc6aa2e3640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T10:50:05.437 INFO:tasks.ceph.osd.5.smithi105.stderr:2024-05-09T10:50:05.444+0000 7f6ae8fcc640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T10:50:05.441 DEBUG:teuthology.orchestra.run.smithi098:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:05.441 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-05-09T10:50:05.448+0000 7f41a125a640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T10:50:05.447 INFO:tasks.ceph.osd.2.smithi098.stderr:2024-05-09T10:50:05.454+0000 7f85c48fa640 -1 osd.2 0 waiting for initial osdmap 2024-05-09T10:50:05.462 INFO:tasks.ceph.osd.2.smithi098.stderr:2024-05-09T10:50:05.469+0000 7f85bcf2c640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T10:50:05.467 INFO:tasks.ceph.osd.4.smithi105.stderr:2024-05-09T10:50:05.474+0000 7f4b1cfe4640 -1 osd.4 0 waiting for initial osdmap 2024-05-09T10:50:05.467 INFO:tasks.ceph.osd.7.smithi105.stderr:2024-05-09T10:50:05.474+0000 7fe98808d640 -1 osd.7 0 waiting for initial osdmap 2024-05-09T10:50:05.473 INFO:tasks.ceph.osd.0.smithi098.stderr:2024-05-09T10:50:05.480+0000 7f996bafb640 -1 osd.0 0 waiting for initial osdmap 2024-05-09T10:50:05.478 INFO:tasks.ceph.osd.4.smithi105.stderr:2024-05-09T10:50:05.485+0000 7f4b15616640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T10:50:05.481 INFO:tasks.ceph.osd.0.smithi098.stderr:2024-05-09T10:50:05.488+0000 7f996412d640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T10:50:05.482 INFO:tasks.ceph.osd.7.smithi105.stderr:2024-05-09T10:50:05.489+0000 7fe9806bf640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T10:50:05.535 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.541+0000 7f9879163640 1 Processor -- start 2024-05-09T10:50:05.535 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.541+0000 7f9879163640 1 -- start start 2024-05-09T10:50:05.537 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.542+0000 7f9879163640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f98740ce940 0x7f98740b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:05.537 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.542+0000 7f9879163640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f98740d0ee0 0x7f9874161ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:05.537 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.542+0000 7f9879163640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98740b5820 con 0x7f98740ced20 2024-05-09T10:50:05.537 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.542+0000 7f9879163640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98740b7300 con 0x7f98740d0ee0 2024-05-09T10:50:05.537 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.542+0000 7f9879163640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98740b5ac0 con 0x7f98740ce940 2024-05-09T10:50:05.537 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.542+0000 7f987359e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f98740d0ee0 0x7f9874161ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:05.537 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.542+0000 7f987359e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f98740d0ee0 0x7f9874161ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50648/0 (socket says 172.21.15.98:50648) 2024-05-09T10:50:05.537 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.542+0000 7f987259c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f98740ced20 0x7f98740d07d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40186/0 (socket says 172.21.15.98:40186) 2024-05-09T10:50:05.537 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.542+0000 7f987359e640 1 -- 172.21.15.98:0/2926148750 learned_addr learned my addr 172.21.15.98:0/2926148750 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:05.537 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.542+0000 7f9872d9d640 1 --2- 172.21.15.98:0/2926148750 >> v2:172.21.15.105:3300/0 conn(0x7f98740ce940 0x7f98740b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:05.537 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.543+0000 7f987359e640 1 -- 172.21.15.98:0/2926148750 >> v2:172.21.15.105:3300/0 conn(0x7f98740ce940 msgr2=0x7f98740b6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:05.537 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.543+0000 7f987359e640 1 --2- 172.21.15.98:0/2926148750 >> v2:172.21.15.105:3300/0 conn(0x7f98740ce940 0x7f98740b6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:05.537 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.543+0000 7f987359e640 1 -- 172.21.15.98:0/2926148750 >> v1:172.21.15.98:6789/0 conn(0x7f98740ced20 legacy=0x7f98740d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:05.538 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.543+0000 7f987359e640 1 -- 172.21.15.98:0/2926148750 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98740b5da0 con 0x7f98740d0ee0 2024-05-09T10:50:05.538 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.543+0000 7f9871d9b640 1 -- 172.21.15.98:0/2926148750 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478494880 0 0) 0x7f98740b5820 con 0x7f98740ced20 2024-05-09T10:50:05.538 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.543+0000 7f987359e640 1 --2- 172.21.15.98:0/2926148750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98740d0ee0 0x7f9874161ab0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9874067800 tx=0x7f986c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155d27405803b1b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:05.538 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.543+0000 7f9871d9b640 1 -- 172.21.15.98:0/2926148750 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c037070 con 0x7f98740d0ee0 2024-05-09T10:50:05.538 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.543+0000 7f9871d9b640 1 -- 172.21.15.98:0/2926148750 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986c032440 con 0x7f98740d0ee0 2024-05-09T10:50:05.538 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.543+0000 7f9871d9b640 1 -- 172.21.15.98:0/2926148750 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c032810 con 0x7f98740d0ee0 2024-05-09T10:50:05.541 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.544+0000 7f9879163640 1 -- 172.21.15.98:0/2926148750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98740d0ee0 msgr2=0x7f9874161ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:05.541 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.544+0000 7f9879163640 1 --2- 172.21.15.98:0/2926148750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98740d0ee0 0x7f9874161ab0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9874067800 tx=0x7f986c02a270 comp rx=0 tx=0).stop 2024-05-09T10:50:05.541 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.544+0000 7f9879163640 1 -- 172.21.15.98:0/2926148750 shutdown_connections 2024-05-09T10:50:05.541 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.544+0000 7f9879163640 1 --2- 172.21.15.98:0/2926148750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98740d0ee0 0x7f9874161ab0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:05.541 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.544+0000 7f9879163640 1 --2- 172.21.15.98:0/2926148750 >> v2:172.21.15.105:3300/0 conn(0x7f98740ce940 0x7f98740b6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:05.541 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.544+0000 7f9879163640 1 -- 172.21.15.98:0/2926148750 >> 172.21.15.98:0/2926148750 conn(0x7f9874066870 msgr2=0x7f9874066c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:05.541 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.544+0000 7f9879163640 1 -- 172.21.15.98:0/2926148750 shutdown_connections 2024-05-09T10:50:05.541 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.544+0000 7f9879163640 1 -- 172.21.15.98:0/2926148750 wait complete. 2024-05-09T10:50:05.541 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.545+0000 7f9879163640 1 Processor -- start 2024-05-09T10:50:05.542 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.545+0000 7f9879163640 1 -- start start 2024-05-09T10:50:05.542 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.546+0000 7f9879163640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98740ce940 0x7f98741593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:05.542 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.546+0000 7f9879163640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98740ced20 0x7f987415f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:05.542 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.546+0000 7f9879163640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98740d0ee0 0x7f987415f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:05.542 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.546+0000 7f9879163640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987415e120 con 0x7f98740ce940 2024-05-09T10:50:05.542 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.546+0000 7f9879163640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98740b6360 con 0x7f98740ced20 2024-05-09T10:50:05.542 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.546+0000 7f9879163640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98740cf180 con 0x7f98740d0ee0 2024-05-09T10:50:05.542 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.546+0000 7f9872d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98740ce940 0x7f98741593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:05.542 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.546+0000 7f9872d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98740ce940 0x7f98741593d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50662/0 (socket says 172.21.15.98:50662) 2024-05-09T10:50:05.542 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.546+0000 7f9872d9d640 1 -- 172.21.15.98:0/978971704 learned_addr learned my addr 172.21.15.98:0/978971704 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:05.542 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.546+0000 7f987359e640 1 --2- 172.21.15.98:0/978971704 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98740d0ee0 0x7f987415f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:05.542 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.546+0000 7f987259c640 1 --2- 172.21.15.98:0/978971704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98740ced20 0x7f987415f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:05.542 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.547+0000 7f9872d9d640 1 -- 172.21.15.98:0/978971704 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98740d0ee0 msgr2=0x7f987415f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:05.543 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.547+0000 7f9872d9d640 1 --2- 172.21.15.98:0/978971704 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98740d0ee0 0x7f987415f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:05.543 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.547+0000 7f9872d9d640 1 -- 172.21.15.98:0/978971704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98740ced20 msgr2=0x7f987415f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:05.543 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.547+0000 7f9872d9d640 1 --2- 172.21.15.98:0/978971704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98740ced20 0x7f987415f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:05.543 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.547+0000 7f9872d9d640 1 -- 172.21.15.98:0/978971704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987415ff40 con 0x7f98740ce940 2024-05-09T10:50:05.543 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.547+0000 7f9872d9d640 1 --2- 172.21.15.98:0/978971704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98740ce940 0x7f98741593d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f986400efc0 tx=0x7f986400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:05.543 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.547+0000 7f985b7fe640 1 -- 172.21.15.98:0/978971704 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864018020 con 0x7f98740ce940 2024-05-09T10:50:05.543 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.547+0000 7f985b7fe640 1 -- 172.21.15.98:0/978971704 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9864014480 con 0x7f98740ce940 2024-05-09T10:50:05.543 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.547+0000 7f9879163640 1 -- 172.21.15.98:0/978971704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9874160220 con 0x7f98740ce940 2024-05-09T10:50:05.543 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.547+0000 7f985b7fe640 1 -- 172.21.15.98:0/978971704 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864014850 con 0x7f98740ce940 2024-05-09T10:50:05.543 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.547+0000 7f9879163640 1 -- 172.21.15.98:0/978971704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98741607e0 con 0x7f98740ce940 2024-05-09T10:50:05.544 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.548+0000 7f9879163640 1 -- 172.21.15.98:0/978971704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9874068560 con 0x7f98740ce940 2024-05-09T10:50:05.544 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.548+0000 7f985b7fe640 1 -- 172.21.15.98:0/978971704 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f9864005d10 con 0x7f98740ce940 2024-05-09T10:50:05.544 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.549+0000 7f985b7fe640 1 --2- 172.21.15.98:0/978971704 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f986006b440 0x7f986006d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:05.544 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.549+0000 7f987259c640 1 --2- 172.21.15.98:0/978971704 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f986006b440 0x7f986006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:05.544 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.549+0000 7f985b7fe640 1 -- 172.21.15.98:0/978971704 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f9864089d30 con 0x7f98740ce940 2024-05-09T10:50:05.544 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.549+0000 7f987259c640 1 --2- 172.21.15.98:0/978971704 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f986006b440 0x7f986006d900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f98740c67c0 tx=0x7f9868007de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:05.546 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.553+0000 7f985b7fe640 1 -- 172.21.15.98:0/978971704 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f986408e070 con 0x7f98740ce940 2024-05-09T10:50:05.718 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.725+0000 7f9879163640 1 -- 172.21.15.98:0/978971704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98740d1910 con 0x7f98740ce940 2024-05-09T10:50:05.719 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.725+0000 7f985b7fe640 1 -- 172.21.15.98:0/978971704 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f9864022070 con 0x7f98740ce940 2024-05-09T10:50:05.719 INFO:teuthology.misc.health.smithi098.stdout: 2024-05-09T10:50:05.719 INFO:teuthology.misc.health.smithi098.stdout:{"epoch":12,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:03.375373+0000","last_up_change":"0.000000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:05.721 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.727+0000 7f9879163640 1 -- 172.21.15.98:0/978971704 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f986006b440 msgr2=0x7f986006d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:05.721 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.727+0000 7f9879163640 1 --2- 172.21.15.98:0/978971704 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f986006b440 0x7f986006d900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f98740c67c0 tx=0x7f9868007de0 comp rx=0 tx=0).stop 2024-05-09T10:50:05.721 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.727+0000 7f9879163640 1 -- 172.21.15.98:0/978971704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98740ce940 msgr2=0x7f98741593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:05.721 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.727+0000 7f9879163640 1 --2- 172.21.15.98:0/978971704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98740ce940 0x7f98741593d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f986400efc0 tx=0x7f986400c490 comp rx=0 tx=0).stop 2024-05-09T10:50:05.721 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.727+0000 7f9879163640 1 -- 172.21.15.98:0/978971704 shutdown_connections 2024-05-09T10:50:05.721 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.727+0000 7f9879163640 1 --2- 172.21.15.98:0/978971704 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f986006b440 0x7f986006d900 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:05.721 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.727+0000 7f9879163640 1 --2- 172.21.15.98:0/978971704 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98740d0ee0 0x7f987415f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:05.721 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.727+0000 7f9879163640 1 --2- 172.21.15.98:0/978971704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98740ced20 0x7f987415f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:05.721 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.727+0000 7f9879163640 1 --2- 172.21.15.98:0/978971704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98740ce940 0x7f98741593d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:05.721 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.727+0000 7f9879163640 1 -- 172.21.15.98:0/978971704 >> 172.21.15.98:0/978971704 conn(0x7f9874066870 msgr2=0x7f98740cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:05.722 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.727+0000 7f9879163640 1 -- 172.21.15.98:0/978971704 shutdown_connections 2024-05-09T10:50:05.722 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:05.727+0000 7f9879163640 1 -- 172.21.15.98:0/978971704 wait complete. 2024-05-09T10:50:05.730 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-09T10:50:12.538 DEBUG:teuthology.orchestra.run.smithi098:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:12.641 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.647+0000 7f91bea76640 1 Processor -- start 2024-05-09T10:50:12.641 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.647+0000 7f91bea76640 1 -- start start 2024-05-09T10:50:12.642 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.648+0000 7f91bea76640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f91b80b6dc0 0x7f91b80d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:12.642 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.648+0000 7f91bea76640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f91b80d0d10 0x7f91b8161ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:12.642 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.648+0000 7f91bea76640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b80b5820 con 0x7f91b80ce940 2024-05-09T10:50:12.643 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.648+0000 7f91bea76640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b80b5ac0 con 0x7f91b80b6dc0 2024-05-09T10:50:12.643 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.648+0000 7f91bea76640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b80b5da0 con 0x7f91b80d0d10 2024-05-09T10:50:12.643 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.648+0000 7f91b7fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f91b80b6dc0 0x7f91b80d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:12.643 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.648+0000 7f91b7fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f91b80b6dc0 0x7f91b80d07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50690/0 (socket says 172.21.15.98:50690) 2024-05-09T10:50:12.643 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.648+0000 7f91b7fff640 1 -- 172.21.15.98:0/1573492991 learned_addr learned my addr 172.21.15.98:0/1573492991 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:12.643 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.648+0000 7f91bd013640 1 --2- 172.21.15.98:0/1573492991 >> v2:172.21.15.105:3300/0 conn(0x7f91b80d0d10 0x7f91b8161ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:12.643 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.649+0000 7f91b7fff640 1 -- 172.21.15.98:0/1573492991 >> v2:172.21.15.105:3300/0 conn(0x7f91b80d0d10 msgr2=0x7f91b8161ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:12.643 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.649+0000 7f91b7fff640 1 --2- 172.21.15.98:0/1573492991 >> v2:172.21.15.105:3300/0 conn(0x7f91b80d0d10 0x7f91b8161ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:12.643 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.649+0000 7f91b7fff640 1 -- 172.21.15.98:0/1573492991 >> v1:172.21.15.105:6789/0 conn(0x7f91b80ce940 legacy=0x7f91b80ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:12.643 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.649+0000 7f91b7fff640 1 -- 172.21.15.98:0/1573492991 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b80b6080 con 0x7f91b80b6dc0 2024-05-09T10:50:12.643 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.649+0000 7f91b7fff640 1 --2- 172.21.15.98:0/1573492991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91b80b6dc0 0x7f91b80d07d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f91b80677b0 tx=0x7f91ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac62c3ae5eb199f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:12.645 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.649+0000 7f91b77fe640 1 -- 172.21.15.98:0/1573492991 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac037070 con 0x7f91b80b6dc0 2024-05-09T10:50:12.645 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.650+0000 7f91b77fe640 1 -- 172.21.15.98:0/1573492991 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ac032440 con 0x7f91b80b6dc0 2024-05-09T10:50:12.645 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.650+0000 7f91b77fe640 1 -- 172.21.15.98:0/1573492991 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac032810 con 0x7f91b80b6dc0 2024-05-09T10:50:12.645 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.650+0000 7f91bea76640 1 -- 172.21.15.98:0/1573492991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91b80b6dc0 msgr2=0x7f91b80d07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:12.645 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.650+0000 7f91bea76640 1 --2- 172.21.15.98:0/1573492991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91b80b6dc0 0x7f91b80d07d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f91b80677b0 tx=0x7f91ac02a270 comp rx=0 tx=0).stop 2024-05-09T10:50:12.645 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.650+0000 7f91bea76640 1 -- 172.21.15.98:0/1573492991 shutdown_connections 2024-05-09T10:50:12.645 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.650+0000 7f91bea76640 1 --2- 172.21.15.98:0/1573492991 >> v2:172.21.15.105:3300/0 conn(0x7f91b80d0d10 0x7f91b8161ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:12.645 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.650+0000 7f91bea76640 1 --2- 172.21.15.98:0/1573492991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91b80b6dc0 0x7f91b80d07d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:12.645 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.650+0000 7f91bea76640 1 -- 172.21.15.98:0/1573492991 >> 172.21.15.98:0/1573492991 conn(0x7f91b8066870 msgr2=0x7f91b8066c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:12.645 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.650+0000 7f91bea76640 1 -- 172.21.15.98:0/1573492991 shutdown_connections 2024-05-09T10:50:12.645 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.650+0000 7f91bea76640 1 -- 172.21.15.98:0/1573492991 wait complete. 2024-05-09T10:50:12.646 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.651+0000 7f91bea76640 1 Processor -- start 2024-05-09T10:50:12.646 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.651+0000 7f91bea76640 1 -- start start 2024-05-09T10:50:12.648 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.652+0000 7f91bea76640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f91b80b6dc0 0x7f91b80c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:12.648 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.652+0000 7f91bea76640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91b80ce940 0x7f91b81e67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:12.648 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.652+0000 7f91bea76640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91b80d0d10 0x7f91b80d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:12.648 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.652+0000 7f91bea76640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b80b7220 con 0x7f91b80d0d10 2024-05-09T10:50:12.648 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.652+0000 7f91bea76640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b80d11a0 con 0x7f91b80ce940 2024-05-09T10:50:12.648 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.652+0000 7f91bea76640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b815e120 con 0x7f91b80b6dc0 2024-05-09T10:50:12.648 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.652+0000 7f91bd013640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91b80d0d10 0x7f91b80d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:12.648 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.652+0000 7f91bd013640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91b80d0d10 0x7f91b80d23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50698/0 (socket says 172.21.15.98:50698) 2024-05-09T10:50:12.648 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.652+0000 7f91bd013640 1 -- 172.21.15.98:0/3464561853 learned_addr learned my addr 172.21.15.98:0/3464561853 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:12.648 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.653+0000 7f91b7fff640 1 --2- 172.21.15.98:0/3464561853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91b80ce940 0x7f91b81e67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:12.648 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.653+0000 7f91bc812640 1 --2- 172.21.15.98:0/3464561853 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f91b80b6dc0 0x7f91b80c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:12.648 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.653+0000 7f91bd013640 1 -- 172.21.15.98:0/3464561853 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f91b80b6dc0 msgr2=0x7f91b80c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:12.649 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.653+0000 7f91bd013640 1 --2- 172.21.15.98:0/3464561853 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f91b80b6dc0 0x7f91b80c6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:12.649 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.653+0000 7f91bd013640 1 -- 172.21.15.98:0/3464561853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91b80ce940 msgr2=0x7f91b81e67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:12.649 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.653+0000 7f91bd013640 1 --2- 172.21.15.98:0/3464561853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91b80ce940 0x7f91b81e67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:12.649 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.653+0000 7f91bd013640 1 -- 172.21.15.98:0/3464561853 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b8145750 con 0x7f91b80d0d10 2024-05-09T10:50:12.649 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.653+0000 7f91b7fff640 1 --2- 172.21.15.98:0/3464561853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91b80ce940 0x7f91b81e67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:12.649 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.653+0000 7f91bd013640 1 --2- 172.21.15.98:0/3464561853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91b80d0d10 0x7f91b80d23c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f91b0007cf0 tx=0x7f91b000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:12.649 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.653+0000 7f91b57fa640 1 -- 172.21.15.98:0/3464561853 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b0014420 con 0x7f91b80d0d10 2024-05-09T10:50:12.649 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.653+0000 7f91b57fa640 1 -- 172.21.15.98:0/3464561853 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b0019070 con 0x7f91b80d0d10 2024-05-09T10:50:12.649 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.653+0000 7f91bea76640 1 -- 172.21.15.98:0/3464561853 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b8145a30 con 0x7f91b80d0d10 2024-05-09T10:50:12.649 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.653+0000 7f91b57fa640 1 -- 172.21.15.98:0/3464561853 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b0014820 con 0x7f91b80d0d10 2024-05-09T10:50:12.649 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.653+0000 7f91bea76640 1 -- 172.21.15.98:0/3464561853 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b8145ff0 con 0x7f91b80d0d10 2024-05-09T10:50:12.649 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.654+0000 7f91bea76640 1 -- 172.21.15.98:0/3464561853 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b8068560 con 0x7f91b80d0d10 2024-05-09T10:50:12.649 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.655+0000 7f91b57fa640 1 -- 172.21.15.98:0/3464561853 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f91b00149c0 con 0x7f91b80d0d10 2024-05-09T10:50:12.650 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.655+0000 7f91b57fa640 1 --2- 172.21.15.98:0/3464561853 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f91a406b4d0 0x7f91a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:12.650 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.655+0000 7f91bc812640 1 --2- 172.21.15.98:0/3464561853 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f91a406b4d0 0x7f91a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:12.650 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.655+0000 7f91b57fa640 1 -- 172.21.15.98:0/3464561853 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f91b008a5e0 con 0x7f91b80d0d10 2024-05-09T10:50:12.650 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.656+0000 7f91bc812640 1 --2- 172.21.15.98:0/3464561853 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f91a406b4d0 0x7f91a406d990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f91a8007fd0 tx=0x7f91a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:12.653 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.660+0000 7f91b57fa640 1 -- 172.21.15.98:0/3464561853 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f91b00597b0 con 0x7f91b80d0d10 2024-05-09T10:50:12.823 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.829+0000 7f91bea76640 1 -- 172.21.15.98:0/3464561853 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91b80d1910 con 0x7f91b80d0d10 2024-05-09T10:50:12.823 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.830+0000 7f91b57fa640 1 -- 172.21.15.98:0/3464561853 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12884 (secure 0 0 0) 0x7f91b00590b0 con 0x7f91b80d0d10 2024-05-09T10:50:12.824 INFO:teuthology.misc.health.smithi098.stdout: 2024-05-09T10:50:12.824 INFO:teuthology.misc.health.smithi098.stdout:{"epoch":16,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:09.414442+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:12.826 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.832+0000 7f91bea76640 1 -- 172.21.15.98:0/3464561853 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f91a406b4d0 msgr2=0x7f91a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:12.826 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.832+0000 7f91bea76640 1 --2- 172.21.15.98:0/3464561853 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f91a406b4d0 0x7f91a406d990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f91a8007fd0 tx=0x7f91a8009040 comp rx=0 tx=0).stop 2024-05-09T10:50:12.826 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.832+0000 7f91bea76640 1 -- 172.21.15.98:0/3464561853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91b80d0d10 msgr2=0x7f91b80d23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:12.826 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.832+0000 7f91bea76640 1 --2- 172.21.15.98:0/3464561853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91b80d0d10 0x7f91b80d23c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f91b0007cf0 tx=0x7f91b000e4b0 comp rx=0 tx=0).stop 2024-05-09T10:50:12.826 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.832+0000 7f91bea76640 1 -- 172.21.15.98:0/3464561853 shutdown_connections 2024-05-09T10:50:12.826 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.832+0000 7f91bea76640 1 --2- 172.21.15.98:0/3464561853 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f91a406b4d0 0x7f91a406d990 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:12.826 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.832+0000 7f91bea76640 1 --2- 172.21.15.98:0/3464561853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91b80d0d10 0x7f91b80d23c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:12.826 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.832+0000 7f91bea76640 1 --2- 172.21.15.98:0/3464561853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91b80ce940 0x7f91b81e67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:12.826 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.832+0000 7f91bea76640 1 --2- 172.21.15.98:0/3464561853 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f91b80b6dc0 0x7f91b80c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:12.827 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.832+0000 7f91bea76640 1 -- 172.21.15.98:0/3464561853 >> 172.21.15.98:0/3464561853 conn(0x7f91b8066870 msgr2=0x7f91b8160cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:12.827 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.832+0000 7f91bea76640 1 -- 172.21.15.98:0/3464561853 shutdown_connections 2024-05-09T10:50:12.827 INFO:teuthology.misc.health.smithi098.stderr:2024-05-09T10:50:12.832+0000 7f91bea76640 1 -- 172.21.15.98:0/3464561853 wait complete. 2024-05-09T10:50:12.835 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-09T10:50:12.835 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-09T10:50:12.836 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-09T10:50:12.836 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-09T10:50:12.838 INFO:tasks.ceph.mds.a:Started 2024-05-09T10:50:12.838 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-09T10:50:12.838 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-09T10:50:12.840 INFO:tasks.ceph.mds.c:Started 2024-05-09T10:50:12.840 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-09T10:50:12.840 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-09T10:50:12.843 INFO:tasks.ceph.mds.b:Started 2024-05-09T10:50:12.843 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-09T10:50:12.843 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-09T10:50:12.846 INFO:tasks.ceph.mds.d:Started 2024-05-09T10:50:12.846 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-09T10:50:12.846 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:12.912 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2024-05-09T10:50:12.912 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2024-05-09T10:50:12.929 INFO:tasks.ceph.mds.b.smithi105.stdout:starting mds.b at 2024-05-09T10:50:12.929 INFO:tasks.ceph.mds.d.smithi105.stdout:starting mds.d at 2024-05-09T10:50:12.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.957+0000 7f6d0eff5640 1 Processor -- start 2024-05-09T10:50:12.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.957+0000 7f6d0eff5640 1 -- start start 2024-05-09T10:50:12.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.958+0000 7f6d0eff5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6d0815dea0 0x7f6d080ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:12.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.958+0000 7f6d0eff5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d080cece0 0x7f6d080cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:12.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.958+0000 7f6d0eff5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6d080d0340 0x7f6d080d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:12.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.958+0000 7f6d0eff5640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d080d2b80 con 0x7f6d080d0340 2024-05-09T10:50:12.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.958+0000 7f6d0eff5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d080d25c0 con 0x7f6d0815dea0 2024-05-09T10:50:12.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.958+0000 7f6d0eff5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d080d28a0 con 0x7f6d080cece0 2024-05-09T10:50:12.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.958+0000 7f6d0d592640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6d080d0340 0x7f6d080d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:12.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.958+0000 7f6d07fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d080cece0 0x7f6d080cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:12.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.958+0000 7f6d0cd91640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6d0815dea0 0x7f6d080ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:12.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.958+0000 7f6d0d592640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6d080d0340 0x7f6d080d07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50736/0 (socket says 172.21.15.98:50736) 2024-05-09T10:50:12.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.958+0000 7f6d0d592640 1 -- 172.21.15.98:0/1960274225 learned_addr learned my addr 172.21.15.98:0/1960274225 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:12.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.958+0000 7f6d07fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d080cece0 0x7f6d080cf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:53716/0 (socket says 172.21.15.98:53716) 2024-05-09T10:50:12.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.958+0000 7f6d0cd91640 1 --2- 172.21.15.98:0/1960274225 >> v2:172.21.15.105:3300/0 conn(0x7f6d0815dea0 0x7f6d080ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.98:42870/0 (socket says 172.21.15.98:42870) 2024-05-09T10:50:12.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.958+0000 7f6d0d592640 1 -- 172.21.15.98:0/1960274225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d080cece0 msgr2=0x7f6d080cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:12.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.958+0000 7f6d0d592640 1 --2- 172.21.15.98:0/1960274225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d080cece0 0x7f6d080cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:12.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.958+0000 7f6d0d592640 1 -- 172.21.15.98:0/1960274225 >> v2:172.21.15.105:3300/0 conn(0x7f6d0815dea0 msgr2=0x7f6d080ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:12.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.958+0000 7f6d0d592640 1 --2- 172.21.15.98:0/1960274225 >> v2:172.21.15.105:3300/0 conn(0x7f6d0815dea0 0x7f6d080ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:12.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.958+0000 7f6d0d592640 1 -- 172.21.15.98:0/1960274225 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d080b6af0 con 0x7f6d080d0340 2024-05-09T10:50:12.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.958+0000 7f6d0cd91640 1 --2- 172.21.15.98:0/1960274225 >> v2:172.21.15.105:3300/0 conn(0x7f6d0815dea0 0x7f6d080ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:50:12.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.958+0000 7f6d07fff640 1 --2- 172.21.15.98:0/1960274225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d080cece0 0x7f6d080cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:50:12.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.959+0000 7f6d0d592640 1 --2- 172.21.15.98:0/1960274225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d080d0340 0x7f6d080d07d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6d0808cad0 tx=0x7f6d0002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35555b4a98643d0a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:12.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.959+0000 7f6d077fe640 1 -- 172.21.15.98:0/1960274225 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d00036070 con 0x7f6d080d0340 2024-05-09T10:50:12.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.959+0000 7f6d077fe640 1 -- 172.21.15.98:0/1960274225 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d00031600 con 0x7f6d080d0340 2024-05-09T10:50:12.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.959+0000 7f6d077fe640 1 -- 172.21.15.98:0/1960274225 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d000319d0 con 0x7f6d080d0340 2024-05-09T10:50:12.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.959+0000 7f6d0eff5640 1 -- 172.21.15.98:0/1960274225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d080d0340 msgr2=0x7f6d080d07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:12.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.959+0000 7f6d0eff5640 1 --2- 172.21.15.98:0/1960274225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d080d0340 0x7f6d080d07d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6d0808cad0 tx=0x7f6d0002a730 comp rx=0 tx=0).stop 2024-05-09T10:50:12.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.959+0000 7f6d0eff5640 1 -- 172.21.15.98:0/1960274225 shutdown_connections 2024-05-09T10:50:12.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.959+0000 7f6d0eff5640 1 --2- 172.21.15.98:0/1960274225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d080d0340 0x7f6d080d07d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:12.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.959+0000 7f6d0eff5640 1 --2- 172.21.15.98:0/1960274225 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d080cece0 0x7f6d080cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:12.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.960+0000 7f6d0eff5640 1 --2- 172.21.15.98:0/1960274225 >> v2:172.21.15.105:3300/0 conn(0x7f6d0815dea0 0x7f6d080ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:12.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.960+0000 7f6d0eff5640 1 -- 172.21.15.98:0/1960274225 >> 172.21.15.98:0/1960274225 conn(0x7f6d0809cee0 msgr2=0x7f6d080d3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:12.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.960+0000 7f6d0eff5640 1 -- 172.21.15.98:0/1960274225 shutdown_connections 2024-05-09T10:50:12.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.960+0000 7f6d0eff5640 1 -- 172.21.15.98:0/1960274225 wait complete. 2024-05-09T10:50:12.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.961+0000 7f6d0eff5640 1 Processor -- start 2024-05-09T10:50:12.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.961+0000 7f6d0eff5640 1 -- start start 2024-05-09T10:50:12.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.962+0000 7f6d0eff5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d080cece0 0x7f6d08157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:12.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.962+0000 7f6d0eff5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d080d0340 0x7f6d08156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:12.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.962+0000 7f6d0eff5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d0815dea0 0x7f6d08157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:12.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.962+0000 7f6d0eff5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d080d21e0 con 0x7f6d0815dea0 2024-05-09T10:50:12.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.962+0000 7f6d0eff5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d080cf660 con 0x7f6d080cece0 2024-05-09T10:50:12.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.962+0000 7f6d0eff5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d080d2360 con 0x7f6d080d0340 2024-05-09T10:50:12.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.962+0000 7f6d0d592640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d0815dea0 0x7f6d08157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:12.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.962+0000 7f6d0d592640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d0815dea0 0x7f6d08157330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50746/0 (socket says 172.21.15.98:50746) 2024-05-09T10:50:12.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.962+0000 7f6d0d592640 1 -- 172.21.15.98:0/3851017035 learned_addr learned my addr 172.21.15.98:0/3851017035 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:12.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.962+0000 7f6d07fff640 1 --2- 172.21.15.98:0/3851017035 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d080d0340 0x7f6d08156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:12.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.962+0000 7f6d0cd91640 1 --2- 172.21.15.98:0/3851017035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d080cece0 0x7f6d08157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:12.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.962+0000 7f6d0d592640 1 -- 172.21.15.98:0/3851017035 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d080d0340 msgr2=0x7f6d08156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:12.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.962+0000 7f6d0d592640 1 --2- 172.21.15.98:0/3851017035 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d080d0340 0x7f6d08156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:12.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.962+0000 7f6d0d592640 1 -- 172.21.15.98:0/3851017035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d080cece0 msgr2=0x7f6d08157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:12.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.962+0000 7f6d0d592640 1 --2- 172.21.15.98:0/3851017035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d080cece0 0x7f6d08157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:12.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.962+0000 7f6d0d592640 1 -- 172.21.15.98:0/3851017035 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d080c8380 con 0x7f6d0815dea0 2024-05-09T10:50:12.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.962+0000 7f6d0d592640 1 --2- 172.21.15.98:0/3851017035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d0815dea0 0x7f6d08157330 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6d00034040 tx=0x7f6d00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:12.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.962+0000 7f6d057fa640 1 -- 172.21.15.98:0/3851017035 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d00041020 con 0x7f6d0815dea0 2024-05-09T10:50:12.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.962+0000 7f6d057fa640 1 -- 172.21.15.98:0/3851017035 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d000077a0 con 0x7f6d0815dea0 2024-05-09T10:50:12.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.962+0000 7f6d0eff5640 1 -- 172.21.15.98:0/3851017035 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d080c8660 con 0x7f6d0815dea0 2024-05-09T10:50:12.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.962+0000 7f6d057fa640 1 -- 172.21.15.98:0/3851017035 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d00036040 con 0x7f6d0815dea0 2024-05-09T10:50:12.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.962+0000 7f6d0eff5640 1 -- 172.21.15.98:0/3851017035 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d080c8c20 con 0x7f6d0815dea0 2024-05-09T10:50:12.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.963+0000 7f6d0eff5640 1 -- 172.21.15.98:0/3851017035 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d080c9100 con 0x7f6d0815dea0 2024-05-09T10:50:12.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.964+0000 7f6d057fa640 1 -- 172.21.15.98:0/3851017035 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6d00044020 con 0x7f6d0815dea0 2024-05-09T10:50:12.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.964+0000 7f6d057fa640 1 --2- 172.21.15.98:0/3851017035 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6cec06b4d0 0x7f6cec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:12.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.965+0000 7f6d0cd91640 1 --2- 172.21.15.98:0/3851017035 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6cec06b4d0 0x7f6cec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:12.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.965+0000 7f6d057fa640 1 -- 172.21.15.98:0/3851017035 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6d000aa9a0 con 0x7f6d0815dea0 2024-05-09T10:50:12.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.965+0000 7f6d0cd91640 1 --2- 172.21.15.98:0/3851017035 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6cec06b4d0 0x7f6cec06d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc011fd0 tx=0x7f6cfc013040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:12.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:12.969+0000 7f6d057fa640 1 -- 172.21.15.98:0/3851017035 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6d080c9100 con 0x7f6d0815dea0 2024-05-09T10:50:13.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.142+0000 7f6d0eff5640 1 -- 172.21.15.98:0/3851017035 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d080720f0 con 0x7f6d0815dea0 2024-05-09T10:50:13.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.143+0000 7f6d057fa640 1 -- 172.21.15.98:0/3851017035 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12884 (secure 0 0 0) 0x7f6d0004b030 con 0x7f6d0815dea0 2024-05-09T10:50:13.136 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:13.137 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":16,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:09.414442+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:13.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.145+0000 7f6d0eff5640 1 -- 172.21.15.98:0/3851017035 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6cec06b4d0 msgr2=0x7f6cec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:13.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.145+0000 7f6d0eff5640 1 --2- 172.21.15.98:0/3851017035 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6cec06b4d0 0x7f6cec06d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc011fd0 tx=0x7f6cfc013040 comp rx=0 tx=0).stop 2024-05-09T10:50:13.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.145+0000 7f6d0eff5640 1 -- 172.21.15.98:0/3851017035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d0815dea0 msgr2=0x7f6d08157330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:13.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.145+0000 7f6d0eff5640 1 --2- 172.21.15.98:0/3851017035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d0815dea0 0x7f6d08157330 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6d00034040 tx=0x7f6d00004a40 comp rx=0 tx=0).stop 2024-05-09T10:50:13.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.145+0000 7f6d0eff5640 1 -- 172.21.15.98:0/3851017035 shutdown_connections 2024-05-09T10:50:13.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.145+0000 7f6d0eff5640 1 --2- 172.21.15.98:0/3851017035 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6cec06b4d0 0x7f6cec06d990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.145+0000 7f6d0eff5640 1 --2- 172.21.15.98:0/3851017035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d0815dea0 0x7f6d08157330 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.145+0000 7f6d0eff5640 1 --2- 172.21.15.98:0/3851017035 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d080d0340 0x7f6d08156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.145+0000 7f6d0eff5640 1 --2- 172.21.15.98:0/3851017035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d080cece0 0x7f6d08157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.145+0000 7f6d0eff5640 1 -- 172.21.15.98:0/3851017035 >> 172.21.15.98:0/3851017035 conn(0x7f6d0809cee0 msgr2=0x7f6d080b7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:13.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.145+0000 7f6d0eff5640 1 -- 172.21.15.98:0/3851017035 shutdown_connections 2024-05-09T10:50:13.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.145+0000 7f6d0eff5640 1 -- 172.21.15.98:0/3851017035 wait complete. 2024-05-09T10:50:13.150 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T10:50:07.396423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T10:50:13.150 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T10:50:13.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.258+0000 7f904aae4640 1 Processor -- start 2024-05-09T10:50:13.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.258+0000 7f904aae4640 1 -- start start 2024-05-09T10:50:13.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.259+0000 7f904aae4640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f904415e140 0x7f90440d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:13.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.259+0000 7f904aae4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f90440d24f0 0x7f90440cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:13.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.259+0000 7f904aae4640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90440cf8e0 con 0x7f904415dd60 2024-05-09T10:50:13.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.259+0000 7f904aae4640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90440cf6b0 con 0x7f904415e140 2024-05-09T10:50:13.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.259+0000 7f904aae4640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90440b2f50 con 0x7f90440d24f0 2024-05-09T10:50:13.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.259+0000 7f9043fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f904415e140 0x7f90440d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:13.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.259+0000 7f9043fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f904415e140 0x7f90440d1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50750/0 (socket says 172.21.15.98:50750) 2024-05-09T10:50:13.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.259+0000 7f9043fff640 1 -- 172.21.15.98:0/2292607310 learned_addr learned my addr 172.21.15.98:0/2292607310 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:13.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.259+0000 7f9049081640 1 --2- 172.21.15.98:0/2292607310 >> v2:172.21.15.105:3300/0 conn(0x7f90440d24f0 0x7f90440cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:13.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.260+0000 7f9043fff640 1 -- 172.21.15.98:0/2292607310 >> v2:172.21.15.105:3300/0 conn(0x7f90440d24f0 msgr2=0x7f90440cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:13.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.260+0000 7f9043fff640 1 --2- 172.21.15.98:0/2292607310 >> v2:172.21.15.105:3300/0 conn(0x7f90440d24f0 0x7f90440cf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.260+0000 7f9043fff640 1 -- 172.21.15.98:0/2292607310 >> v1:172.21.15.105:6789/0 conn(0x7f904415dd60 legacy=0x7f90440d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:13.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.260+0000 7f9043fff640 1 -- 172.21.15.98:0/2292607310 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90440b31a0 con 0x7f904415e140 2024-05-09T10:50:13.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.260+0000 7f9043fff640 1 --2- 172.21.15.98:0/2292607310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f904415e140 0x7f90440d1790 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f90440b30d0 tx=0x7f903802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b62da8e606c7336 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:13.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.261+0000 7f90437fe640 1 -- 172.21.15.98:0/2292607310 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038037070 con 0x7f904415e140 2024-05-09T10:50:13.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.261+0000 7f90437fe640 1 -- 172.21.15.98:0/2292607310 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9038032440 con 0x7f904415e140 2024-05-09T10:50:13.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.261+0000 7f90437fe640 1 -- 172.21.15.98:0/2292607310 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038032810 con 0x7f904415e140 2024-05-09T10:50:13.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.261+0000 7f904aae4640 1 -- 172.21.15.98:0/2292607310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f904415e140 msgr2=0x7f90440d1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:13.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.261+0000 7f904aae4640 1 --2- 172.21.15.98:0/2292607310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f904415e140 0x7f90440d1790 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f90440b30d0 tx=0x7f903802a270 comp rx=0 tx=0).stop 2024-05-09T10:50:13.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.261+0000 7f904aae4640 1 -- 172.21.15.98:0/2292607310 shutdown_connections 2024-05-09T10:50:13.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.261+0000 7f904aae4640 1 --2- 172.21.15.98:0/2292607310 >> v2:172.21.15.105:3300/0 conn(0x7f90440d24f0 0x7f90440cf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.261+0000 7f904aae4640 1 --2- 172.21.15.98:0/2292607310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f904415e140 0x7f90440d1790 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.261+0000 7f904aae4640 1 -- 172.21.15.98:0/2292607310 >> 172.21.15.98:0/2292607310 conn(0x7f90440d3b50 msgr2=0x7f90440d3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:13.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.261+0000 7f904aae4640 1 -- 172.21.15.98:0/2292607310 shutdown_connections 2024-05-09T10:50:13.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.261+0000 7f904aae4640 1 -- 172.21.15.98:0/2292607310 wait complete. 2024-05-09T10:50:13.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.262+0000 7f904aae4640 1 Processor -- start 2024-05-09T10:50:13.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.262+0000 7f904aae4640 1 -- start start 2024-05-09T10:50:13.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.263+0000 7f904aae4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90440d24f0 0x7f90441f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:13.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.263+0000 7f904aae4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f904415dd60 0x7f90440b88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:13.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.263+0000 7f904aae4640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f90440c9260 0x7f904415b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:13.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.263+0000 7f904aae4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90440b3c80 con 0x7f904415dd60 2024-05-09T10:50:13.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.263+0000 7f904aae4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90440b3760 con 0x7f90440d24f0 2024-05-09T10:50:13.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.263+0000 7f904aae4640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90440b3e00 con 0x7f90440c9260 2024-05-09T10:50:13.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.263+0000 7f9043fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f904415dd60 0x7f90440b88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:13.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.263+0000 7f9043fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f904415dd60 0x7f90440b88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50762/0 (socket says 172.21.15.98:50762) 2024-05-09T10:50:13.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.263+0000 7f9043fff640 1 -- 172.21.15.98:0/1204231344 learned_addr learned my addr 172.21.15.98:0/1204231344 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:13.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.264+0000 7f9048880640 1 --2- 172.21.15.98:0/1204231344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90440d24f0 0x7f90441f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:13.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.264+0000 7f9049081640 1 --2- 172.21.15.98:0/1204231344 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f90440c9260 0x7f904415b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:13.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.264+0000 7f9043fff640 1 -- 172.21.15.98:0/1204231344 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f90440c9260 msgr2=0x7f904415b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:13.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.264+0000 7f9043fff640 1 --2- 172.21.15.98:0/1204231344 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f90440c9260 0x7f904415b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.264+0000 7f9043fff640 1 -- 172.21.15.98:0/1204231344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90440d24f0 msgr2=0x7f90441f4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:13.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.264+0000 7f9043fff640 1 --2- 172.21.15.98:0/1204231344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90440d24f0 0x7f90441f4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.264+0000 7f9043fff640 1 -- 172.21.15.98:0/1204231344 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90440c87f0 con 0x7f904415dd60 2024-05-09T10:50:13.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.264+0000 7f9049081640 1 --2- 172.21.15.98:0/1204231344 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f90440c9260 0x7f904415b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:13.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.264+0000 7f9043fff640 1 --2- 172.21.15.98:0/1204231344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f904415dd60 0x7f90440b88b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f903802a2b0 tx=0x7f90380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:13.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.264+0000 7f90417fa640 1 -- 172.21.15.98:0/1204231344 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038037040 con 0x7f904415dd60 2024-05-09T10:50:13.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.264+0000 7f90417fa640 1 -- 172.21.15.98:0/1204231344 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9038042070 con 0x7f904415dd60 2024-05-09T10:50:13.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.264+0000 7f904aae4640 1 -- 172.21.15.98:0/1204231344 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90441f52b0 con 0x7f904415dd60 2024-05-09T10:50:13.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.264+0000 7f90417fa640 1 -- 172.21.15.98:0/1204231344 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038030880 con 0x7f904415dd60 2024-05-09T10:50:13.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.264+0000 7f904aae4640 1 -- 172.21.15.98:0/1204231344 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90440c8d40 con 0x7f904415dd60 2024-05-09T10:50:13.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.265+0000 7f904aae4640 1 -- 172.21.15.98:0/1204231344 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90440a1b60 con 0x7f904415dd60 2024-05-09T10:50:13.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.266+0000 7f90417fa640 1 -- 172.21.15.98:0/1204231344 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9038030a20 con 0x7f904415dd60 2024-05-09T10:50:13.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.266+0000 7f90417fa640 1 --2- 172.21.15.98:0/1204231344 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f903006b4d0 0x7f903006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:13.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.266+0000 7f90417fa640 1 -- 172.21.15.98:0/1204231344 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f90380aa710 con 0x7f904415dd60 2024-05-09T10:50:13.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.266+0000 7f9048880640 1 --2- 172.21.15.98:0/1204231344 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f903006b4d0 0x7f903006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:13.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.266+0000 7f9048880640 1 --2- 172.21.15.98:0/1204231344 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f903006b4d0 0x7f903006d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9034001640 tx=0x7f9034009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:13.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.270+0000 7f90417fa640 1 -- 172.21.15.98:0/1204231344 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90441f52b0 con 0x7f904415dd60 2024-05-09T10:50:13.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.442+0000 7f904aae4640 1 -- 172.21.15.98:0/1204231344 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9044161560 con 0x7f904415dd60 2024-05-09T10:50:13.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.442+0000 7f90417fa640 1 -- 172.21.15.98:0/1204231344 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f90380798e0 con 0x7f904415dd60 2024-05-09T10:50:13.436 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-05-09T10:50:13.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.444+0000 7f904aae4640 1 -- 172.21.15.98:0/1204231344 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f903006b4d0 msgr2=0x7f903006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:13.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.444+0000 7f904aae4640 1 --2- 172.21.15.98:0/1204231344 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f903006b4d0 0x7f903006d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9034001640 tx=0x7f9034009040 comp rx=0 tx=0).stop 2024-05-09T10:50:13.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.444+0000 7f904aae4640 1 -- 172.21.15.98:0/1204231344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f904415dd60 msgr2=0x7f90440b88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:13.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.444+0000 7f904aae4640 1 --2- 172.21.15.98:0/1204231344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f904415dd60 0x7f90440b88b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f903802a2b0 tx=0x7f90380046c0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.444+0000 7f904aae4640 1 -- 172.21.15.98:0/1204231344 shutdown_connections 2024-05-09T10:50:13.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.444+0000 7f904aae4640 1 --2- 172.21.15.98:0/1204231344 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f903006b4d0 0x7f903006d990 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.444+0000 7f904aae4640 1 --2- 172.21.15.98:0/1204231344 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f90440c9260 0x7f904415b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.444+0000 7f904aae4640 1 --2- 172.21.15.98:0/1204231344 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f904415dd60 0x7f90440b88b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.444+0000 7f904aae4640 1 --2- 172.21.15.98:0/1204231344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90440d24f0 0x7f90441f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.444+0000 7f904aae4640 1 -- 172.21.15.98:0/1204231344 >> 172.21.15.98:0/1204231344 conn(0x7f90440d3b50 msgr2=0x7f90440d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:13.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.444+0000 7f904aae4640 1 -- 172.21.15.98:0/1204231344 shutdown_connections 2024-05-09T10:50:13.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.444+0000 7f904aae4640 1 -- 172.21.15.98:0/1204231344 wait complete. 2024-05-09T10:50:13.448 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T10:50:13.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.555+0000 7f5e2af5c640 1 Processor -- start 2024-05-09T10:50:13.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.555+0000 7f5e2af5c640 1 -- start start 2024-05-09T10:50:13.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.555+0000 7f5e2af5c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5e2415de40 0x7f5e240d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:13.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.555+0000 7f5e2af5c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e240cf250 0x7f5e240cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:13.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.555+0000 7f5e2af5c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2415e220 con 0x7f5e240d0470 2024-05-09T10:50:13.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.555+0000 7f5e2af5c640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e240d1790 con 0x7f5e2415de40 2024-05-09T10:50:13.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.555+0000 7f5e2af5c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e240d19e0 con 0x7f5e240cf250 2024-05-09T10:50:13.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.555+0000 7f5e28cf8640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5e2415de40 0x7f5e240d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:13.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.556+0000 7f5e28cf8640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5e2415de40 0x7f5e240d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34214/0 (socket says 172.21.15.98:34214) 2024-05-09T10:50:13.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.556+0000 7f5e28cf8640 1 -- 172.21.15.98:0/541530028 learned_addr learned my addr 172.21.15.98:0/541530028 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:13.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.556+0000 7f5e23fff640 1 --2- 172.21.15.98:0/541530028 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e240cf250 0x7f5e240cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:13.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.556+0000 7f5e28cf8640 1 -- 172.21.15.98:0/541530028 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e240cf250 msgr2=0x7f5e240cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:13.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.556+0000 7f5e28cf8640 1 --2- 172.21.15.98:0/541530028 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e240cf250 0x7f5e240cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.556+0000 7f5e28cf8640 1 -- 172.21.15.98:0/541530028 >> v1:172.21.15.105:6789/0 conn(0x7f5e240d0470 legacy=0x7f5e240b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:13.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.556+0000 7f5e28cf8640 1 -- 172.21.15.98:0/541530028 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e240d1cc0 con 0x7f5e2415de40 2024-05-09T10:50:13.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.556+0000 7f5e23fff640 1 --2- 172.21.15.98:0/541530028 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e240cf250 0x7f5e240cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:13.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.556+0000 7f5e28cf8640 1 --2- 172.21.15.98:0/541530028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e2415de40 0x7f5e240d27a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009870 tx=0x7f5e1802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8eb7a5073ab57a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:13.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.557+0000 7f5e237fe640 1 -- 172.21.15.98:0/541530028 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e18037070 con 0x7f5e2415de40 2024-05-09T10:50:13.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.557+0000 7f5e237fe640 1 -- 172.21.15.98:0/541530028 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e18032440 con 0x7f5e2415de40 2024-05-09T10:50:13.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.557+0000 7f5e237fe640 1 -- 172.21.15.98:0/541530028 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e18032810 con 0x7f5e2415de40 2024-05-09T10:50:13.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.557+0000 7f5e2af5c640 1 -- 172.21.15.98:0/541530028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e2415de40 msgr2=0x7f5e240d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:13.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.557+0000 7f5e2af5c640 1 --2- 172.21.15.98:0/541530028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e2415de40 0x7f5e240d27a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009870 tx=0x7f5e1802a270 comp rx=0 tx=0).stop 2024-05-09T10:50:13.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.557+0000 7f5e2af5c640 1 -- 172.21.15.98:0/541530028 shutdown_connections 2024-05-09T10:50:13.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.557+0000 7f5e2af5c640 1 --2- 172.21.15.98:0/541530028 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e240cf250 0x7f5e240cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.557+0000 7f5e2af5c640 1 --2- 172.21.15.98:0/541530028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e2415de40 0x7f5e240d27a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.557+0000 7f5e2af5c640 1 -- 172.21.15.98:0/541530028 >> 172.21.15.98:0/541530028 conn(0x7f5e2409ce90 msgr2=0x7f5e240d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:13.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.557+0000 7f5e2af5c640 1 -- 172.21.15.98:0/541530028 shutdown_connections 2024-05-09T10:50:13.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.557+0000 7f5e2af5c640 1 -- 172.21.15.98:0/541530028 wait complete. 2024-05-09T10:50:13.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.558+0000 7f5e2af5c640 1 Processor -- start 2024-05-09T10:50:13.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.558+0000 7f5e2af5c640 1 -- start start 2024-05-09T10:50:13.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.559+0000 7f5e2af5c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e240cf250 0x7f5e2415c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:13.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.559+0000 7f5e2af5c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e240d0470 0x7f5e241e9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:13.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.559+0000 7f5e2af5c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e2415de40 0x7f5e241e9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:13.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.559+0000 7f5e2af5c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e240d4680 con 0x7f5e240d0470 2024-05-09T10:50:13.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.559+0000 7f5e2af5c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e240d2280 con 0x7f5e2415de40 2024-05-09T10:50:13.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.559+0000 7f5e2af5c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e240d4800 con 0x7f5e240cf250 2024-05-09T10:50:13.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.559+0000 7f5e23fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e240d0470 0x7f5e241e9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:13.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.559+0000 7f5e23fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e240d0470 0x7f5e241e9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34218/0 (socket says 172.21.15.98:34218) 2024-05-09T10:50:13.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.559+0000 7f5e23fff640 1 -- 172.21.15.98:0/1662183957 learned_addr learned my addr 172.21.15.98:0/1662183957 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:13.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.560+0000 7f5e23fff640 1 -- 172.21.15.98:0/1662183957 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e240cf250 msgr2=0x7f5e2415c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:13.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.560+0000 7f5e23fff640 1 --2- 172.21.15.98:0/1662183957 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e240cf250 0x7f5e2415c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.560+0000 7f5e23fff640 1 -- 172.21.15.98:0/1662183957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e2415de40 msgr2=0x7f5e241e9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:13.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.560+0000 7f5e23fff640 1 --2- 172.21.15.98:0/1662183957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e2415de40 0x7f5e241e9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.560+0000 7f5e23fff640 1 -- 172.21.15.98:0/1662183957 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e240cbc80 con 0x7f5e240d0470 2024-05-09T10:50:13.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.560+0000 7f5e23fff640 1 --2- 172.21.15.98:0/1662183957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e240d0470 0x7f5e241e9900 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000e7e0 tx=0x7f5e1000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:13.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.560+0000 7f5e217fa640 1 -- 172.21.15.98:0/1662183957 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1000f040 con 0x7f5e240d0470 2024-05-09T10:50:13.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.560+0000 7f5e217fa640 1 -- 172.21.15.98:0/1662183957 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e10004030 con 0x7f5e240d0470 2024-05-09T10:50:13.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.560+0000 7f5e2af5c640 1 -- 172.21.15.98:0/1662183957 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e24161870 con 0x7f5e240d0470 2024-05-09T10:50:13.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.560+0000 7f5e217fa640 1 -- 172.21.15.98:0/1662183957 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10004400 con 0x7f5e240d0470 2024-05-09T10:50:13.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.560+0000 7f5e2af5c640 1 -- 172.21.15.98:0/1662183957 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e24162110 con 0x7f5e240d0470 2024-05-09T10:50:13.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.561+0000 7f5e2af5c640 1 -- 172.21.15.98:0/1662183957 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e24161c60 con 0x7f5e240d0470 2024-05-09T10:50:13.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.561+0000 7f5e217fa640 1 -- 172.21.15.98:0/1662183957 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e1000c6d0 con 0x7f5e240d0470 2024-05-09T10:50:13.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.562+0000 7f5e217fa640 1 --2- 172.21.15.98:0/1662183957 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5e0806b4d0 0x7f5e0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:13.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.562+0000 7f5e28cf8640 1 --2- 172.21.15.98:0/1662183957 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5e0806b4d0 0x7f5e0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:13.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.562+0000 7f5e217fa640 1 -- 172.21.15.98:0/1662183957 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5e10089be0 con 0x7f5e240d0470 2024-05-09T10:50:13.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.562+0000 7f5e28cf8640 1 --2- 172.21.15.98:0/1662183957 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5e0806b4d0 0x7f5e0806d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5e18034000 tx=0x7f5e1802c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:13.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.566+0000 7f5e217fa640 1 -- 172.21.15.98:0/1662183957 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e10017020 con 0x7f5e240d0470 2024-05-09T10:50:13.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.770+0000 7f5e2af5c640 1 -- 172.21.15.98:0/1662183957 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e24069cb0 con 0x7f5e240d0470 2024-05-09T10:50:13.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.771+0000 7f5e217fa640 1 -- 172.21.15.98:0/1662183957 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1321 (secure 0 0 0) 0x7f5e10058e40 con 0x7f5e240d0470 2024-05-09T10:50:13.764 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:13.764 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/937700297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":937700297},{"type":"v1","addr":"172.21.15.98:6835","nonce":937700297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-05-09T10:50:13.765 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 2 2024-05-09T10:50:13.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.772+0000 7f5e2af5c640 1 -- 172.21.15.98:0/1662183957 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5e0806b4d0 msgr2=0x7f5e0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:13.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.772+0000 7f5e2af5c640 1 --2- 172.21.15.98:0/1662183957 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5e0806b4d0 0x7f5e0806d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5e18034000 tx=0x7f5e1802c000 comp rx=0 tx=0).stop 2024-05-09T10:50:13.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.772+0000 7f5e2af5c640 1 -- 172.21.15.98:0/1662183957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e240d0470 msgr2=0x7f5e241e9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:13.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.772+0000 7f5e2af5c640 1 --2- 172.21.15.98:0/1662183957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e240d0470 0x7f5e241e9900 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000e7e0 tx=0x7f5e1000eca0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.773+0000 7f5e2af5c640 1 -- 172.21.15.98:0/1662183957 shutdown_connections 2024-05-09T10:50:13.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.773+0000 7f5e2af5c640 1 --2- 172.21.15.98:0/1662183957 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5e0806b4d0 0x7f5e0806d990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.773+0000 7f5e2af5c640 1 --2- 172.21.15.98:0/1662183957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e2415de40 0x7f5e241e9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.773+0000 7f5e2af5c640 1 --2- 172.21.15.98:0/1662183957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e240d0470 0x7f5e241e9900 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.773+0000 7f5e2af5c640 1 --2- 172.21.15.98:0/1662183957 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e240cf250 0x7f5e2415c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:13.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.773+0000 7f5e2af5c640 1 -- 172.21.15.98:0/1662183957 >> 172.21.15.98:0/1662183957 conn(0x7f5e2409ce90 msgr2=0x7f5e240ce820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:13.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.773+0000 7f5e2af5c640 1 -- 172.21.15.98:0/1662183957 shutdown_connections 2024-05-09T10:50:13.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:13.773+0000 7f5e2af5c640 1 -- 172.21.15.98:0/1662183957 wait complete. 2024-05-09T10:50:15.778 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T10:50:15.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.896+0000 7f571e99e640 1 Processor -- start 2024-05-09T10:50:15.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.896+0000 7f571e99e640 1 -- start start 2024-05-09T10:50:15.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.897+0000 7f571e99e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f571815db90 0x7f571806d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:15.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.897+0000 7f571e99e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57180cdb30 con 0x7f571815df70 2024-05-09T10:50:15.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.897+0000 7f571e99e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5718071f40 con 0x7f571806abe0 2024-05-09T10:50:15.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.897+0000 7f571e99e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5718072220 con 0x7f571815db90 2024-05-09T10:50:15.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.897+0000 7f57177fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f571815df70 0x7f57180713d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42266/0 (socket says 172.21.15.98:42266) 2024-05-09T10:50:15.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.897+0000 7f57177fe640 1 -- 172.21.15.98:0/568287364 learned_addr learned my addr 172.21.15.98:0/568287364 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:15.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.897+0000 7f5716ffd640 1 -- 172.21.15.98:0/568287364 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3240100232 0 0) 0x7f570c001110 con 0x7f571815df70 2024-05-09T10:50:15.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.897+0000 7f5717fff640 1 --2- 172.21.15.98:0/568287364 >> v2:172.21.15.105:3300/0 conn(0x7f571815db90 0x7f571806d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:15.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.898+0000 7f5716ffd640 1 -- 172.21.15.98:0/568287364 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5700003520 con 0x7f571815df70 2024-05-09T10:50:15.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.898+0000 7f5716ffd640 1 -- 172.21.15.98:0/568287364 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1471600693 0 0) 0x7f5700003520 con 0x7f571815df70 2024-05-09T10:50:15.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.898+0000 7f5716ffd640 1 -- 172.21.15.98:0/568287364 >> v2:172.21.15.105:3300/0 conn(0x7f571815db90 msgr2=0x7f571806d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:15.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.898+0000 7f5716ffd640 1 --2- 172.21.15.98:0/568287364 >> v2:172.21.15.105:3300/0 conn(0x7f571815db90 0x7f571806d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:15.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.898+0000 7f5716ffd640 1 -- 172.21.15.98:0/568287364 >> v1:172.21.15.105:6789/0 conn(0x7f571806abe0 legacy=0x7f57180cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:15.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.898+0000 7f5716ffd640 1 -- 172.21.15.98:0/568287364 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5718072500 con 0x7f571815df70 2024-05-09T10:50:15.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.898+0000 7f5716ffd640 1 -- 172.21.15.98:0/568287364 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f570c001840 con 0x7f571815df70 2024-05-09T10:50:15.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.898+0000 7f5716ffd640 1 -- 172.21.15.98:0/568287364 >> v1:172.21.15.98:6789/0 conn(0x7f571815df70 legacy=0x7f57180713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:15.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.898+0000 7f5716ffd640 1 --2- 172.21.15.98:0/568287364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57000039b0 0x7f5700003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:15.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.898+0000 7f5716ffd640 1 --2- 172.21.15.98:0/568287364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5700004800 0x7f5700004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:15.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.898+0000 7f5716ffd640 1 --2- 172.21.15.98:0/568287364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57000051c0 0x7f570000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:15.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.898+0000 7f5716ffd640 1 -- 172.21.15.98:0/568287364 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5718072220 con 0x7f57000039b0 2024-05-09T10:50:15.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.898+0000 7f5716ffd640 1 -- 172.21.15.98:0/568287364 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570000c160 con 0x7f57000051c0 2024-05-09T10:50:15.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.898+0000 7f5716ffd640 1 -- 172.21.15.98:0/568287364 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570000c360 con 0x7f5700004800 2024-05-09T10:50:15.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.898+0000 7f5717fff640 1 --2- 172.21.15.98:0/568287364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57000039b0 0x7f5700003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:15.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.898+0000 7f57177fe640 1 --2- 172.21.15.98:0/568287364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5700004800 0x7f5700004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:15.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.898+0000 7f571cf3b640 1 --2- 172.21.15.98:0/568287364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57000051c0 0x7f570000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:15.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.898+0000 7f5717fff640 1 -- 172.21.15.98:0/568287364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5700004800 msgr2=0x7f5700004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:15.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.898+0000 7f5717fff640 1 --2- 172.21.15.98:0/568287364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5700004800 0x7f5700004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:15.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.898+0000 7f5717fff640 1 -- 172.21.15.98:0/568287364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57000051c0 msgr2=0x7f570000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:15.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.898+0000 7f5717fff640 1 --2- 172.21.15.98:0/568287364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57000051c0 0x7f570000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:15.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.898+0000 7f5717fff640 1 -- 172.21.15.98:0/568287364 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5700003520 con 0x7f57000039b0 2024-05-09T10:50:15.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.898+0000 7f571cf3b640 1 -- 172.21.15.98:0/568287364 reap_dead start 2024-05-09T10:50:15.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.899+0000 7f5717fff640 1 --2- 172.21.15.98:0/568287364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57000039b0 0x7f5700003dd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5708009ac0 tx=0x7f5700005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f55b5946ac4115 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:15.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.899+0000 7f5716ffd640 1 -- 172.21.15.98:0/568287364 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5708036070 con 0x7f57000039b0 2024-05-09T10:50:15.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.899+0000 7f5716ffd640 1 -- 172.21.15.98:0/568287364 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5708031400 con 0x7f57000039b0 2024-05-09T10:50:15.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.899+0000 7f5716ffd640 1 -- 172.21.15.98:0/568287364 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5708009690 con 0x7f57000039b0 2024-05-09T10:50:15.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.900+0000 7f571e99e640 1 -- 172.21.15.98:0/568287364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57000039b0 msgr2=0x7f5700003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:15.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.900+0000 7f571e99e640 1 --2- 172.21.15.98:0/568287364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57000039b0 0x7f5700003dd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5708009ac0 tx=0x7f5700005830 comp rx=0 tx=0).stop 2024-05-09T10:50:15.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.900+0000 7f571e99e640 1 -- 172.21.15.98:0/568287364 shutdown_connections 2024-05-09T10:50:15.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.900+0000 7f571e99e640 1 --2- 172.21.15.98:0/568287364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57000039b0 0x7f5700003dd0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:15.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.900+0000 7f571e99e640 1 -- 172.21.15.98:0/568287364 >> 172.21.15.98:0/568287364 conn(0x7f5718066f20 msgr2=0x7f571806bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:15.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.900+0000 7f571e99e640 1 -- 172.21.15.98:0/568287364 shutdown_connections 2024-05-09T10:50:15.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.900+0000 7f571e99e640 1 -- 172.21.15.98:0/568287364 wait complete. 2024-05-09T10:50:15.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.901+0000 7f571e99e640 1 Processor -- start 2024-05-09T10:50:15.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.901+0000 7f571e99e640 1 -- start start 2024-05-09T10:50:15.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.902+0000 7f571e99e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57000039b0 0x7f57181e86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:15.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.902+0000 7f571e99e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f571815ee00 0x7f571806d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:15.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.902+0000 7f571e99e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5718147ec0 0x7f5718148340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:15.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.902+0000 7f571e99e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571806b070 con 0x7f5718147ec0 2024-05-09T10:50:15.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.902+0000 7f571e99e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57180cefd0 con 0x7f57000039b0 2024-05-09T10:50:15.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.902+0000 7f571e99e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571806b1f0 con 0x7f571815ee00 2024-05-09T10:50:15.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.902+0000 7f571cf3b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5718147ec0 0x7f5718148340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:15.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.902+0000 7f571cf3b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5718147ec0 0x7f5718148340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34238/0 (socket says 172.21.15.98:34238) 2024-05-09T10:50:15.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.902+0000 7f571cf3b640 1 -- 172.21.15.98:0/469295849 learned_addr learned my addr 172.21.15.98:0/469295849 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:15.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.902+0000 7f5717fff640 1 --2- 172.21.15.98:0/469295849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57000039b0 0x7f57181e86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:15.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.902+0000 7f57177fe640 1 --2- 172.21.15.98:0/469295849 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f571815ee00 0x7f571806d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:15.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.902+0000 7f571cf3b640 1 -- 172.21.15.98:0/469295849 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f571815ee00 msgr2=0x7f571806d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:15.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.902+0000 7f571cf3b640 1 --2- 172.21.15.98:0/469295849 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f571815ee00 0x7f571806d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:15.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.902+0000 7f571cf3b640 1 -- 172.21.15.98:0/469295849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57000039b0 msgr2=0x7f57181e86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:15.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.902+0000 7f571cf3b640 1 --2- 172.21.15.98:0/469295849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57000039b0 0x7f57181e86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:15.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.902+0000 7f571cf3b640 1 -- 172.21.15.98:0/469295849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571815f650 con 0x7f5718147ec0 2024-05-09T10:50:15.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.902+0000 7f571cf3b640 1 --2- 172.21.15.98:0/469295849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5718147ec0 0x7f5718148340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5718162a90 tx=0x7f571000e8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:15.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.903+0000 7f5714ff9640 1 -- 172.21.15.98:0/469295849 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5710019070 con 0x7f5718147ec0 2024-05-09T10:50:15.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.903+0000 7f5714ff9640 1 -- 172.21.15.98:0/469295849 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5710006d60 con 0x7f5718147ec0 2024-05-09T10:50:15.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.903+0000 7f571e99e640 1 -- 172.21.15.98:0/469295849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571815f930 con 0x7f5718147ec0 2024-05-09T10:50:15.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.903+0000 7f5714ff9640 1 -- 172.21.15.98:0/469295849 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5710003b90 con 0x7f5718147ec0 2024-05-09T10:50:15.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.903+0000 7f571e99e640 1 -- 172.21.15.98:0/469295849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571815fef0 con 0x7f5718147ec0 2024-05-09T10:50:15.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.904+0000 7f571e99e640 1 -- 172.21.15.98:0/469295849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571806b870 con 0x7f5718147ec0 2024-05-09T10:50:15.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.904+0000 7f5714ff9640 1 -- 172.21.15.98:0/469295849 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5710005070 con 0x7f5718147ec0 2024-05-09T10:50:15.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.905+0000 7f5714ff9640 1 --2- 172.21.15.98:0/469295849 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f570406b500 0x7f570406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:15.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.905+0000 7f5717fff640 1 --2- 172.21.15.98:0/469295849 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f570406b500 0x7f570406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:15.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.905+0000 7f5714ff9640 1 -- 172.21.15.98:0/469295849 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5710089280 con 0x7f5718147ec0 2024-05-09T10:50:15.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.905+0000 7f5717fff640 1 --2- 172.21.15.98:0/469295849 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f570406b500 0x7f570406d9c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5708033000 tx=0x7f5708008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:15.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:15.909+0000 7f5714ff9640 1 -- 172.21.15.98:0/469295849 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f571806b870 con 0x7f5718147ec0 2024-05-09T10:50:16.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.117+0000 7f571e99e640 1 -- 172.21.15.98:0/469295849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5718069cb0 con 0x7f5718147ec0 2024-05-09T10:50:16.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.117+0000 7f5714ff9640 1 -- 172.21.15.98:0/469295849 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3694 (secure 0 0 0) 0x7f5710057e00 con 0x7f5718147ec0 2024-05-09T10:50:16.111 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:16.111 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2903091926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2903091926},{"type":"v1","addr":"172.21.15.98:6837","nonce":2903091926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/968936316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":968936316},{"type":"v1","addr":"172.21.15.105:6834","nonce":968936316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/937700297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":937700297},{"type":"v1","addr":"172.21.15.98:6835","nonce":937700297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1780062557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1780062557},{"type":"v1","addr":"172.21.15.105:6835","nonce":1780062557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-05-09T10:50:16.111 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 3 2024-05-09T10:50:16.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.119+0000 7f571e99e640 1 -- 172.21.15.98:0/469295849 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f570406b500 msgr2=0x7f570406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:16.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.119+0000 7f571e99e640 1 --2- 172.21.15.98:0/469295849 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f570406b500 0x7f570406d9c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5708033000 tx=0x7f5708008030 comp rx=0 tx=0).stop 2024-05-09T10:50:16.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.119+0000 7f571e99e640 1 -- 172.21.15.98:0/469295849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5718147ec0 msgr2=0x7f5718148340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:16.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.119+0000 7f571e99e640 1 --2- 172.21.15.98:0/469295849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5718147ec0 0x7f5718148340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5718162a90 tx=0x7f571000e8c0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.120+0000 7f571e99e640 1 -- 172.21.15.98:0/469295849 shutdown_connections 2024-05-09T10:50:16.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.120+0000 7f571e99e640 1 --2- 172.21.15.98:0/469295849 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f570406b500 0x7f570406d9c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.120+0000 7f571e99e640 1 --2- 172.21.15.98:0/469295849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5718147ec0 0x7f5718148340 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.120+0000 7f571e99e640 1 --2- 172.21.15.98:0/469295849 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f571815ee00 0x7f571806d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.120+0000 7f571e99e640 1 --2- 172.21.15.98:0/469295849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57000039b0 0x7f57181e86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.120+0000 7f571e99e640 1 -- 172.21.15.98:0/469295849 >> 172.21.15.98:0/469295849 conn(0x7f5718066f20 msgr2=0x7f5718071600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:16.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.120+0000 7f571e99e640 1 -- 172.21.15.98:0/469295849 shutdown_connections 2024-05-09T10:50:16.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.120+0000 7f571e99e640 1 -- 172.21.15.98:0/469295849 wait complete. 2024-05-09T10:50:16.124 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:16.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.230+0000 7f6fed309640 1 Processor -- start 2024-05-09T10:50:16.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.230+0000 7f6fed309640 1 -- start start 2024-05-09T10:50:16.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.231+0000 7f6fed309640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6fe80d0b80 0x7f6fe80b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:16.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.231+0000 7f6fed309640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe80d2c30 con 0x7f6fe80d27a0 2024-05-09T10:50:16.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.231+0000 7f6fed309640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe815e220 con 0x7f6fe815de40 2024-05-09T10:50:16.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.231+0000 7f6fed309640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe80d1790 con 0x7f6fe80d0b80 2024-05-09T10:50:16.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.231+0000 7f6fe77fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6fe80d0b80 0x7f6fe80b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:16.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.231+0000 7f6fde7fc640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f6fe80d27a0 0x7f6fe80d0470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42280/0 (socket says 172.21.15.98:42280) 2024-05-09T10:50:16.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.231+0000 7f6fde7fc640 1 -- 172.21.15.98:0/3437575356 learned_addr learned my addr 172.21.15.98:0/3437575356 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:16.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.231+0000 7f6fe77fe640 1 -- 172.21.15.98:0/3437575356 >> v1:172.21.15.105:6789/0 conn(0x7f6fe815de40 legacy=0x7f6fe80cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:16.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.231+0000 7f6fe77fe640 1 -- 172.21.15.98:0/3437575356 >> v1:172.21.15.98:6789/0 conn(0x7f6fe80d27a0 legacy=0x7f6fe80d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:16.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.231+0000 7f6fe77fe640 1 -- 172.21.15.98:0/3437575356 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe80d19e0 con 0x7f6fe80d0b80 2024-05-09T10:50:16.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.232+0000 7f6fe77fe640 1 --2- 172.21.15.98:0/3437575356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fe80d0b80 0x7f6fe80b6a10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009d90 tx=0x7f6fe002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a96c0c15437195c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:16.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.232+0000 7f6fe67fc640 1 -- 172.21.15.98:0/3437575356 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0007740 con 0x7f6fe80d0b80 2024-05-09T10:50:16.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.232+0000 7f6fe67fc640 1 -- 172.21.15.98:0/3437575356 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe00078e0 con 0x7f6fe80d0b80 2024-05-09T10:50:16.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.232+0000 7f6fe67fc640 1 -- 172.21.15.98:0/3437575356 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0007cb0 con 0x7f6fe80d0b80 2024-05-09T10:50:16.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.233+0000 7f6fed309640 1 -- 172.21.15.98:0/3437575356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fe80d0b80 msgr2=0x7f6fe80b6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:16.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.233+0000 7f6fed309640 1 --2- 172.21.15.98:0/3437575356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fe80d0b80 0x7f6fe80b6a10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009d90 tx=0x7f6fe002a6f0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.233+0000 7f6fed309640 1 -- 172.21.15.98:0/3437575356 shutdown_connections 2024-05-09T10:50:16.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.233+0000 7f6fed309640 1 --2- 172.21.15.98:0/3437575356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fe80d0b80 0x7f6fe80b6a10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.233+0000 7f6fed309640 1 -- 172.21.15.98:0/3437575356 >> 172.21.15.98:0/3437575356 conn(0x7f6fe809ce90 msgr2=0x7f6fe80d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:16.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.233+0000 7f6fed309640 1 -- 172.21.15.98:0/3437575356 shutdown_connections 2024-05-09T10:50:16.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.233+0000 7f6fed309640 1 -- 172.21.15.98:0/3437575356 wait complete. 2024-05-09T10:50:16.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.234+0000 7f6fed309640 1 Processor -- start 2024-05-09T10:50:16.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.234+0000 7f6fed309640 1 -- start start 2024-05-09T10:50:16.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.235+0000 7f6fed309640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fe80d0b80 0x7f6fe8155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:16.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.235+0000 7f6fed309640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fe80d27a0 0x7f6fe814fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:16.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.235+0000 7f6fed309640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fe815de40 0x7f6fe80c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:16.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.235+0000 7f6fed309640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe80d2280 con 0x7f6fe815de40 2024-05-09T10:50:16.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.235+0000 7f6fed309640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe80d1fa0 con 0x7f6fe80d0b80 2024-05-09T10:50:16.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.235+0000 7f6fed309640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe80cf960 con 0x7f6fe80d27a0 2024-05-09T10:50:16.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.235+0000 7f6fe77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fe815de40 0x7f6fe80c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:16.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.235+0000 7f6fe77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fe815de40 0x7f6fe80c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34264/0 (socket says 172.21.15.98:34264) 2024-05-09T10:50:16.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.235+0000 7f6fe77fe640 1 -- 172.21.15.98:0/3910064294 learned_addr learned my addr 172.21.15.98:0/3910064294 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:16.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.235+0000 7f6fe6ffd640 1 --2- 172.21.15.98:0/3910064294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fe80d0b80 0x7f6fe8155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:16.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.235+0000 7f6fde7fc640 1 --2- 172.21.15.98:0/3910064294 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fe80d27a0 0x7f6fe814fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:16.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.235+0000 7f6fe77fe640 1 -- 172.21.15.98:0/3910064294 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fe80d27a0 msgr2=0x7f6fe814fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:16.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.235+0000 7f6fe77fe640 1 --2- 172.21.15.98:0/3910064294 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fe80d27a0 0x7f6fe814fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.235+0000 7f6fe77fe640 1 -- 172.21.15.98:0/3910064294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fe80d0b80 msgr2=0x7f6fe8155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:16.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.235+0000 7f6fe77fe640 1 --2- 172.21.15.98:0/3910064294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fe80d0b80 0x7f6fe8155ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.235+0000 7f6fe77fe640 1 -- 172.21.15.98:0/3910064294 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe8155ff0 con 0x7f6fe815de40 2024-05-09T10:50:16.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.235+0000 7f6fde7fc640 1 --2- 172.21.15.98:0/3910064294 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fe80d27a0 0x7f6fe814fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:16.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.235+0000 7f6fe77fe640 1 --2- 172.21.15.98:0/3910064294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fe815de40 0x7f6fe80c84e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6fe002ace0 tx=0x7f6fe0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:16.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.235+0000 7f6fdffff640 1 -- 172.21.15.98:0/3910064294 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0040070 con 0x7f6fe815de40 2024-05-09T10:50:16.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.235+0000 7f6fdffff640 1 -- 172.21.15.98:0/3910064294 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe0031c90 con 0x7f6fe815de40 2024-05-09T10:50:16.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.236+0000 7f6fed309640 1 -- 172.21.15.98:0/3910064294 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe80c91b0 con 0x7f6fe815de40 2024-05-09T10:50:16.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.236+0000 7f6fdffff640 1 -- 172.21.15.98:0/3910064294 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0045680 con 0x7f6fe815de40 2024-05-09T10:50:16.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.236+0000 7f6fed309640 1 -- 172.21.15.98:0/3910064294 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe80ca3a0 con 0x7f6fe815de40 2024-05-09T10:50:16.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.236+0000 7f6fed309640 1 -- 172.21.15.98:0/3910064294 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe8069cb0 con 0x7f6fe815de40 2024-05-09T10:50:16.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.237+0000 7f6fdffff640 1 -- 172.21.15.98:0/3910064294 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6fe0039040 con 0x7f6fe815de40 2024-05-09T10:50:16.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.237+0000 7f6fdffff640 1 --2- 172.21.15.98:0/3910064294 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6fc806b500 0x7f6fc806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:16.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.237+0000 7f6fdffff640 1 -- 172.21.15.98:0/3910064294 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6fe00ab7f0 con 0x7f6fe815de40 2024-05-09T10:50:16.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.238+0000 7f6fe6ffd640 1 --2- 172.21.15.98:0/3910064294 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6fc806b500 0x7f6fc806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:16.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.238+0000 7f6fe6ffd640 1 --2- 172.21.15.98:0/3910064294 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6fc806b500 0x7f6fc806d9c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6fd80016e0 tx=0x7f6fd8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:16.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.242+0000 7f6fdffff640 1 -- 172.21.15.98:0/3910064294 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6fe007aa60 con 0x7f6fe815de40 2024-05-09T10:50:16.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.411+0000 7f6fed309640 1 -- 172.21.15.98:0/3910064294 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fe80d42d0 con 0x7f6fe815de40 2024-05-09T10:50:16.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.412+0000 7f6fdffff640 1 -- 172.21.15.98:0/3910064294 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12884 (secure 0 0 0) 0x7f6fe007a360 con 0x7f6fe815de40 2024-05-09T10:50:16.405 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:16.406 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":16,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:09.414442+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:16.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.414+0000 7f6fed309640 1 -- 172.21.15.98:0/3910064294 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6fc806b500 msgr2=0x7f6fc806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:16.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.414+0000 7f6fed309640 1 --2- 172.21.15.98:0/3910064294 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6fc806b500 0x7f6fc806d9c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6fd80016e0 tx=0x7f6fd8009040 comp rx=0 tx=0).stop 2024-05-09T10:50:16.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.414+0000 7f6fed309640 1 -- 172.21.15.98:0/3910064294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fe815de40 msgr2=0x7f6fe80c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:16.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.414+0000 7f6fed309640 1 --2- 172.21.15.98:0/3910064294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fe815de40 0x7f6fe80c84e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6fe002ace0 tx=0x7f6fe0004a40 comp rx=0 tx=0).stop 2024-05-09T10:50:16.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.414+0000 7f6fed309640 1 -- 172.21.15.98:0/3910064294 shutdown_connections 2024-05-09T10:50:16.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.414+0000 7f6fed309640 1 --2- 172.21.15.98:0/3910064294 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6fc806b500 0x7f6fc806d9c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.414+0000 7f6fed309640 1 --2- 172.21.15.98:0/3910064294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fe815de40 0x7f6fe80c84e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.414+0000 7f6fed309640 1 --2- 172.21.15.98:0/3910064294 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6fe80d27a0 0x7f6fe814fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.414+0000 7f6fed309640 1 --2- 172.21.15.98:0/3910064294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fe80d0b80 0x7f6fe8155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.414+0000 7f6fed309640 1 -- 172.21.15.98:0/3910064294 >> 172.21.15.98:0/3910064294 conn(0x7f6fe809ce90 msgr2=0x7f6fe80b67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:16.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.414+0000 7f6fed309640 1 -- 172.21.15.98:0/3910064294 shutdown_connections 2024-05-09T10:50:16.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.414+0000 7f6fed309640 1 -- 172.21.15.98:0/3910064294 wait complete. 2024-05-09T10:50:16.419 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T10:50:07.396423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T10:50:16.420 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T10:50:16.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.529+0000 7fec7022a640 1 Processor -- start 2024-05-09T10:50:16.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.529+0000 7fec7022a640 1 -- start start 2024-05-09T10:50:16.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.530+0000 7fec7022a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fec6815dd60 0x7fec680d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:16.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.530+0000 7fec7022a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fec680d24f0 0x7fec680cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:16.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.530+0000 7fec7022a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec680cf8e0 con 0x7fec6815e140 2024-05-09T10:50:16.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.530+0000 7fec7022a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec680cf6b0 con 0x7fec680d24f0 2024-05-09T10:50:16.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.530+0000 7fec7022a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec680b2f50 con 0x7fec6815dd60 2024-05-09T10:50:16.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.530+0000 7fec6e7c7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fec680d24f0 0x7fec680cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:16.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.530+0000 7fec6d7c5640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fec6815e140 0x7fec680d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42284/0 (socket says 172.21.15.98:42284) 2024-05-09T10:50:16.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.530+0000 7fec6d7c5640 1 -- 172.21.15.98:0/780561128 learned_addr learned my addr 172.21.15.98:0/780561128 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:16.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.530+0000 7fec6dfc6640 1 --2- 172.21.15.98:0/780561128 >> v2:172.21.15.105:3300/0 conn(0x7fec6815dd60 0x7fec680d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:16.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.530+0000 7fec6e7c7640 1 -- 172.21.15.98:0/780561128 >> v2:172.21.15.105:3300/0 conn(0x7fec6815dd60 msgr2=0x7fec680d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:16.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.530+0000 7fec6e7c7640 1 --2- 172.21.15.98:0/780561128 >> v2:172.21.15.105:3300/0 conn(0x7fec6815dd60 0x7fec680d1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.530+0000 7fec6e7c7640 1 -- 172.21.15.98:0/780561128 >> v1:172.21.15.98:6789/0 conn(0x7fec6815e140 legacy=0x7fec680d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:16.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.530+0000 7fec6cfc4640 1 -- 172.21.15.98:0/780561128 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2646650115 0 0) 0x7fec680cf8e0 con 0x7fec6815e140 2024-05-09T10:50:16.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.530+0000 7fec6e7c7640 1 -- 172.21.15.98:0/780561128 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec680b31a0 con 0x7fec680d24f0 2024-05-09T10:50:16.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.531+0000 7fec6e7c7640 1 --2- 172.21.15.98:0/780561128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec680d24f0 0x7fec680cf170 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fec680b30d0 tx=0x7fec6002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc298b1426460a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:16.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.531+0000 7fec6cfc4640 1 -- 172.21.15.98:0/780561128 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60007740 con 0x7fec680d24f0 2024-05-09T10:50:16.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.531+0000 7fec6cfc4640 1 -- 172.21.15.98:0/780561128 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec600078e0 con 0x7fec680d24f0 2024-05-09T10:50:16.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.531+0000 7fec6cfc4640 1 -- 172.21.15.98:0/780561128 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60007cb0 con 0x7fec680d24f0 2024-05-09T10:50:16.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.531+0000 7fec7022a640 1 -- 172.21.15.98:0/780561128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec680d24f0 msgr2=0x7fec680cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:16.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.531+0000 7fec7022a640 1 --2- 172.21.15.98:0/780561128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec680d24f0 0x7fec680cf170 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fec680b30d0 tx=0x7fec6002a660 comp rx=0 tx=0).stop 2024-05-09T10:50:16.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.531+0000 7fec7022a640 1 -- 172.21.15.98:0/780561128 shutdown_connections 2024-05-09T10:50:16.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.531+0000 7fec7022a640 1 --2- 172.21.15.98:0/780561128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec680d24f0 0x7fec680cf170 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.531+0000 7fec7022a640 1 --2- 172.21.15.98:0/780561128 >> v2:172.21.15.105:3300/0 conn(0x7fec6815dd60 0x7fec680d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.531+0000 7fec7022a640 1 -- 172.21.15.98:0/780561128 >> 172.21.15.98:0/780561128 conn(0x7fec680d3b50 msgr2=0x7fec680d3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:16.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.532+0000 7fec7022a640 1 -- 172.21.15.98:0/780561128 shutdown_connections 2024-05-09T10:50:16.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.532+0000 7fec7022a640 1 -- 172.21.15.98:0/780561128 wait complete. 2024-05-09T10:50:16.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.533+0000 7fec7022a640 1 Processor -- start 2024-05-09T10:50:16.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.533+0000 7fec7022a640 1 -- start start 2024-05-09T10:50:16.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.533+0000 7fec7022a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec680d24f0 0x7fec68160f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:16.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.533+0000 7fec7022a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec6815dd60 0x7fec68159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:16.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.533+0000 7fec7022a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec680c8f70 0x7fec680c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:16.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.533+0000 7fec7022a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec680b3c80 con 0x7fec6815dd60 2024-05-09T10:50:16.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.533+0000 7fec7022a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec680b3760 con 0x7fec680c8f70 2024-05-09T10:50:16.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.533+0000 7fec7022a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec680b3e00 con 0x7fec680d24f0 2024-05-09T10:50:16.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.534+0000 7fec6d7c5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec6815dd60 0x7fec68159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:16.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.534+0000 7fec6d7c5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec6815dd60 0x7fec68159a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34274/0 (socket says 172.21.15.98:34274) 2024-05-09T10:50:16.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.534+0000 7fec6d7c5640 1 -- 172.21.15.98:0/3993789694 learned_addr learned my addr 172.21.15.98:0/3993789694 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:16.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.534+0000 7fec6e7c7640 1 --2- 172.21.15.98:0/3993789694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec680c8f70 0x7fec680c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:16.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.534+0000 7fec6dfc6640 1 --2- 172.21.15.98:0/3993789694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec680d24f0 0x7fec68160f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:16.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.534+0000 7fec6d7c5640 1 -- 172.21.15.98:0/3993789694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec680d24f0 msgr2=0x7fec68160f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:16.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.534+0000 7fec6d7c5640 1 --2- 172.21.15.98:0/3993789694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec680d24f0 0x7fec68160f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.534+0000 7fec6d7c5640 1 -- 172.21.15.98:0/3993789694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec680c8f70 msgr2=0x7fec680c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:16.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.534+0000 7fec6d7c5640 1 --2- 172.21.15.98:0/3993789694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec680c8f70 0x7fec680c9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.534+0000 7fec6d7c5640 1 -- 172.21.15.98:0/3993789694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec680c9df0 con 0x7fec6815dd60 2024-05-09T10:50:16.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.534+0000 7fec6e7c7640 1 --2- 172.21.15.98:0/3993789694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec680c8f70 0x7fec680c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:16.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.534+0000 7fec6d7c5640 1 --2- 172.21.15.98:0/3993789694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec6815dd60 0x7fec68159a50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fec5800c430 tx=0x7fec5800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:16.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.534+0000 7fec5e7fc640 1 -- 172.21.15.98:0/3993789694 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec580074e0 con 0x7fec6815dd60 2024-05-09T10:50:16.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.534+0000 7fec5e7fc640 1 -- 172.21.15.98:0/3993789694 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec580047e0 con 0x7fec6815dd60 2024-05-09T10:50:16.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.534+0000 7fec7022a640 1 -- 172.21.15.98:0/3993789694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec680ca0d0 con 0x7fec6815dd60 2024-05-09T10:50:16.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.534+0000 7fec5e7fc640 1 -- 172.21.15.98:0/3993789694 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec58004bd0 con 0x7fec6815dd60 2024-05-09T10:50:16.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.534+0000 7fec7022a640 1 -- 172.21.15.98:0/3993789694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec680ca690 con 0x7fec6815dd60 2024-05-09T10:50:16.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.535+0000 7fec7022a640 1 -- 172.21.15.98:0/3993789694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec680a1b60 con 0x7fec6815dd60 2024-05-09T10:50:16.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.536+0000 7fec5e7fc640 1 -- 172.21.15.98:0/3993789694 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec58011070 con 0x7fec6815dd60 2024-05-09T10:50:16.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.536+0000 7fec5e7fc640 1 --2- 172.21.15.98:0/3993789694 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fec4c06b4d0 0x7fec4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:16.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.536+0000 7fec6dfc6640 1 --2- 172.21.15.98:0/3993789694 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fec4c06b4d0 0x7fec4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:16.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.536+0000 7fec5e7fc640 1 -- 172.21.15.98:0/3993789694 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fec58089b70 con 0x7fec6815dd60 2024-05-09T10:50:16.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.537+0000 7fec6dfc6640 1 --2- 172.21.15.98:0/3993789694 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fec4c06b4d0 0x7fec4c06d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fec64006fd0 tx=0x7fec64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:16.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.540+0000 7fec5e7fc640 1 -- 172.21.15.98:0/3993789694 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fec58058d40 con 0x7fec6815dd60 2024-05-09T10:50:16.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.709+0000 7fec7022a640 1 -- 172.21.15.98:0/3993789694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fec6804f000 con 0x7fec6815dd60 2024-05-09T10:50:16.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.710+0000 7fec5e7fc640 1 -- 172.21.15.98:0/3993789694 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fec58058640 con 0x7fec6815dd60 2024-05-09T10:50:16.704 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-05-09T10:50:16.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.712+0000 7fec7022a640 1 -- 172.21.15.98:0/3993789694 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fec4c06b4d0 msgr2=0x7fec4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:16.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.712+0000 7fec7022a640 1 --2- 172.21.15.98:0/3993789694 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fec4c06b4d0 0x7fec4c06d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fec64006fd0 tx=0x7fec64008040 comp rx=0 tx=0).stop 2024-05-09T10:50:16.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.712+0000 7fec7022a640 1 -- 172.21.15.98:0/3993789694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec6815dd60 msgr2=0x7fec68159a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:16.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.712+0000 7fec7022a640 1 --2- 172.21.15.98:0/3993789694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec6815dd60 0x7fec68159a50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fec5800c430 tx=0x7fec5800c8f0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.712+0000 7fec7022a640 1 -- 172.21.15.98:0/3993789694 shutdown_connections 2024-05-09T10:50:16.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.712+0000 7fec7022a640 1 --2- 172.21.15.98:0/3993789694 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fec4c06b4d0 0x7fec4c06d990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.712+0000 7fec7022a640 1 --2- 172.21.15.98:0/3993789694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec680c8f70 0x7fec680c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.712+0000 7fec7022a640 1 --2- 172.21.15.98:0/3993789694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec6815dd60 0x7fec68159a50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.712+0000 7fec7022a640 1 --2- 172.21.15.98:0/3993789694 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fec680d24f0 0x7fec68160f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.712+0000 7fec7022a640 1 -- 172.21.15.98:0/3993789694 >> 172.21.15.98:0/3993789694 conn(0x7fec680d3b50 msgr2=0x7fec680b5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:16.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.712+0000 7fec7022a640 1 -- 172.21.15.98:0/3993789694 shutdown_connections 2024-05-09T10:50:16.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.712+0000 7fec7022a640 1 -- 172.21.15.98:0/3993789694 wait complete. 2024-05-09T10:50:16.716 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-09T10:50:16.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.822+0000 7f8f2efb0640 1 Processor -- start 2024-05-09T10:50:16.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.822+0000 7f8f2efb0640 1 -- start start 2024-05-09T10:50:16.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.822+0000 7f8f2efb0640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8f2815de40 0x7f8f280d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:16.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.822+0000 7f8f2efb0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f280d0b80 0x7f8f280b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:16.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.822+0000 7f8f2efb0640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f2815e220 con 0x7f8f280cf250 2024-05-09T10:50:16.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.822+0000 7f8f2efb0640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f280d1790 con 0x7f8f2815de40 2024-05-09T10:50:16.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.823+0000 7f8f2efb0640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f280d19e0 con 0x7f8f280d0b80 2024-05-09T10:50:16.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.823+0000 7f8f2cd4c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8f2815de40 0x7f8f280d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:16.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.823+0000 7f8f2cd4c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8f2815de40 0x7f8f280d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34276/0 (socket says 172.21.15.98:34276) 2024-05-09T10:50:16.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.823+0000 7f8f2cd4c640 1 -- 172.21.15.98:0/2628417579 learned_addr learned my addr 172.21.15.98:0/2628417579 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:16.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.823+0000 7f8f2d54d640 1 --2- 172.21.15.98:0/2628417579 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f280d0b80 0x7f8f280b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:16.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.823+0000 7f8f2cd4c640 1 -- 172.21.15.98:0/2628417579 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f280d0b80 msgr2=0x7f8f280b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:16.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.823+0000 7f8f2cd4c640 1 --2- 172.21.15.98:0/2628417579 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f280d0b80 0x7f8f280b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.823+0000 7f8f2cd4c640 1 -- 172.21.15.98:0/2628417579 >> v1:172.21.15.105:6789/0 conn(0x7f8f280cf250 legacy=0x7f8f280d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:16.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.823+0000 7f8f2cd4c640 1 -- 172.21.15.98:0/2628417579 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f280d1cc0 con 0x7f8f2815de40 2024-05-09T10:50:16.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.824+0000 7f8f2cd4c640 1 --2- 172.21.15.98:0/2628417579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f2815de40 0x7f8f280d27a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009870 tx=0x7f8f1c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5cec63bbd3876f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:16.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.824+0000 7f8f277fe640 1 -- 172.21.15.98:0/2628417579 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c037070 con 0x7f8f2815de40 2024-05-09T10:50:16.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.824+0000 7f8f277fe640 1 -- 172.21.15.98:0/2628417579 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f1c032440 con 0x7f8f2815de40 2024-05-09T10:50:16.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.824+0000 7f8f277fe640 1 -- 172.21.15.98:0/2628417579 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c032810 con 0x7f8f2815de40 2024-05-09T10:50:16.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.824+0000 7f8f2efb0640 1 -- 172.21.15.98:0/2628417579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f2815de40 msgr2=0x7f8f280d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:16.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.824+0000 7f8f2efb0640 1 --2- 172.21.15.98:0/2628417579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f2815de40 0x7f8f280d27a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009870 tx=0x7f8f1c02a270 comp rx=0 tx=0).stop 2024-05-09T10:50:16.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.824+0000 7f8f2efb0640 1 -- 172.21.15.98:0/2628417579 shutdown_connections 2024-05-09T10:50:16.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.824+0000 7f8f2efb0640 1 --2- 172.21.15.98:0/2628417579 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f280d0b80 0x7f8f280b6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.824+0000 7f8f2efb0640 1 --2- 172.21.15.98:0/2628417579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f2815de40 0x7f8f280d27a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.824+0000 7f8f2efb0640 1 -- 172.21.15.98:0/2628417579 >> 172.21.15.98:0/2628417579 conn(0x7f8f2809ce90 msgr2=0x7f8f280d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:16.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.824+0000 7f8f2efb0640 1 -- 172.21.15.98:0/2628417579 shutdown_connections 2024-05-09T10:50:16.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.824+0000 7f8f2efb0640 1 -- 172.21.15.98:0/2628417579 wait complete. 2024-05-09T10:50:16.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.825+0000 7f8f2efb0640 1 Processor -- start 2024-05-09T10:50:16.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.826+0000 7f8f2efb0640 1 -- start start 2024-05-09T10:50:16.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.826+0000 7f8f2efb0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f280cf250 0x7f8f28154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:16.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.826+0000 7f8f2efb0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f280d0b80 0x7f8f280c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:16.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.826+0000 7f8f2efb0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f2815de40 0x7f8f280c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:16.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.826+0000 7f8f2efb0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f280d4680 con 0x7f8f2815de40 2024-05-09T10:50:16.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.826+0000 7f8f2efb0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f280d2280 con 0x7f8f280d0b80 2024-05-09T10:50:16.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.826+0000 7f8f2efb0640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f280d4800 con 0x7f8f280cf250 2024-05-09T10:50:16.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.826+0000 7f8f2d54d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f2815de40 0x7f8f280c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:16.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.826+0000 7f8f2d54d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f2815de40 0x7f8f280c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34278/0 (socket says 172.21.15.98:34278) 2024-05-09T10:50:16.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.826+0000 7f8f2d54d640 1 -- 172.21.15.98:0/712375430 learned_addr learned my addr 172.21.15.98:0/712375430 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:16.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.827+0000 7f8f27fff640 1 --2- 172.21.15.98:0/712375430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f280d0b80 0x7f8f280c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:16.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.827+0000 7f8f2cd4c640 1 --2- 172.21.15.98:0/712375430 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f280cf250 0x7f8f28154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:16.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.827+0000 7f8f2d54d640 1 -- 172.21.15.98:0/712375430 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f280cf250 msgr2=0x7f8f28154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:16.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.827+0000 7f8f2d54d640 1 --2- 172.21.15.98:0/712375430 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f280cf250 0x7f8f28154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.827+0000 7f8f2d54d640 1 -- 172.21.15.98:0/712375430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f280d0b80 msgr2=0x7f8f280c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:16.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.827+0000 7f8f2d54d640 1 --2- 172.21.15.98:0/712375430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f280d0b80 0x7f8f280c9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:16.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.827+0000 7f8f2d54d640 1 -- 172.21.15.98:0/712375430 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f280caee0 con 0x7f8f2815de40 2024-05-09T10:50:16.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.827+0000 7f8f2d54d640 1 --2- 172.21.15.98:0/712375430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f2815de40 0x7f8f280c98d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8f20007de0 tx=0x7f8f2000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:16.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.827+0000 7f8f257fa640 1 -- 172.21.15.98:0/712375430 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f20014620 con 0x7f8f2815de40 2024-05-09T10:50:16.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.827+0000 7f8f257fa640 1 -- 172.21.15.98:0/712375430 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f20019070 con 0x7f8f2815de40 2024-05-09T10:50:16.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.827+0000 7f8f2efb0640 1 -- 172.21.15.98:0/712375430 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f280cb1c0 con 0x7f8f2815de40 2024-05-09T10:50:16.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.827+0000 7f8f257fa640 1 -- 172.21.15.98:0/712375430 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f20014a20 con 0x7f8f2815de40 2024-05-09T10:50:16.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.827+0000 7f8f2efb0640 1 -- 172.21.15.98:0/712375430 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f28155080 con 0x7f8f2815de40 2024-05-09T10:50:16.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.828+0000 7f8f2efb0640 1 -- 172.21.15.98:0/712375430 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f280b5300 con 0x7f8f2815de40 2024-05-09T10:50:16.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.828+0000 7f8f257fa640 1 -- 172.21.15.98:0/712375430 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f20014bc0 con 0x7f8f2815de40 2024-05-09T10:50:16.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.829+0000 7f8f257fa640 1 --2- 172.21.15.98:0/712375430 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8f0c06b4d0 0x7f8f0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:16.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.829+0000 7f8f2cd4c640 1 --2- 172.21.15.98:0/712375430 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8f0c06b4d0 0x7f8f0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:16.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.829+0000 7f8f257fa640 1 -- 172.21.15.98:0/712375430 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8f2008a760 con 0x7f8f2815de40 2024-05-09T10:50:16.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.829+0000 7f8f2cd4c640 1 --2- 172.21.15.98:0/712375430 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8f0c06b4d0 0x7f8f0c06d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c034000 tx=0x7f8f1c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:16.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:16.833+0000 7f8f257fa640 1 -- 172.21.15.98:0/712375430 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8f280b5300 con 0x7f8f2815de40 2024-05-09T10:50:16.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.004+0000 7f8f2efb0640 1 -- 172.21.15.98:0/712375430 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8f28069cb0 con 0x7f8f2815de40 2024-05-09T10:50:16.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.005+0000 7f8f257fa640 1 -- 172.21.15.98:0/712375430 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8f200599c0 con 0x7f8f2815de40 2024-05-09T10:50:16.998 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:16.998 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-05-09T10:50:16.999 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:50:16.999 INFO:teuthology.orchestra.run.smithi098.stdout: "poolnum": 1, 2024-05-09T10:50:16.999 INFO:teuthology.orchestra.run.smithi098.stdout: "poolname": ".mgr" 2024-05-09T10:50:16.999 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:50:16.999 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-05-09T10:50:17.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.007+0000 7f8f2efb0640 1 -- 172.21.15.98:0/712375430 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8f0c06b4d0 msgr2=0x7f8f0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:17.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.007+0000 7f8f2efb0640 1 --2- 172.21.15.98:0/712375430 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8f0c06b4d0 0x7f8f0c06d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c034000 tx=0x7f8f1c02e000 comp rx=0 tx=0).stop 2024-05-09T10:50:17.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.007+0000 7f8f2efb0640 1 -- 172.21.15.98:0/712375430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f2815de40 msgr2=0x7f8f280c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:17.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.007+0000 7f8f2efb0640 1 --2- 172.21.15.98:0/712375430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f2815de40 0x7f8f280c98d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8f20007de0 tx=0x7f8f2000e4b0 comp rx=0 tx=0).stop 2024-05-09T10:50:17.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.007+0000 7f8f2efb0640 1 -- 172.21.15.98:0/712375430 shutdown_connections 2024-05-09T10:50:17.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.007+0000 7f8f2efb0640 1 --2- 172.21.15.98:0/712375430 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8f0c06b4d0 0x7f8f0c06d990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:17.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.007+0000 7f8f2efb0640 1 --2- 172.21.15.98:0/712375430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f2815de40 0x7f8f280c98d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:17.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.007+0000 7f8f2efb0640 1 --2- 172.21.15.98:0/712375430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f280d0b80 0x7f8f280c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:17.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.007+0000 7f8f2efb0640 1 --2- 172.21.15.98:0/712375430 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f280cf250 0x7f8f28154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:17.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.007+0000 7f8f2efb0640 1 -- 172.21.15.98:0/712375430 >> 172.21.15.98:0/712375430 conn(0x7f8f2809ce90 msgr2=0x7f8f280b4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:17.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.007+0000 7f8f2efb0640 1 -- 172.21.15.98:0/712375430 shutdown_connections 2024-05-09T10:50:17.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.007+0000 7f8f2efb0640 1 -- 172.21.15.98:0/712375430 wait complete. 2024-05-09T10:50:17.011 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T10:50:17.011 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T10:50:17.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.121+0000 7f4b5e73b640 1 Processor -- start 2024-05-09T10:50:17.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.121+0000 7f4b5e73b640 1 -- start start 2024-05-09T10:50:17.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.122+0000 7f4b5e73b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4b580b20e0 0x7f4b580ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:17.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.122+0000 7f4b5e73b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b580b2780 0x7f4b580ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:17.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.122+0000 7f4b5e73b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4b580cec10 0x7f4b5806bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:17.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.122+0000 7f4b5e73b640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b580d1710 con 0x7f4b580b20e0 2024-05-09T10:50:17.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.122+0000 7f4b5e73b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b580b2e70 con 0x7f4b580cec10 2024-05-09T10:50:17.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.122+0000 7f4b5e73b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b580d05e0 con 0x7f4b580b2780 2024-05-09T10:50:17.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.122+0000 7f4b57fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4b580b20e0 0x7f4b580ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:17.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.122+0000 7f4b57fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4b580b20e0 0x7f4b580ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34282/0 (socket says 172.21.15.98:34282) 2024-05-09T10:50:17.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.122+0000 7f4b57fff640 1 -- 172.21.15.98:0/3367726135 learned_addr learned my addr 172.21.15.98:0/3367726135 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:17.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.122+0000 7f4b5ccd8640 1 --2- 172.21.15.98:0/3367726135 >> v2:172.21.15.105:3300/0 conn(0x7f4b580cec10 0x7f4b5806bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:17.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.122+0000 7f4b57fff640 1 -- 172.21.15.98:0/3367726135 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b580b2780 msgr2=0x7f4b580ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:17.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.122+0000 7f4b57fff640 1 --2- 172.21.15.98:0/3367726135 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b580b2780 0x7f4b580ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:17.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.122+0000 7f4b57fff640 1 -- 172.21.15.98:0/3367726135 >> v2:172.21.15.105:3300/0 conn(0x7f4b580cec10 msgr2=0x7f4b5806bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:17.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.122+0000 7f4b57fff640 1 --2- 172.21.15.98:0/3367726135 >> v2:172.21.15.105:3300/0 conn(0x7f4b580cec10 0x7f4b5806bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:17.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.122+0000 7f4b57fff640 1 -- 172.21.15.98:0/3367726135 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b580d08c0 con 0x7f4b580b20e0 2024-05-09T10:50:17.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.123+0000 7f4b57fff640 1 --2- 172.21.15.98:0/3367726135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b580b20e0 0x7f4b580ce190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4b58071ab0 tx=0x7f4b4802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4afd2a993db03f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:17.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.123+0000 7f4b577fe640 1 -- 172.21.15.98:0/3367726135 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b48037070 con 0x7f4b580b20e0 2024-05-09T10:50:17.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.123+0000 7f4b577fe640 1 -- 172.21.15.98:0/3367726135 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b48032440 con 0x7f4b580b20e0 2024-05-09T10:50:17.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.123+0000 7f4b577fe640 1 -- 172.21.15.98:0/3367726135 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b48032810 con 0x7f4b580b20e0 2024-05-09T10:50:17.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.123+0000 7f4b5e73b640 1 -- 172.21.15.98:0/3367726135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b580b20e0 msgr2=0x7f4b580ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:17.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.123+0000 7f4b5e73b640 1 --2- 172.21.15.98:0/3367726135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b580b20e0 0x7f4b580ce190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4b58071ab0 tx=0x7f4b4802a1e0 comp rx=0 tx=0).stop 2024-05-09T10:50:17.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.123+0000 7f4b5e73b640 1 -- 172.21.15.98:0/3367726135 shutdown_connections 2024-05-09T10:50:17.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.123+0000 7f4b5e73b640 1 --2- 172.21.15.98:0/3367726135 >> v2:172.21.15.105:3300/0 conn(0x7f4b580cec10 0x7f4b5806bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:17.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.123+0000 7f4b5e73b640 1 --2- 172.21.15.98:0/3367726135 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b580b2780 0x7f4b580ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:17.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.123+0000 7f4b5e73b640 1 --2- 172.21.15.98:0/3367726135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b580b20e0 0x7f4b580ce190 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:17.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.123+0000 7f4b5e73b640 1 -- 172.21.15.98:0/3367726135 >> 172.21.15.98:0/3367726135 conn(0x7f4b580b34e0 msgr2=0x7f4b580b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:17.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.123+0000 7f4b5e73b640 1 -- 172.21.15.98:0/3367726135 shutdown_connections 2024-05-09T10:50:17.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.123+0000 7f4b5e73b640 1 -- 172.21.15.98:0/3367726135 wait complete. 2024-05-09T10:50:17.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.125+0000 7f4b5e73b640 1 Processor -- start 2024-05-09T10:50:17.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.125+0000 7f4b5e73b640 1 -- start start 2024-05-09T10:50:17.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.125+0000 7f4b5e73b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b580b20e0 0x7f4b581f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:17.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.125+0000 7f4b5e73b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b580b2780 0x7f4b581f6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:17.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.125+0000 7f4b5e73b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b580cec10 0x7f4b5806e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:17.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.125+0000 7f4b5e73b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5806c230 con 0x7f4b580cec10 2024-05-09T10:50:17.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.125+0000 7f4b5e73b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b580b3150 con 0x7f4b580b20e0 2024-05-09T10:50:17.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.125+0000 7f4b5e73b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5806c3b0 con 0x7f4b580b2780 2024-05-09T10:50:17.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.125+0000 7f4b5ccd8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b580cec10 0x7f4b5806e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:17.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.125+0000 7f4b5ccd8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b580cec10 0x7f4b5806e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34286/0 (socket says 172.21.15.98:34286) 2024-05-09T10:50:17.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.125+0000 7f4b5ccd8640 1 -- 172.21.15.98:0/3383019879 learned_addr learned my addr 172.21.15.98:0/3383019879 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:17.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.125+0000 7f4b4ffff640 1 --2- 172.21.15.98:0/3383019879 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b580b2780 0x7f4b581f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:17.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.125+0000 7f4b57fff640 1 --2- 172.21.15.98:0/3383019879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b580b20e0 0x7f4b581f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:17.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.126+0000 7f4b5ccd8640 1 -- 172.21.15.98:0/3383019879 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b580b2780 msgr2=0x7f4b581f6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:17.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.126+0000 7f4b5ccd8640 1 --2- 172.21.15.98:0/3383019879 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b580b2780 0x7f4b581f6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:17.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.126+0000 7f4b5ccd8640 1 -- 172.21.15.98:0/3383019879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b580b20e0 msgr2=0x7f4b581f44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:17.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.126+0000 7f4b5ccd8640 1 --2- 172.21.15.98:0/3383019879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b580b20e0 0x7f4b581f44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:17.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.126+0000 7f4b5ccd8640 1 -- 172.21.15.98:0/3383019879 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b580b48f0 con 0x7f4b580cec10 2024-05-09T10:50:17.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.126+0000 7f4b5ccd8640 1 --2- 172.21.15.98:0/3383019879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b580cec10 0x7f4b5806e5e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000cc30 tx=0x7f4b5000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:17.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.126+0000 7f4b557fa640 1 -- 172.21.15.98:0/3383019879 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5000cc60 con 0x7f4b580cec10 2024-05-09T10:50:17.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.126+0000 7f4b557fa640 1 -- 172.21.15.98:0/3383019879 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5000ce00 con 0x7f4b580cec10 2024-05-09T10:50:17.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.126+0000 7f4b5e73b640 1 -- 172.21.15.98:0/3383019879 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b580b4bd0 con 0x7f4b580cec10 2024-05-09T10:50:17.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.126+0000 7f4b557fa640 1 -- 172.21.15.98:0/3383019879 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50003bb0 con 0x7f4b580cec10 2024-05-09T10:50:17.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.126+0000 7f4b5e73b640 1 -- 172.21.15.98:0/3383019879 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b580b5190 con 0x7f4b580cec10 2024-05-09T10:50:17.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.127+0000 7f4b5e73b640 1 -- 172.21.15.98:0/3383019879 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b5809fb40 con 0x7f4b580cec10 2024-05-09T10:50:17.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.128+0000 7f4b557fa640 1 -- 172.21.15.98:0/3383019879 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b5000f3f0 con 0x7f4b580cec10 2024-05-09T10:50:17.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.128+0000 7f4b557fa640 1 --2- 172.21.15.98:0/3383019879 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f4b3806b500 0x7f4b3806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:17.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.128+0000 7f4b57fff640 1 --2- 172.21.15.98:0/3383019879 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f4b3806b500 0x7f4b3806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:17.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.128+0000 7f4b557fa640 1 -- 172.21.15.98:0/3383019879 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4b5008a130 con 0x7f4b580cec10 2024-05-09T10:50:17.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.128+0000 7f4b57fff640 1 --2- 172.21.15.98:0/3383019879 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f4b3806b500 0x7f4b3806d9c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4b48034000 tx=0x7f4b4802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:17.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.132+0000 7f4b557fa640 1 -- 172.21.15.98:0/3383019879 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4b50014030 con 0x7f4b580cec10 2024-05-09T10:50:17.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:17.304+0000 7f4b5e73b640 1 -- 172.21.15.98:0/3383019879 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4b58161660 con 0x7f4b580cec10 2024-05-09T10:50:18.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.146+0000 7f4b557fa640 1 -- 172.21.15.98:0/3383019879 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4b50058d40 con 0x7f4b580cec10 2024-05-09T10:50:18.140 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' created 2024-05-09T10:50:18.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.150+0000 7f4b5e73b640 1 -- 172.21.15.98:0/3383019879 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f4b3806b500 msgr2=0x7f4b3806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:18.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.150+0000 7f4b5e73b640 1 --2- 172.21.15.98:0/3383019879 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f4b3806b500 0x7f4b3806d9c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4b48034000 tx=0x7f4b4802e000 comp rx=0 tx=0).stop 2024-05-09T10:50:18.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.150+0000 7f4b5e73b640 1 -- 172.21.15.98:0/3383019879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b580cec10 msgr2=0x7f4b5806e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:18.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.150+0000 7f4b5e73b640 1 --2- 172.21.15.98:0/3383019879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b580cec10 0x7f4b5806e5e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000cc30 tx=0x7f4b5000a5c0 comp rx=0 tx=0).stop 2024-05-09T10:50:18.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.150+0000 7f4b5e73b640 1 -- 172.21.15.98:0/3383019879 shutdown_connections 2024-05-09T10:50:18.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.150+0000 7f4b5e73b640 1 --2- 172.21.15.98:0/3383019879 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f4b3806b500 0x7f4b3806d9c0 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4b48034000 tx=0x7f4b4802e000 comp rx=0 tx=0).stop 2024-05-09T10:50:18.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.150+0000 7f4b5e73b640 1 --2- 172.21.15.98:0/3383019879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b580cec10 0x7f4b5806e5e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:18.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.150+0000 7f4b5e73b640 1 --2- 172.21.15.98:0/3383019879 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b580b2780 0x7f4b581f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:18.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.150+0000 7f4b5e73b640 1 --2- 172.21.15.98:0/3383019879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b580b20e0 0x7f4b581f44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:18.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.150+0000 7f4b5e73b640 1 -- 172.21.15.98:0/3383019879 >> 172.21.15.98:0/3383019879 conn(0x7f4b580b34e0 msgr2=0x7f4b5806f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:18.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.151+0000 7f4b5e73b640 1 -- 172.21.15.98:0/3383019879 shutdown_connections 2024-05-09T10:50:18.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.153+0000 7f4b5e73b640 1 -- 172.21.15.98:0/3383019879 wait complete. 2024-05-09T10:50:18.161 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T10:50:18.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.279+0000 7f7dafa9a640 1 Processor -- start 2024-05-09T10:50:18.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.279+0000 7f7dafa9a640 1 -- start start 2024-05-09T10:50:18.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.280+0000 7f7dafa9a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7da815dd60 0x7f7da80d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:18.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.280+0000 7f7dafa9a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7da80d24f0 0x7f7da80cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:18.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.280+0000 7f7dafa9a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da80cf8e0 con 0x7f7da815e140 2024-05-09T10:50:18.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.280+0000 7f7dafa9a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7da80cf6b0 con 0x7f7da815dd60 2024-05-09T10:50:18.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.280+0000 7f7dafa9a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da80b2f50 con 0x7f7da80d24f0 2024-05-09T10:50:18.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.280+0000 7f7dad035640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f7da815e140 0x7f7da80d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42286/0 (socket says 172.21.15.98:42286) 2024-05-09T10:50:18.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.280+0000 7f7dad035640 1 -- 172.21.15.98:0/2716469008 learned_addr learned my addr 172.21.15.98:0/2716469008 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:18.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.280+0000 7f7dad836640 1 --2- 172.21.15.98:0/2716469008 >> v2:172.21.15.105:3300/0 conn(0x7f7da815dd60 0x7f7da80d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:18.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.280+0000 7f7dae037640 1 --2- 172.21.15.98:0/2716469008 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7da80d24f0 0x7f7da80cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:18.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.280+0000 7f7dac834640 1 -- 172.21.15.98:0/2716469008 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1828960129 0 0) 0x7f7da80cf8e0 con 0x7f7da815e140 2024-05-09T10:50:18.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.280+0000 7f7dac834640 1 -- 172.21.15.98:0/2716469008 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d94003520 con 0x7f7da815e140 2024-05-09T10:50:18.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.280+0000 7f7dac834640 1 -- 172.21.15.98:0/2716469008 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2615740635 0 0) 0x7f7d94003520 con 0x7f7da815e140 2024-05-09T10:50:18.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.280+0000 7f7dae037640 1 -- 172.21.15.98:0/2716469008 >> v2:172.21.15.105:3300/0 conn(0x7f7da815dd60 msgr2=0x7f7da80d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:18.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.280+0000 7f7dae037640 1 --2- 172.21.15.98:0/2716469008 >> v2:172.21.15.105:3300/0 conn(0x7f7da815dd60 0x7f7da80d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:18.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.280+0000 7f7dae037640 1 -- 172.21.15.98:0/2716469008 >> v1:172.21.15.98:6789/0 conn(0x7f7da815e140 legacy=0x7f7da80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:18.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.280+0000 7f7dae037640 1 -- 172.21.15.98:0/2716469008 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da80b31a0 con 0x7f7da80d24f0 2024-05-09T10:50:18.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.280+0000 7f7dad836640 1 --2- 172.21.15.98:0/2716469008 >> v2:172.21.15.105:3300/0 conn(0x7f7da815dd60 0x7f7da80d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:50:18.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.281+0000 7f7dae037640 1 --2- 172.21.15.98:0/2716469008 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7da80d24f0 0x7f7da80cf170 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7da00097a0 tx=0x7f7da002a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=799703c7ec9834c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:18.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.281+0000 7f7dac834640 1 -- 172.21.15.98:0/2716469008 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da0037070 con 0x7f7da80d24f0 2024-05-09T10:50:18.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.281+0000 7f7dac834640 1 -- 172.21.15.98:0/2716469008 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da002aba0 con 0x7f7da80d24f0 2024-05-09T10:50:18.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.281+0000 7f7dac834640 1 -- 172.21.15.98:0/2716469008 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d98001670 con 0x7f7da80d24f0 2024-05-09T10:50:18.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.282+0000 7f7dafa9a640 1 -- 172.21.15.98:0/2716469008 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7da80d24f0 msgr2=0x7f7da80cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:18.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.282+0000 7f7dafa9a640 1 --2- 172.21.15.98:0/2716469008 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7da80d24f0 0x7f7da80cf170 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7da00097a0 tx=0x7f7da002a110 comp rx=0 tx=0).stop 2024-05-09T10:50:18.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.282+0000 7f7dafa9a640 1 -- 172.21.15.98:0/2716469008 shutdown_connections 2024-05-09T10:50:18.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.282+0000 7f7dafa9a640 1 --2- 172.21.15.98:0/2716469008 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7da80d24f0 0x7f7da80cf170 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:18.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.282+0000 7f7dafa9a640 1 --2- 172.21.15.98:0/2716469008 >> v2:172.21.15.105:3300/0 conn(0x7f7da815dd60 0x7f7da80d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:18.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.282+0000 7f7dafa9a640 1 -- 172.21.15.98:0/2716469008 >> 172.21.15.98:0/2716469008 conn(0x7f7da80d3b50 msgr2=0x7f7da80d3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:18.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.282+0000 7f7dafa9a640 1 -- 172.21.15.98:0/2716469008 shutdown_connections 2024-05-09T10:50:18.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.282+0000 7f7dafa9a640 1 -- 172.21.15.98:0/2716469008 wait complete. 2024-05-09T10:50:18.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.283+0000 7f7dafa9a640 1 Processor -- start 2024-05-09T10:50:18.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.283+0000 7f7dafa9a640 1 -- start start 2024-05-09T10:50:18.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.284+0000 7f7dafa9a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7da80d24f0 0x7f7da81f7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:18.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.284+0000 7f7dafa9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7da815dd60 0x7f7da815be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:18.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.284+0000 7f7dafa9a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7da80b6730 0x7f7da80c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:18.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.284+0000 7f7dafa9a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da80b3c80 con 0x7f7da80d24f0 2024-05-09T10:50:18.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.284+0000 7f7dafa9a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da80b3760 con 0x7f7da815dd60 2024-05-09T10:50:18.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.284+0000 7f7dafa9a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da80b3e00 con 0x7f7da80b6730 2024-05-09T10:50:18.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.284+0000 7f7dad836640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7da80d24f0 0x7f7da81f7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:18.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.284+0000 7f7dad836640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7da80d24f0 0x7f7da81f7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34288/0 (socket says 172.21.15.98:34288) 2024-05-09T10:50:18.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.284+0000 7f7dad836640 1 -- 172.21.15.98:0/1098334067 learned_addr learned my addr 172.21.15.98:0/1098334067 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:18.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.284+0000 7f7dad035640 1 --2- 172.21.15.98:0/1098334067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7da815dd60 0x7f7da815be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:18.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.284+0000 7f7dae037640 1 --2- 172.21.15.98:0/1098334067 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7da80b6730 0x7f7da80c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:18.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.284+0000 7f7dad836640 1 -- 172.21.15.98:0/1098334067 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7da80b6730 msgr2=0x7f7da80c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:18.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.284+0000 7f7dad836640 1 --2- 172.21.15.98:0/1098334067 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7da80b6730 0x7f7da80c7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:18.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.284+0000 7f7dad836640 1 -- 172.21.15.98:0/1098334067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7da815dd60 msgr2=0x7f7da815be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:18.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.284+0000 7f7dad836640 1 --2- 172.21.15.98:0/1098334067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7da815dd60 0x7f7da815be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:18.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.284+0000 7f7dad836640 1 -- 172.21.15.98:0/1098334067 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da80b7860 con 0x7f7da80d24f0 2024-05-09T10:50:18.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.284+0000 7f7dad836640 1 --2- 172.21.15.98:0/1098334067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7da80d24f0 0x7f7da81f7030 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7da400efc0 tx=0x7f7da400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:18.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.285+0000 7f7d9e7fc640 1 -- 172.21.15.98:0/1098334067 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da4018020 con 0x7f7da80d24f0 2024-05-09T10:50:18.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.285+0000 7f7d9e7fc640 1 -- 172.21.15.98:0/1098334067 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da4014680 con 0x7f7da80d24f0 2024-05-09T10:50:18.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.285+0000 7f7dafa9a640 1 -- 172.21.15.98:0/1098334067 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da81f7880 con 0x7f7da80d24f0 2024-05-09T10:50:18.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.285+0000 7f7d9e7fc640 1 -- 172.21.15.98:0/1098334067 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da4014a50 con 0x7f7da80d24f0 2024-05-09T10:50:18.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.285+0000 7f7dafa9a640 1 -- 172.21.15.98:0/1098334067 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da81f7e40 con 0x7f7da80d24f0 2024-05-09T10:50:18.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.285+0000 7f7dafa9a640 1 -- 172.21.15.98:0/1098334067 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7da80a1b60 con 0x7f7da80d24f0 2024-05-09T10:50:18.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.286+0000 7f7d9e7fc640 1 -- 172.21.15.98:0/1098334067 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7da4005d10 con 0x7f7da80d24f0 2024-05-09T10:50:18.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.287+0000 7f7d9e7fc640 1 --2- 172.21.15.98:0/1098334067 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f7d8c06b4d0 0x7f7d8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:18.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.287+0000 7f7d9e7fc640 1 -- 172.21.15.98:0/1098334067 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f7da408ae00 con 0x7f7da80d24f0 2024-05-09T10:50:18.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.287+0000 7f7dad035640 1 --2- 172.21.15.98:0/1098334067 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f7d8c06b4d0 0x7f7d8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:18.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.287+0000 7f7dad035640 1 --2- 172.21.15.98:0/1098334067 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f7d8c06b4d0 0x7f7d8c06d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7da81615c0 tx=0x7f7d98009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:18.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.291+0000 7f7d9e7fc640 1 -- 172.21.15.98:0/1098334067 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7da4022030 con 0x7f7da80d24f0 2024-05-09T10:50:18.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:18.463+0000 7f7dafa9a640 1 -- 172.21.15.98:0/1098334067 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7da804f000 con 0x7f7da80d24f0 2024-05-09T10:50:19.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.149+0000 7f7d9e7fc640 1 -- 172.21.15.98:0/1098334067 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7da4010040 con 0x7f7da80d24f0 2024-05-09T10:50:19.142 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' created 2024-05-09T10:50:19.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.151+0000 7f7dafa9a640 1 -- 172.21.15.98:0/1098334067 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f7d8c06b4d0 msgr2=0x7f7d8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:19.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.151+0000 7f7dafa9a640 1 --2- 172.21.15.98:0/1098334067 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f7d8c06b4d0 0x7f7d8c06d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7da81615c0 tx=0x7f7d98009040 comp rx=0 tx=0).stop 2024-05-09T10:50:19.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.151+0000 7f7dafa9a640 1 -- 172.21.15.98:0/1098334067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7da80d24f0 msgr2=0x7f7da81f7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:19.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.151+0000 7f7dafa9a640 1 --2- 172.21.15.98:0/1098334067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7da80d24f0 0x7f7da81f7030 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7da400efc0 tx=0x7f7da400c490 comp rx=0 tx=0).stop 2024-05-09T10:50:19.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.152+0000 7f7dafa9a640 1 -- 172.21.15.98:0/1098334067 shutdown_connections 2024-05-09T10:50:19.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.152+0000 7f7dafa9a640 1 --2- 172.21.15.98:0/1098334067 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f7d8c06b4d0 0x7f7d8c06d990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:19.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.152+0000 7f7dafa9a640 1 --2- 172.21.15.98:0/1098334067 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7da80b6730 0x7f7da80c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:19.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.152+0000 7f7dafa9a640 1 --2- 172.21.15.98:0/1098334067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7da815dd60 0x7f7da815be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:19.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.152+0000 7f7dafa9a640 1 --2- 172.21.15.98:0/1098334067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7da80d24f0 0x7f7da81f7030 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7da400efc0 tx=0x7f7da400c490 comp rx=0 tx=0).stop 2024-05-09T10:50:19.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.152+0000 7f7dafa9a640 1 -- 172.21.15.98:0/1098334067 >> 172.21.15.98:0/1098334067 conn(0x7f7da80d3b50 msgr2=0x7f7da80b5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:19.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.152+0000 7f7dafa9a640 1 -- 172.21.15.98:0/1098334067 shutdown_connections 2024-05-09T10:50:19.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.153+0000 7f7dafa9a640 1 -- 172.21.15.98:0/1098334067 wait complete. 2024-05-09T10:50:19.151 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T10:50:19.155 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T10:50:19.171 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T10:50:19.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.323+0000 7f0ad8a9e640 1 Processor -- start 2024-05-09T10:50:19.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.323+0000 7f0ad8a9e640 1 -- start start 2024-05-09T10:50:19.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.324+0000 7f0ad8a9e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0ad408b2a0 0x7f0ad408b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:19.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.324+0000 7f0ad8a9e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ad40ce5c0 0x7f0ad40cea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:19.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.324+0000 7f0ad8a9e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ad40cab80 con 0x7f0ad40ca7a0 2024-05-09T10:50:19.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.324+0000 7f0ad8a9e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ad40c4d30 con 0x7f0ad408b2a0 2024-05-09T10:50:19.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.324+0000 7f0ad8a9e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad408bc20 con 0x7f0ad40ce5c0 2024-05-09T10:50:19.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.324+0000 7f0ad1d9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0ad408b2a0 0x7f0ad408b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:19.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.324+0000 7f0ad1d9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0ad408b2a0 0x7f0ad408b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34304/0 (socket says 172.21.15.98:34304) 2024-05-09T10:50:19.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.324+0000 7f0ad1d9b640 1 -- 172.21.15.98:0/4245626472 learned_addr learned my addr 172.21.15.98:0/4245626472 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:19.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.324+0000 7f0ad259c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f0ad40ca7a0 0x7f0ad40c4620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42292/0 (socket says 172.21.15.98:42292) 2024-05-09T10:50:19.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.324+0000 7f0ad159a640 1 -- 172.21.15.98:0/4245626472 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2446287928 0 0) 0x7f0ad40cab80 con 0x7f0ad40ca7a0 2024-05-09T10:50:19.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.324+0000 7f0ad2d9d640 1 --2- 172.21.15.98:0/4245626472 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ad40ce5c0 0x7f0ad40cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:19.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.325+0000 7f0ad159a640 1 -- 172.21.15.98:0/4245626472 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0abc003500 con 0x7f0ad40ca7a0 2024-05-09T10:50:19.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.325+0000 7f0ad1d9b640 1 -- 172.21.15.98:0/4245626472 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ad40ce5c0 msgr2=0x7f0ad40cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:19.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.325+0000 7f0ad1d9b640 1 --2- 172.21.15.98:0/4245626472 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ad40ce5c0 0x7f0ad40cea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:19.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.325+0000 7f0ad1d9b640 1 -- 172.21.15.98:0/4245626472 >> v1:172.21.15.98:6789/0 conn(0x7f0ad40ca7a0 legacy=0x7f0ad40c4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:19.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.325+0000 7f0ad159a640 1 -- 172.21.15.98:0/4245626472 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2223192571 0 0) 0x7f0abc003500 con 0x7f0ad40ca7a0 2024-05-09T10:50:19.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.325+0000 7f0ad1d9b640 1 -- 172.21.15.98:0/4245626472 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad40cef90 con 0x7f0ad408b2a0 2024-05-09T10:50:19.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.326+0000 7f0ad1d9b640 1 --2- 172.21.15.98:0/4245626472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ad408b2a0 0x7f0ad408b6e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ac800b970 tx=0x7f0ac802a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e263bfeefc108271 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:19.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.326+0000 7f0ad159a640 1 -- 172.21.15.98:0/4245626472 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8038070 con 0x7f0ad408b2a0 2024-05-09T10:50:19.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.326+0000 7f0ad159a640 1 -- 172.21.15.98:0/4245626472 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac8030480 con 0x7f0ad408b2a0 2024-05-09T10:50:19.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.326+0000 7f0ad159a640 1 -- 172.21.15.98:0/4245626472 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac40019f0 con 0x7f0ad408b2a0 2024-05-09T10:50:19.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.326+0000 7f0ad8a9e640 1 -- 172.21.15.98:0/4245626472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ad408b2a0 msgr2=0x7f0ad408b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:19.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.326+0000 7f0ad8a9e640 1 --2- 172.21.15.98:0/4245626472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ad408b2a0 0x7f0ad408b6e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ac800b970 tx=0x7f0ac802a870 comp rx=0 tx=0).stop 2024-05-09T10:50:19.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.327+0000 7f0ad8a9e640 1 -- 172.21.15.98:0/4245626472 shutdown_connections 2024-05-09T10:50:19.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.327+0000 7f0ad8a9e640 1 --2- 172.21.15.98:0/4245626472 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ad40ce5c0 0x7f0ad40cea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:19.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.327+0000 7f0ad8a9e640 1 --2- 172.21.15.98:0/4245626472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ad408b2a0 0x7f0ad408b6e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:19.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.327+0000 7f0ad8a9e640 1 -- 172.21.15.98:0/4245626472 >> 172.21.15.98:0/4245626472 conn(0x7f0ad40cc1e0 msgr2=0x7f0ad40cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:19.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.327+0000 7f0ad8a9e640 1 -- 172.21.15.98:0/4245626472 shutdown_connections 2024-05-09T10:50:19.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.327+0000 7f0ad8a9e640 1 -- 172.21.15.98:0/4245626472 wait complete. 2024-05-09T10:50:19.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.328+0000 7f0ad8a9e640 1 Processor -- start 2024-05-09T10:50:19.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.328+0000 7f0ad8a9e640 1 -- start start 2024-05-09T10:50:19.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.329+0000 7f0ad8a9e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ad408b2a0 0x7f0ad40c6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:19.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.329+0000 7f0ad8a9e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ad40ca7a0 0x7f0ad406d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:19.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.329+0000 7f0ad8a9e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ad40ce5c0 0x7f0ad406d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:19.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.329+0000 7f0ad8a9e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad40cddc0 con 0x7f0ad408b2a0 2024-05-09T10:50:19.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.329+0000 7f0ad8a9e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad40cdae0 con 0x7f0ad40ce5c0 2024-05-09T10:50:19.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.329+0000 7f0ad8a9e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad40ce0a0 con 0x7f0ad40ca7a0 2024-05-09T10:50:19.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.329+0000 7f0ad259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ad408b2a0 0x7f0ad40c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:19.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.329+0000 7f0ad259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ad408b2a0 0x7f0ad40c6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34308/0 (socket says 172.21.15.98:34308) 2024-05-09T10:50:19.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.329+0000 7f0ad259c640 1 -- 172.21.15.98:0/2733533936 learned_addr learned my addr 172.21.15.98:0/2733533936 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:19.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.329+0000 7f0ad1d9b640 1 --2- 172.21.15.98:0/2733533936 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ad40ca7a0 0x7f0ad406d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:19.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.329+0000 7f0ad2d9d640 1 --2- 172.21.15.98:0/2733533936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ad40ce5c0 0x7f0ad406d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:19.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.329+0000 7f0ad259c640 1 -- 172.21.15.98:0/2733533936 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ad40ca7a0 msgr2=0x7f0ad406d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:19.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.329+0000 7f0ad259c640 1 --2- 172.21.15.98:0/2733533936 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ad40ca7a0 0x7f0ad406d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:19.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.329+0000 7f0ad259c640 1 -- 172.21.15.98:0/2733533936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ad40ce5c0 msgr2=0x7f0ad406d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:19.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.329+0000 7f0ad259c640 1 --2- 172.21.15.98:0/2733533936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ad40ce5c0 0x7f0ad406d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:19.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.329+0000 7f0ad259c640 1 -- 172.21.15.98:0/2733533936 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad40b2480 con 0x7f0ad408b2a0 2024-05-09T10:50:19.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.329+0000 7f0ad1d9b640 1 --2- 172.21.15.98:0/2733533936 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ad40ca7a0 0x7f0ad406d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:19.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.329+0000 7f0ad259c640 1 --2- 172.21.15.98:0/2733533936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ad408b2a0 0x7f0ad40c6830 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0ac400cb20 tx=0x7f0ac4007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:19.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.329+0000 7f0ac2ffd640 1 -- 172.21.15.98:0/2733533936 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4017070 con 0x7f0ad408b2a0 2024-05-09T10:50:19.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.329+0000 7f0ac2ffd640 1 -- 172.21.15.98:0/2733533936 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac400a700 con 0x7f0ad408b2a0 2024-05-09T10:50:19.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.329+0000 7f0ac2ffd640 1 -- 172.21.15.98:0/2733533936 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac400aaf0 con 0x7f0ad408b2a0 2024-05-09T10:50:19.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.329+0000 7f0ad8a9e640 1 -- 172.21.15.98:0/2733533936 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad40b08d0 con 0x7f0ad408b2a0 2024-05-09T10:50:19.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.329+0000 7f0ad8a9e640 1 -- 172.21.15.98:0/2733533936 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad4166a30 con 0x7f0ad408b2a0 2024-05-09T10:50:19.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.330+0000 7f0ad8a9e640 1 -- 172.21.15.98:0/2733533936 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ad4072c80 con 0x7f0ad408b2a0 2024-05-09T10:50:19.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.331+0000 7f0ac2ffd640 1 -- 172.21.15.98:0/2733533936 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ac401a070 con 0x7f0ad408b2a0 2024-05-09T10:50:19.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.331+0000 7f0ac2ffd640 1 --2- 172.21.15.98:0/2733533936 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0ab406b4d0 0x7f0ab406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:19.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.331+0000 7f0ac2ffd640 1 -- 172.21.15.98:0/2733533936 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f0ac4089a60 con 0x7f0ad408b2a0 2024-05-09T10:50:19.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.331+0000 7f0ad1d9b640 1 --2- 172.21.15.98:0/2733533936 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0ab406b4d0 0x7f0ab406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:19.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.332+0000 7f0ad1d9b640 1 --2- 172.21.15.98:0/2733533936 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0ab406b4d0 0x7f0ab406d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4068510 tx=0x7f0ac802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:19.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.336+0000 7f0ac2ffd640 1 -- 172.21.15.98:0/2733533936 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ad40b2480 con 0x7f0ad408b2a0 2024-05-09T10:50:19.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:19.542+0000 7f0ad8a9e640 1 -- 172.21.15.98:0/2733533936 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0ad404f000 con 0x7f0ad408b2a0 2024-05-09T10:50:20.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.164+0000 7f0ac2ffd640 1 -- 172.21.15.98:0/2733533936 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0ac4058940 con 0x7f0ad408b2a0 2024-05-09T10:50:20.157 INFO:teuthology.orchestra.run.smithi098.stderr:new fs with metadata pool 2 and data pool 3 2024-05-09T10:50:20.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.166+0000 7f0ad8a9e640 1 -- 172.21.15.98:0/2733533936 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0ab406b4d0 msgr2=0x7f0ab406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:20.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.166+0000 7f0ad8a9e640 1 --2- 172.21.15.98:0/2733533936 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0ab406b4d0 0x7f0ab406d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4068510 tx=0x7f0ac802e000 comp rx=0 tx=0).stop 2024-05-09T10:50:20.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.166+0000 7f0ad8a9e640 1 -- 172.21.15.98:0/2733533936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ad408b2a0 msgr2=0x7f0ad40c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:20.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.166+0000 7f0ad8a9e640 1 --2- 172.21.15.98:0/2733533936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ad408b2a0 0x7f0ad40c6830 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0ac400cb20 tx=0x7f0ac4007e20 comp rx=0 tx=0).stop 2024-05-09T10:50:20.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.166+0000 7f0ad8a9e640 1 -- 172.21.15.98:0/2733533936 shutdown_connections 2024-05-09T10:50:20.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.166+0000 7f0ad8a9e640 1 --2- 172.21.15.98:0/2733533936 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0ab406b4d0 0x7f0ab406d990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.166+0000 7f0ad8a9e640 1 --2- 172.21.15.98:0/2733533936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ad40ce5c0 0x7f0ad406d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.166+0000 7f0ad8a9e640 1 --2- 172.21.15.98:0/2733533936 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ad40ca7a0 0x7f0ad406d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.166+0000 7f0ad8a9e640 1 --2- 172.21.15.98:0/2733533936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ad408b2a0 0x7f0ad40c6830 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.166+0000 7f0ad8a9e640 1 -- 172.21.15.98:0/2733533936 >> 172.21.15.98:0/2733533936 conn(0x7f0ad40cc1e0 msgr2=0x7f0ad416d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:20.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.166+0000 7f0ad8a9e640 1 -- 172.21.15.98:0/2733533936 shutdown_connections 2024-05-09T10:50:20.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.166+0000 7f0ad8a9e640 1 -- 172.21.15.98:0/2733533936 wait complete. 2024-05-09T10:50:20.179 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:20.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.329+0000 7fb5b93f5640 1 Processor -- start 2024-05-09T10:50:20.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.329+0000 7fb5b93f5640 1 -- start start 2024-05-09T10:50:20.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.330+0000 7fb5b93f5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5b415de40 0x7fb5b40d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:20.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.330+0000 7fb5b93f5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb5b40cf250 0x7fb5b40cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:20.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.330+0000 7fb5b93f5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb5b40d0470 0x7fb5b40d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:20.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.330+0000 7fb5b93f5640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5b40d1790 con 0x7fb5b40cf250 2024-05-09T10:50:20.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.330+0000 7fb5b93f5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5b415e220 con 0x7fb5b40d0470 2024-05-09T10:50:20.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.330+0000 7fb5b93f5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b40d19e0 con 0x7fb5b415de40 2024-05-09T10:50:20.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.330+0000 7fb5aa7fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb5b40cf250 0x7fb5b40cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:20.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.330+0000 7fb5aa7fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb5b40cf250 0x7fb5b40cf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34318/0 (socket says 172.21.15.98:34318) 2024-05-09T10:50:20.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.330+0000 7fb5aa7fc640 1 -- 172.21.15.98:0/2347649519 learned_addr learned my addr 172.21.15.98:0/2347649519 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:20.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.330+0000 7fb5b37fe640 1 --2- 172.21.15.98:0/2347649519 >> v2:172.21.15.105:3300/0 conn(0x7fb5b40d0470 0x7fb5b40d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:20.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.330+0000 7fb5b2ffd640 1 --2- 172.21.15.98:0/2347649519 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5b415de40 0x7fb5b40d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:20.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.330+0000 7fb5aa7fc640 1 -- 172.21.15.98:0/2347649519 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5b415de40 msgr2=0x7fb5b40d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:20.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.330+0000 7fb5aa7fc640 1 --2- 172.21.15.98:0/2347649519 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5b415de40 0x7fb5b40d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.330+0000 7fb5aa7fc640 1 -- 172.21.15.98:0/2347649519 >> v2:172.21.15.105:3300/0 conn(0x7fb5b40d0470 msgr2=0x7fb5b40d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:20.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.330+0000 7fb5aa7fc640 1 --2- 172.21.15.98:0/2347649519 >> v2:172.21.15.105:3300/0 conn(0x7fb5b40d0470 0x7fb5b40d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.330+0000 7fb5aa7fc640 1 -- 172.21.15.98:0/2347649519 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b40d1cc0 con 0x7fb5b40cf250 2024-05-09T10:50:20.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.331+0000 7fb5aa7fc640 1 --2- 172.21.15.98:0/2347649519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5b40cf250 0x7fb5b40cf6b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4072ed0 tx=0x7fb59c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5752ed00b3d64 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:20.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.331+0000 7fb5b27fc640 1 -- 172.21.15.98:0/2347649519 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c037070 con 0x7fb5b40cf250 2024-05-09T10:50:20.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.331+0000 7fb5b27fc640 1 -- 172.21.15.98:0/2347649519 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb59c032440 con 0x7fb5b40cf250 2024-05-09T10:50:20.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.331+0000 7fb5b27fc640 1 -- 172.21.15.98:0/2347649519 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c032810 con 0x7fb5b40cf250 2024-05-09T10:50:20.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.331+0000 7fb5b93f5640 1 -- 172.21.15.98:0/2347649519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5b40cf250 msgr2=0x7fb5b40cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:20.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.331+0000 7fb5b93f5640 1 --2- 172.21.15.98:0/2347649519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5b40cf250 0x7fb5b40cf6b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4072ed0 tx=0x7fb59c02a1e0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.332+0000 7fb5b93f5640 1 -- 172.21.15.98:0/2347649519 shutdown_connections 2024-05-09T10:50:20.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.332+0000 7fb5b93f5640 1 --2- 172.21.15.98:0/2347649519 >> v2:172.21.15.105:3300/0 conn(0x7fb5b40d0470 0x7fb5b40d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.332+0000 7fb5b93f5640 1 --2- 172.21.15.98:0/2347649519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5b40cf250 0x7fb5b40cf6b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.332+0000 7fb5b93f5640 1 --2- 172.21.15.98:0/2347649519 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5b415de40 0x7fb5b40d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.332+0000 7fb5b93f5640 1 -- 172.21.15.98:0/2347649519 >> 172.21.15.98:0/2347649519 conn(0x7fb5b409ce90 msgr2=0x7fb5b40d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:20.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.332+0000 7fb5b93f5640 1 -- 172.21.15.98:0/2347649519 shutdown_connections 2024-05-09T10:50:20.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.332+0000 7fb5b93f5640 1 -- 172.21.15.98:0/2347649519 wait complete. 2024-05-09T10:50:20.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.333+0000 7fb5b93f5640 1 Processor -- start 2024-05-09T10:50:20.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.333+0000 7fb5b93f5640 1 -- start start 2024-05-09T10:50:20.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.334+0000 7fb5b93f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5b40cf250 0x7fb5b41f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:20.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.334+0000 7fb5b93f5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5b40d0470 0x7fb5b40cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:20.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.334+0000 7fb5b93f5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5b415de40 0x7fb5b40cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:20.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.334+0000 7fb5b93f5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b40d4680 con 0x7fb5b415de40 2024-05-09T10:50:20.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.334+0000 7fb5b93f5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b40d2280 con 0x7fb5b40cf250 2024-05-09T10:50:20.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.334+0000 7fb5b93f5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b40d4800 con 0x7fb5b40d0470 2024-05-09T10:50:20.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.334+0000 7fb5b37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5b415de40 0x7fb5b40cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:20.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.334+0000 7fb5b37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5b415de40 0x7fb5b40cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34330/0 (socket says 172.21.15.98:34330) 2024-05-09T10:50:20.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.334+0000 7fb5b37fe640 1 -- 172.21.15.98:0/3541107250 learned_addr learned my addr 172.21.15.98:0/3541107250 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:20.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.334+0000 7fb5b2ffd640 1 --2- 172.21.15.98:0/3541107250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5b40cf250 0x7fb5b41f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:20.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.334+0000 7fb5aa7fc640 1 --2- 172.21.15.98:0/3541107250 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5b40d0470 0x7fb5b40cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:20.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.334+0000 7fb5b37fe640 1 -- 172.21.15.98:0/3541107250 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5b40d0470 msgr2=0x7fb5b40cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:20.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.334+0000 7fb5b37fe640 1 --2- 172.21.15.98:0/3541107250 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5b40d0470 0x7fb5b40cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.334+0000 7fb5b37fe640 1 -- 172.21.15.98:0/3541107250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5b40cf250 msgr2=0x7fb5b41f68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:20.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.334+0000 7fb5b37fe640 1 --2- 172.21.15.98:0/3541107250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5b40cf250 0x7fb5b41f68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.334+0000 7fb5b37fe640 1 -- 172.21.15.98:0/3541107250 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b41588e0 con 0x7fb5b415de40 2024-05-09T10:50:20.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.334+0000 7fb5aa7fc640 1 --2- 172.21.15.98:0/3541107250 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5b40d0470 0x7fb5b40cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:20.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.334+0000 7fb5b37fe640 1 --2- 172.21.15.98:0/3541107250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5b415de40 0x7fb5b40cca40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac007e60 tx=0x7fb5ac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:20.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.334+0000 7fb5abfff640 1 -- 172.21.15.98:0/3541107250 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac014620 con 0x7fb5b415de40 2024-05-09T10:50:20.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.334+0000 7fb5abfff640 1 -- 172.21.15.98:0/3541107250 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ac019070 con 0x7fb5b415de40 2024-05-09T10:50:20.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.334+0000 7fb5b93f5640 1 -- 172.21.15.98:0/3541107250 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b4158bc0 con 0x7fb5b415de40 2024-05-09T10:50:20.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.334+0000 7fb5abfff640 1 -- 172.21.15.98:0/3541107250 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac014a20 con 0x7fb5b415de40 2024-05-09T10:50:20.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.334+0000 7fb5b93f5640 1 -- 172.21.15.98:0/3541107250 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b4159180 con 0x7fb5b415de40 2024-05-09T10:50:20.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.335+0000 7fb5b93f5640 1 -- 172.21.15.98:0/3541107250 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5b4069cb0 con 0x7fb5b415de40 2024-05-09T10:50:20.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.336+0000 7fb5abfff640 1 -- 172.21.15.98:0/3541107250 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb5ac014bc0 con 0x7fb5b415de40 2024-05-09T10:50:20.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.336+0000 7fb5abfff640 1 --2- 172.21.15.98:0/3541107250 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb59406b500 0x7fb59406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:20.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.336+0000 7fb5b2ffd640 1 --2- 172.21.15.98:0/3541107250 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb59406b500 0x7fb59406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:20.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.336+0000 7fb5abfff640 1 -- 172.21.15.98:0/3541107250 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5ac08ab20 con 0x7fb5b415de40 2024-05-09T10:50:20.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.337+0000 7fb5b2ffd640 1 --2- 172.21.15.98:0/3541107250 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb59406b500 0x7fb59406d9c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4006fd0 tx=0x7fb5a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:20.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.340+0000 7fb5abfff640 1 -- 172.21.15.98:0/3541107250 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb5ac059910 con 0x7fb5b415de40 2024-05-09T10:50:20.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.516+0000 7fb5b93f5640 1 -- 172.21.15.98:0/3541107250 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5b40d42d0 con 0x7fb5b415de40 2024-05-09T10:50:20.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.517+0000 7fb5abfff640 1 -- 172.21.15.98:0/3541107250 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16618 (secure 0 0 0) 0x7fb5ac059210 con 0x7fb5b415de40 2024-05-09T10:50:20.511 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:20.511 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":19,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:19.544125+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:50:17.323444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T10:50:18.482315+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:20.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.519+0000 7fb5b93f5640 1 -- 172.21.15.98:0/3541107250 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb59406b500 msgr2=0x7fb59406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:20.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.519+0000 7fb5b93f5640 1 --2- 172.21.15.98:0/3541107250 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb59406b500 0x7fb59406d9c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4006fd0 tx=0x7fb5a4008040 comp rx=0 tx=0).stop 2024-05-09T10:50:20.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.519+0000 7fb5b93f5640 1 -- 172.21.15.98:0/3541107250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5b415de40 msgr2=0x7fb5b40cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:20.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.519+0000 7fb5b93f5640 1 --2- 172.21.15.98:0/3541107250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5b415de40 0x7fb5b40cca40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac007e60 tx=0x7fb5ac00e4b0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.519+0000 7fb5b93f5640 1 -- 172.21.15.98:0/3541107250 shutdown_connections 2024-05-09T10:50:20.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.519+0000 7fb5b93f5640 1 --2- 172.21.15.98:0/3541107250 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb59406b500 0x7fb59406d9c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.519+0000 7fb5b93f5640 1 --2- 172.21.15.98:0/3541107250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5b415de40 0x7fb5b40cca40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.519+0000 7fb5b93f5640 1 --2- 172.21.15.98:0/3541107250 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb5b40d0470 0x7fb5b40cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.519+0000 7fb5b93f5640 1 --2- 172.21.15.98:0/3541107250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5b40cf250 0x7fb5b41f68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.519+0000 7fb5b93f5640 1 -- 172.21.15.98:0/3541107250 >> 172.21.15.98:0/3541107250 conn(0x7fb5b409ce90 msgr2=0x7fb5b40b8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:20.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.519+0000 7fb5b93f5640 1 -- 172.21.15.98:0/3541107250 shutdown_connections 2024-05-09T10:50:20.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.519+0000 7fb5b93f5640 1 -- 172.21.15.98:0/3541107250 wait complete. 2024-05-09T10:50:20.523 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:20.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.628+0000 7f9b13d63640 1 Processor -- start 2024-05-09T10:50:20.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.628+0000 7f9b13d63640 1 -- start start 2024-05-09T10:50:20.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.629+0000 7f9b13d63640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9b0c15de40 0x7f9b0c0d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:20.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.629+0000 7f9b13d63640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b0c0d0b80 0x7f9b0c0b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:20.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.629+0000 7f9b13d63640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b0c15e220 con 0x7f9b0c0cf250 2024-05-09T10:50:20.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.629+0000 7f9b13d63640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b0c0d1790 con 0x7f9b0c15de40 2024-05-09T10:50:20.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.629+0000 7f9b13d63640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c0d19e0 con 0x7f9b0c0d0b80 2024-05-09T10:50:20.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.629+0000 7f9b112fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f9b0c0cf250 0x7f9b0c0d0470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42308/0 (socket says 172.21.15.98:42308) 2024-05-09T10:50:20.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.629+0000 7f9b112fe640 1 -- 172.21.15.98:0/1631536383 learned_addr learned my addr 172.21.15.98:0/1631536383 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:20.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.629+0000 7f9b12300640 1 --2- 172.21.15.98:0/1631536383 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b0c0d0b80 0x7f9b0c0b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.629+0000 7f9b11aff640 1 --2- 172.21.15.98:0/1631536383 >> v2:172.21.15.105:3300/0 conn(0x7f9b0c15de40 0x7f9b0c0d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.629+0000 7f9b10afd640 1 -- 172.21.15.98:0/1631536383 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2183068280 0 0) 0x7f9b0c15e220 con 0x7f9b0c0cf250 2024-05-09T10:50:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.629+0000 7f9b10afd640 1 -- 172.21.15.98:0/1631536383 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9af8003520 con 0x7f9b0c0cf250 2024-05-09T10:50:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.630+0000 7f9b10afd640 1 -- 172.21.15.98:0/1631536383 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 139986471 0 0) 0x7f9af8003520 con 0x7f9b0c0cf250 2024-05-09T10:50:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.630+0000 7f9b10afd640 1 -- 172.21.15.98:0/1631536383 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b0c0d0b80 msgr2=0x7f9b0c0b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.630+0000 7f9b10afd640 1 --2- 172.21.15.98:0/1631536383 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b0c0d0b80 0x7f9b0c0b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.630+0000 7f9b10afd640 1 -- 172.21.15.98:0/1631536383 >> v2:172.21.15.105:3300/0 conn(0x7f9b0c15de40 msgr2=0x7f9b0c0d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.630+0000 7f9b10afd640 1 --2- 172.21.15.98:0/1631536383 >> v2:172.21.15.105:3300/0 conn(0x7f9b0c15de40 0x7f9b0c0d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.630+0000 7f9b10afd640 1 -- 172.21.15.98:0/1631536383 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0c0d1cc0 con 0x7f9b0c0cf250 2024-05-09T10:50:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.630+0000 7f9b11aff640 1 --2- 172.21.15.98:0/1631536383 >> v2:172.21.15.105:3300/0 conn(0x7f9b0c15de40 0x7f9b0c0d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:50:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.630+0000 7f9b10afd640 1 -- 172.21.15.98:0/1631536383 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f9afc001670 con 0x7f9b0c0cf250 2024-05-09T10:50:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.630+0000 7f9b10afd640 1 -- 172.21.15.98:0/1631536383 >> v1:172.21.15.98:6789/0 conn(0x7f9b0c0cf250 legacy=0x7f9b0c0d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.630+0000 7f9b10afd640 1 --2- 172.21.15.98:0/1631536383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9af80039b0 0x7f9af8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:20.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.630+0000 7f9b10afd640 1 --2- 172.21.15.98:0/1631536383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9af8004800 0x7f9af8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:20.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.630+0000 7f9b10afd640 1 --2- 172.21.15.98:0/1631536383 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b0c0d0b80 0x7f9af80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:20.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.630+0000 7f9b10afd640 1 -- 172.21.15.98:0/1631536383 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c0d1790 con 0x7f9af80039b0 2024-05-09T10:50:20.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.630+0000 7f9b10afd640 1 -- 172.21.15.98:0/1631536383 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c0d19e0 con 0x7f9af8004800 2024-05-09T10:50:20.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.630+0000 7f9b10afd640 1 -- 172.21.15.98:0/1631536383 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af800b9c0 con 0x7f9b0c0d0b80 2024-05-09T10:50:20.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.630+0000 7f9b11aff640 1 --2- 172.21.15.98:0/1631536383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9af80039b0 0x7f9af8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:20.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.630+0000 7f9b112fe640 1 --2- 172.21.15.98:0/1631536383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9af8004800 0x7f9af8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:20.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.630+0000 7f9b12300640 1 --2- 172.21.15.98:0/1631536383 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b0c0d0b80 0x7f9af80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:20.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.630+0000 7f9b11aff640 1 -- 172.21.15.98:0/1631536383 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b0c0d0b80 msgr2=0x7f9af80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:20.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.630+0000 7f9b11aff640 1 --2- 172.21.15.98:0/1631536383 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b0c0d0b80 0x7f9af80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.630+0000 7f9b11aff640 1 -- 172.21.15.98:0/1631536383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9af8004800 msgr2=0x7f9af8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:20.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.630+0000 7f9b11aff640 1 --2- 172.21.15.98:0/1631536383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9af8004800 0x7f9af8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.630+0000 7f9b11aff640 1 -- 172.21.15.98:0/1631536383 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af8003520 con 0x7f9af80039b0 2024-05-09T10:50:20.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.631+0000 7f9b11aff640 1 --2- 172.21.15.98:0/1631536383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9af80039b0 0x7f9af8003dd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009c90 tx=0x7f9af8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa87df9052b00bbc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:20.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.631+0000 7f9b10afd640 1 -- 172.21.15.98:0/1631536383 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b08036070 con 0x7f9af80039b0 2024-05-09T10:50:20.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.631+0000 7f9b10afd640 1 -- 172.21.15.98:0/1631536383 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b08031400 con 0x7f9af80039b0 2024-05-09T10:50:20.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.631+0000 7f9b10afd640 1 -- 172.21.15.98:0/1631536383 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b08009860 con 0x7f9af80039b0 2024-05-09T10:50:20.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.631+0000 7f9b13d63640 1 -- 172.21.15.98:0/1631536383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9af80039b0 msgr2=0x7f9af8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:20.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.631+0000 7f9b13d63640 1 --2- 172.21.15.98:0/1631536383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9af80039b0 0x7f9af8003dd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009c90 tx=0x7f9af8005830 comp rx=0 tx=0).stop 2024-05-09T10:50:20.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.631+0000 7f9b12300640 1 -- 172.21.15.98:0/1631536383 reap_dead start 2024-05-09T10:50:20.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.632+0000 7f9b13d63640 1 -- 172.21.15.98:0/1631536383 shutdown_connections 2024-05-09T10:50:20.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.632+0000 7f9b13d63640 1 -- 172.21.15.98:0/1631536383 >> 172.21.15.98:0/1631536383 conn(0x7f9b0c09ce90 msgr2=0x7f9b0c0d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:20.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.632+0000 7f9b13d63640 1 -- 172.21.15.98:0/1631536383 shutdown_connections 2024-05-09T10:50:20.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.632+0000 7f9b13d63640 1 -- 172.21.15.98:0/1631536383 wait complete. 2024-05-09T10:50:20.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.633+0000 7f9b13d63640 1 Processor -- start 2024-05-09T10:50:20.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.633+0000 7f9b13d63640 1 -- start start 2024-05-09T10:50:20.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.633+0000 7f9b13d63640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9af80039b0 0x7f9b0c15af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:20.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.633+0000 7f9b13d63640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b0c0c9190 0x7f9b0c06be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:20.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.633+0000 7f9b13d63640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b0c06c3b0 0x7f9b0c0b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:20.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.633+0000 7f9b13d63640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c0d4680 con 0x7f9b0c0c9190 2024-05-09T10:50:20.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.633+0000 7f9b13d63640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c0d2280 con 0x7f9b0c06c3b0 2024-05-09T10:50:20.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.633+0000 7f9b13d63640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0c0d4800 con 0x7f9af80039b0 2024-05-09T10:50:20.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.634+0000 7f9b112fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b0c0c9190 0x7f9b0c06be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:20.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.634+0000 7f9b112fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b0c0c9190 0x7f9b0c06be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34354/0 (socket says 172.21.15.98:34354) 2024-05-09T10:50:20.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.634+0000 7f9b112fe640 1 -- 172.21.15.98:0/1275667169 learned_addr learned my addr 172.21.15.98:0/1275667169 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:20.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.634+0000 7f9b11aff640 1 --2- 172.21.15.98:0/1275667169 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9af80039b0 0x7f9b0c15af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:20.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.634+0000 7f9b12300640 1 --2- 172.21.15.98:0/1275667169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b0c06c3b0 0x7f9b0c0b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:20.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.634+0000 7f9b112fe640 1 -- 172.21.15.98:0/1275667169 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9af80039b0 msgr2=0x7f9b0c15af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:20.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.634+0000 7f9b112fe640 1 --2- 172.21.15.98:0/1275667169 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9af80039b0 0x7f9b0c15af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.634+0000 7f9b112fe640 1 -- 172.21.15.98:0/1275667169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b0c06c3b0 msgr2=0x7f9b0c0b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:20.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.634+0000 7f9b112fe640 1 --2- 172.21.15.98:0/1275667169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b0c06c3b0 0x7f9b0c0b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.634+0000 7f9b112fe640 1 -- 172.21.15.98:0/1275667169 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0c15b4c0 con 0x7f9b0c0c9190 2024-05-09T10:50:20.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.634+0000 7f9b112fe640 1 --2- 172.21.15.98:0/1275667169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b0c0c9190 0x7f9b0c06be70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9afc000fb0 tx=0x7f9afc0077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:20.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.634+0000 7f9b027fc640 1 -- 172.21.15.98:0/1275667169 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc00f930 con 0x7f9b0c0c9190 2024-05-09T10:50:20.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.634+0000 7f9b027fc640 1 -- 172.21.15.98:0/1275667169 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9afc018070 con 0x7f9b0c0c9190 2024-05-09T10:50:20.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.634+0000 7f9b13d63640 1 -- 172.21.15.98:0/1275667169 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b0c1f3b80 con 0x7f9b0c0c9190 2024-05-09T10:50:20.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.634+0000 7f9b027fc640 1 -- 172.21.15.98:0/1275667169 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc00fd30 con 0x7f9b0c0c9190 2024-05-09T10:50:20.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.634+0000 7f9b13d63640 1 -- 172.21.15.98:0/1275667169 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b0c1f4140 con 0x7f9b0c0c9190 2024-05-09T10:50:20.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.635+0000 7f9b13d63640 1 -- 172.21.15.98:0/1275667169 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b0c069cb0 con 0x7f9b0c0c9190 2024-05-09T10:50:20.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.635+0000 7f9b027fc640 1 -- 172.21.15.98:0/1275667169 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9afc01d460 con 0x7f9b0c0c9190 2024-05-09T10:50:20.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.636+0000 7f9b027fc640 1 --2- 172.21.15.98:0/1275667169 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9af006b500 0x7f9af006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:20.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.636+0000 7f9b027fc640 1 -- 172.21.15.98:0/1275667169 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9afc08a1c0 con 0x7f9b0c0c9190 2024-05-09T10:50:20.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.636+0000 7f9b11aff640 1 --2- 172.21.15.98:0/1275667169 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9af006b500 0x7f9af006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:20.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.636+0000 7f9b11aff640 1 --2- 172.21.15.98:0/1275667169 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9af006b500 0x7f9af006d9c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9b08033000 tx=0x7f9b08008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:20.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.640+0000 7f9b027fc640 1 -- 172.21.15.98:0/1275667169 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9b0c15b4c0 con 0x7f9b0c0c9190 2024-05-09T10:50:20.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.809+0000 7f9b13d63640 1 -- 172.21.15.98:0/1275667169 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b0c0d42d0 con 0x7f9b0c0c9190 2024-05-09T10:50:20.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.810+0000 7f9b027fc640 1 -- 172.21.15.98:0/1275667169 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16618 (secure 0 0 0) 0x7f9afc059040 con 0x7f9b0c0c9190 2024-05-09T10:50:20.804 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:20.804 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":19,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:19.544125+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:50:17.323444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T10:50:18.482315+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:20.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.812+0000 7f9b13d63640 1 -- 172.21.15.98:0/1275667169 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9af006b500 msgr2=0x7f9af006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:20.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.812+0000 7f9b13d63640 1 --2- 172.21.15.98:0/1275667169 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9af006b500 0x7f9af006d9c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9b08033000 tx=0x7f9b08008030 comp rx=0 tx=0).stop 2024-05-09T10:50:20.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.812+0000 7f9b13d63640 1 -- 172.21.15.98:0/1275667169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b0c0c9190 msgr2=0x7f9b0c06be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:20.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.812+0000 7f9b13d63640 1 --2- 172.21.15.98:0/1275667169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b0c0c9190 0x7f9b0c06be70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9afc000fb0 tx=0x7f9afc0077a0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.812+0000 7f9b13d63640 1 -- 172.21.15.98:0/1275667169 shutdown_connections 2024-05-09T10:50:20.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.812+0000 7f9b13d63640 1 --2- 172.21.15.98:0/1275667169 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9af006b500 0x7f9af006d9c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.812+0000 7f9b13d63640 1 --2- 172.21.15.98:0/1275667169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b0c06c3b0 0x7f9b0c0b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.812+0000 7f9b13d63640 1 --2- 172.21.15.98:0/1275667169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b0c0c9190 0x7f9b0c06be70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.812+0000 7f9b13d63640 1 --2- 172.21.15.98:0/1275667169 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9af80039b0 0x7f9b0c15af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.812+0000 7f9b13d63640 1 -- 172.21.15.98:0/1275667169 >> 172.21.15.98:0/1275667169 conn(0x7f9b0c09ce90 msgr2=0x7f9b0c0b7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:20.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.812+0000 7f9b13d63640 1 -- 172.21.15.98:0/1275667169 shutdown_connections 2024-05-09T10:50:20.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.812+0000 7f9b13d63640 1 -- 172.21.15.98:0/1275667169 wait complete. 2024-05-09T10:50:20.817 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-09T10:50:20.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.926+0000 7fe94ad69640 1 Processor -- start 2024-05-09T10:50:20.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.926+0000 7fe94ad69640 1 -- start start 2024-05-09T10:50:20.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.927+0000 7fe94ad69640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe94415de50 0x7fe9440d42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:20.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.927+0000 7fe94ad69640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9440cf300 con 0x7fe9440d0c90 2024-05-09T10:50:20.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.927+0000 7fe94ad69640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9440cf5e0 con 0x7fe9440b2a00 2024-05-09T10:50:20.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.927+0000 7fe94ad69640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9440b34a0 con 0x7fe94415de50 2024-05-09T10:50:20.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.927+0000 7fe948b05640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe94415de50 0x7fe9440d42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:20.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.927+0000 7fe949306640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe9440d0c90 0x7fe9440b3930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42312/0 (socket says 172.21.15.98:42312) 2024-05-09T10:50:20.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.927+0000 7fe949306640 1 -- 172.21.15.98:0/2073741481 learned_addr learned my addr 172.21.15.98:0/2073741481 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:20.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.927+0000 7fe948b05640 1 -- 172.21.15.98:0/2073741481 >> v1:172.21.15.105:6789/0 conn(0x7fe9440b2a00 legacy=0x7fe9440d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:20.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.927+0000 7fe948b05640 1 -- 172.21.15.98:0/2073741481 >> v1:172.21.15.98:6789/0 conn(0x7fe9440d0c90 legacy=0x7fe9440b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:20.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.927+0000 7fe948b05640 1 -- 172.21.15.98:0/2073741481 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9440cf8c0 con 0x7fe94415de50 2024-05-09T10:50:20.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.927+0000 7fe9437fe640 1 -- 172.21.15.98:0/2073741481 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1293700968 0 0) 0x7fe9440cf300 con 0x7fe9440d0c90 2024-05-09T10:50:20.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.928+0000 7fe948b05640 1 --2- 172.21.15.98:0/2073741481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe94415de50 0x7fe9440d42c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe9440b3620 tx=0x7fe93802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf63befd7d9452ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:20.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.928+0000 7fe9437fe640 1 -- 172.21.15.98:0/2073741481 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe938037070 con 0x7fe94415de50 2024-05-09T10:50:20.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.928+0000 7fe9437fe640 1 -- 172.21.15.98:0/2073741481 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93802ae40 con 0x7fe94415de50 2024-05-09T10:50:20.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.928+0000 7fe9437fe640 1 -- 172.21.15.98:0/2073741481 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe938032660 con 0x7fe94415de50 2024-05-09T10:50:20.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.928+0000 7fe94ad69640 1 -- 172.21.15.98:0/2073741481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe94415de50 msgr2=0x7fe9440d42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:20.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.928+0000 7fe94ad69640 1 --2- 172.21.15.98:0/2073741481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe94415de50 0x7fe9440d42c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe9440b3620 tx=0x7fe93802a130 comp rx=0 tx=0).stop 2024-05-09T10:50:20.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.928+0000 7fe94ad69640 1 -- 172.21.15.98:0/2073741481 shutdown_connections 2024-05-09T10:50:20.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.928+0000 7fe94ad69640 1 --2- 172.21.15.98:0/2073741481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe94415de50 0x7fe9440d42c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.928+0000 7fe94ad69640 1 -- 172.21.15.98:0/2073741481 >> 172.21.15.98:0/2073741481 conn(0x7fe94409cef0 msgr2=0x7fe9440d3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:20.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.928+0000 7fe94ad69640 1 -- 172.21.15.98:0/2073741481 shutdown_connections 2024-05-09T10:50:20.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.928+0000 7fe94ad69640 1 -- 172.21.15.98:0/2073741481 wait complete. 2024-05-09T10:50:20.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.930+0000 7fe94ad69640 1 Processor -- start 2024-05-09T10:50:20.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.930+0000 7fe94ad69640 1 -- start start 2024-05-09T10:50:20.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe94ad69640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9440b2a00 0x7fe9441f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:20.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe94ad69640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9440d0c90 0x7fe9440ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:20.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe94ad69640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe94415de50 0x7fe944157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:20.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe94ad69640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9440b4040 con 0x7fe9440d0c90 2024-05-09T10:50:20.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe94ad69640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94415e230 con 0x7fe94415de50 2024-05-09T10:50:20.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe94ad69640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9440b41c0 con 0x7fe9440b2a00 2024-05-09T10:50:20.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe943fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9440d0c90 0x7fe9440ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:20.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe943fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9440d0c90 0x7fe9440ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34378/0 (socket says 172.21.15.98:34378) 2024-05-09T10:50:20.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe943fff640 1 -- 172.21.15.98:0/787221213 learned_addr learned my addr 172.21.15.98:0/787221213 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:20.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe948b05640 1 --2- 172.21.15.98:0/787221213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9440b2a00 0x7fe9441f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:20.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe949306640 1 --2- 172.21.15.98:0/787221213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe94415de50 0x7fe944157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:20.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe943fff640 1 -- 172.21.15.98:0/787221213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9440b2a00 msgr2=0x7fe9441f6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:20.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe943fff640 1 --2- 172.21.15.98:0/787221213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9440b2a00 0x7fe9441f6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe943fff640 1 -- 172.21.15.98:0/787221213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe94415de50 msgr2=0x7fe944157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:20.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe943fff640 1 --2- 172.21.15.98:0/787221213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe94415de50 0x7fe944157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:20.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe943fff640 1 -- 172.21.15.98:0/787221213 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94415e530 con 0x7fe9440d0c90 2024-05-09T10:50:20.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe948b05640 1 --2- 172.21.15.98:0/787221213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9440b2a00 0x7fe9441f6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:20.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe949306640 1 --2- 172.21.15.98:0/787221213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe94415de50 0x7fe944157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:20.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe943fff640 1 --2- 172.21.15.98:0/787221213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9440d0c90 0x7fe9440ccf00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe93000c440 tx=0x7fe93000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:20.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe9417fa640 1 -- 172.21.15.98:0/787221213 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe930017070 con 0x7fe9440d0c90 2024-05-09T10:50:20.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe9417fa640 1 -- 172.21.15.98:0/787221213 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9300035f0 con 0x7fe9440d0c90 2024-05-09T10:50:20.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe94ad69640 1 -- 172.21.15.98:0/787221213 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9441f7640 con 0x7fe9440d0c90 2024-05-09T10:50:20.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe9417fa640 1 -- 172.21.15.98:0/787221213 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9300039e0 con 0x7fe9440d0c90 2024-05-09T10:50:20.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.931+0000 7fe94ad69640 1 -- 172.21.15.98:0/787221213 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9441f7c00 con 0x7fe9440d0c90 2024-05-09T10:50:20.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.932+0000 7fe94ad69640 1 -- 172.21.15.98:0/787221213 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9440a1b60 con 0x7fe9440d0c90 2024-05-09T10:50:20.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.933+0000 7fe9417fa640 1 -- 172.21.15.98:0/787221213 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe93001a070 con 0x7fe9440d0c90 2024-05-09T10:50:20.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.933+0000 7fe9417fa640 1 --2- 172.21.15.98:0/787221213 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe92806b4d0 0x7fe92806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:20.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.933+0000 7fe9417fa640 1 -- 172.21.15.98:0/787221213 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9300894f0 con 0x7fe9440d0c90 2024-05-09T10:50:20.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.933+0000 7fe948b05640 1 --2- 172.21.15.98:0/787221213 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe92806b4d0 0x7fe92806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:20.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.934+0000 7fe948b05640 1 --2- 172.21.15.98:0/787221213 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe92806b4d0 0x7fe92806d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe938034000 tx=0x7fe93802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:20.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:20.937+0000 7fe9417fa640 1 -- 172.21.15.98:0/787221213 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe930058400 con 0x7fe9440d0c90 2024-05-09T10:50:21.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.144+0000 7fe94ad69640 1 -- 172.21.15.98:0/787221213 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe94415e700 con 0x7fe9440d0c90 2024-05-09T10:50:21.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.176+0000 7fe9417fa640 1 -- 172.21.15.98:0/787221213 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe9300010c0 con 0x7fe9440d0c90 2024-05-09T10:50:21.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.178+0000 7fe94ad69640 1 -- 172.21.15.98:0/787221213 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe92806b4d0 msgr2=0x7fe92806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:21.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.178+0000 7fe94ad69640 1 --2- 172.21.15.98:0/787221213 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe92806b4d0 0x7fe92806d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe938034000 tx=0x7fe93802e000 comp rx=0 tx=0).stop 2024-05-09T10:50:21.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.178+0000 7fe94ad69640 1 -- 172.21.15.98:0/787221213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9440d0c90 msgr2=0x7fe9440ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:21.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.178+0000 7fe94ad69640 1 --2- 172.21.15.98:0/787221213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9440d0c90 0x7fe9440ccf00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe93000c440 tx=0x7fe93000c900 comp rx=0 tx=0).stop 2024-05-09T10:50:21.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.178+0000 7fe94ad69640 1 -- 172.21.15.98:0/787221213 shutdown_connections 2024-05-09T10:50:21.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.178+0000 7fe94ad69640 1 --2- 172.21.15.98:0/787221213 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe92806b4d0 0x7fe92806d990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:21.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.178+0000 7fe94ad69640 1 --2- 172.21.15.98:0/787221213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe94415de50 0x7fe944157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:21.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.178+0000 7fe94ad69640 1 --2- 172.21.15.98:0/787221213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9440d0c90 0x7fe9440ccf00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:21.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.178+0000 7fe94ad69640 1 --2- 172.21.15.98:0/787221213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9440b2a00 0x7fe9441f6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:21.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.178+0000 7fe94ad69640 1 -- 172.21.15.98:0/787221213 >> 172.21.15.98:0/787221213 conn(0x7fe94409cef0 msgr2=0x7fe9440b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:21.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.178+0000 7fe94ad69640 1 -- 172.21.15.98:0/787221213 shutdown_connections 2024-05-09T10:50:21.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.178+0000 7fe94ad69640 1 -- 172.21.15.98:0/787221213 wait complete. 2024-05-09T10:50:21.184 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-09T10:50:21.184 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-09T10:50:21.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.292+0000 7f6e838ac640 1 Processor -- start 2024-05-09T10:50:21.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.292+0000 7f6e838ac640 1 -- start start 2024-05-09T10:50:21.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e838ac640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6e7c0cfbb0 0x7f6e7c161b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:21.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e838ac640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e7c162070 0x7f6e7c15d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:21.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e838ac640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7c162470 con 0x7f6e7c0b2d40 2024-05-09T10:50:21.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e838ac640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e7c0d0090 con 0x7f6e7c0cfbb0 2024-05-09T10:50:21.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e838ac640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0d0370 con 0x7f6e7c162070 2024-05-09T10:50:21.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e81648640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f6e7c0b2d40 0x7f6e7c06c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42324/0 (socket says 172.21.15.98:42324) 2024-05-09T10:50:21.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e81648640 1 -- 172.21.15.98:0/2798131275 learned_addr learned my addr 172.21.15.98:0/2798131275 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:21.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e73fff640 1 -- 172.21.15.98:0/2798131275 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3849667620 0 0) 0x7f6e7c162470 con 0x7f6e7c0b2d40 2024-05-09T10:50:21.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e73fff640 1 -- 172.21.15.98:0/2798131275 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e68003520 con 0x7f6e7c0b2d40 2024-05-09T10:50:21.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e80e47640 1 --2- 172.21.15.98:0/2798131275 >> v2:172.21.15.105:3300/0 conn(0x7f6e7c0cfbb0 0x7f6e7c161b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:21.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e81e49640 1 --2- 172.21.15.98:0/2798131275 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e7c162070 0x7f6e7c15d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:21.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e73fff640 1 -- 172.21.15.98:0/2798131275 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3406679341 0 0) 0x7f6e68003520 con 0x7f6e7c0b2d40 2024-05-09T10:50:21.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e73fff640 1 -- 172.21.15.98:0/2798131275 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e7c162070 msgr2=0x7f6e7c15d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:21.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e73fff640 1 --2- 172.21.15.98:0/2798131275 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e7c162070 0x7f6e7c15d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:21.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e73fff640 1 -- 172.21.15.98:0/2798131275 >> v2:172.21.15.105:3300/0 conn(0x7f6e7c0cfbb0 msgr2=0x7f6e7c161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:21.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e73fff640 1 --2- 172.21.15.98:0/2798131275 >> v2:172.21.15.105:3300/0 conn(0x7f6e7c0cfbb0 0x7f6e7c161b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:21.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e73fff640 1 -- 172.21.15.98:0/2798131275 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c0d0650 con 0x7f6e7c0b2d40 2024-05-09T10:50:21.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e73fff640 1 -- 172.21.15.98:0/2798131275 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f6e78001670 con 0x7f6e7c0b2d40 2024-05-09T10:50:21.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e81e49640 1 --2- 172.21.15.98:0/2798131275 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e7c162070 0x7f6e7c15d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:21.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e73fff640 1 -- 172.21.15.98:0/2798131275 >> v1:172.21.15.98:6789/0 conn(0x7f6e7c0b2d40 legacy=0x7f6e7c06c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:21.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e73fff640 1 --2- 172.21.15.98:0/2798131275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e680039b0 0x7f6e68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:21.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e73fff640 1 --2- 172.21.15.98:0/2798131275 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e7c162070 0x7f6e68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:21.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e73fff640 1 --2- 172.21.15.98:0/2798131275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e68004d40 0x7f6e680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:21.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e73fff640 1 -- 172.21.15.98:0/2798131275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0d0090 con 0x7f6e68004d40 2024-05-09T10:50:21.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e73fff640 1 -- 172.21.15.98:0/2798131275 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0d0370 con 0x7f6e680039b0 2024-05-09T10:50:21.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e73fff640 1 -- 172.21.15.98:0/2798131275 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6800ba10 con 0x7f6e7c162070 2024-05-09T10:50:21.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e81e49640 1 --2- 172.21.15.98:0/2798131275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e68004d40 0x7f6e680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:21.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e80e47640 1 --2- 172.21.15.98:0/2798131275 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e7c162070 0x7f6e68004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:21.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.293+0000 7f6e81648640 1 --2- 172.21.15.98:0/2798131275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e680039b0 0x7f6e68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:21.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.294+0000 7f6e81e49640 1 -- 172.21.15.98:0/2798131275 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e7c162070 msgr2=0x7f6e68004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:21.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.294+0000 7f6e81e49640 1 --2- 172.21.15.98:0/2798131275 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e7c162070 0x7f6e68004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:21.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.294+0000 7f6e81e49640 1 -- 172.21.15.98:0/2798131275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e680039b0 msgr2=0x7f6e68003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:21.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.294+0000 7f6e81e49640 1 --2- 172.21.15.98:0/2798131275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e680039b0 0x7f6e68003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:21.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.294+0000 7f6e81e49640 1 -- 172.21.15.98:0/2798131275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e68003520 con 0x7f6e68004d40 2024-05-09T10:50:21.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.294+0000 7f6e81e49640 1 --2- 172.21.15.98:0/2798131275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e68004d40 0x7f6e680051c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6e740073e0 tx=0x7f6e68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca24bb153a71b6d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:21.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.295+0000 7f6e73fff640 1 -- 172.21.15.98:0/2798131275 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74009c80 con 0x7f6e68004d40 2024-05-09T10:50:21.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.295+0000 7f6e73fff640 1 -- 172.21.15.98:0/2798131275 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7402e660 con 0x7f6e68004d40 2024-05-09T10:50:21.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.295+0000 7f6e73fff640 1 -- 172.21.15.98:0/2798131275 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7402ea10 con 0x7f6e68004d40 2024-05-09T10:50:21.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.295+0000 7f6e838ac640 1 -- 172.21.15.98:0/2798131275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e68004d40 msgr2=0x7f6e680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:21.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.295+0000 7f6e838ac640 1 --2- 172.21.15.98:0/2798131275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e68004d40 0x7f6e680051c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6e740073e0 tx=0x7f6e68005830 comp rx=0 tx=0).stop 2024-05-09T10:50:21.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.295+0000 7f6e81e49640 1 -- 172.21.15.98:0/2798131275 reap_dead start 2024-05-09T10:50:21.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.295+0000 7f6e838ac640 1 -- 172.21.15.98:0/2798131275 shutdown_connections 2024-05-09T10:50:21.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.295+0000 7f6e838ac640 1 -- 172.21.15.98:0/2798131275 >> 172.21.15.98:0/2798131275 conn(0x7f6e7c0d1970 msgr2=0x7f6e7c0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:21.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.295+0000 7f6e838ac640 1 -- 172.21.15.98:0/2798131275 shutdown_connections 2024-05-09T10:50:21.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.295+0000 7f6e838ac640 1 -- 172.21.15.98:0/2798131275 wait complete. 2024-05-09T10:50:21.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.296+0000 7f6e838ac640 1 Processor -- start 2024-05-09T10:50:21.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.296+0000 7f6e838ac640 1 -- start start 2024-05-09T10:50:21.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.297+0000 7f6e838ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e68004d40 0x7f6e7c0b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:21.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.297+0000 7f6e838ac640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e7c147e50 0x7f6e7c0c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:21.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.297+0000 7f6e838ac640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e7c1482d0 0x7f6e7c0c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:21.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.297+0000 7f6e838ac640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c06d020 con 0x7f6e7c1482d0 2024-05-09T10:50:21.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.297+0000 7f6e838ac640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0d0c10 con 0x7f6e68004d40 2024-05-09T10:50:21.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.297+0000 7f6e838ac640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0b3120 con 0x7f6e7c147e50 2024-05-09T10:50:21.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.297+0000 7f6e81e49640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e7c1482d0 0x7f6e7c0c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:21.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.297+0000 7f6e81e49640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e7c1482d0 0x7f6e7c0c71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34386/0 (socket says 172.21.15.98:34386) 2024-05-09T10:50:21.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.297+0000 7f6e81e49640 1 -- 172.21.15.98:0/170002282 learned_addr learned my addr 172.21.15.98:0/170002282 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:21.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.297+0000 7f6e81648640 1 --2- 172.21.15.98:0/170002282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e68004d40 0x7f6e7c0b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:21.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.297+0000 7f6e80e47640 1 --2- 172.21.15.98:0/170002282 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e7c147e50 0x7f6e7c0c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:21.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.297+0000 7f6e81e49640 1 -- 172.21.15.98:0/170002282 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e7c147e50 msgr2=0x7f6e7c0c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:21.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.297+0000 7f6e81e49640 1 --2- 172.21.15.98:0/170002282 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e7c147e50 0x7f6e7c0c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:21.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.297+0000 7f6e81e49640 1 -- 172.21.15.98:0/170002282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e68004d40 msgr2=0x7f6e7c0b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:21.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.297+0000 7f6e81e49640 1 --2- 172.21.15.98:0/170002282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e68004d40 0x7f6e7c0b1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:21.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.297+0000 7f6e81e49640 1 -- 172.21.15.98:0/170002282 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c0b0390 con 0x7f6e7c1482d0 2024-05-09T10:50:21.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.298+0000 7f6e81e49640 1 --2- 172.21.15.98:0/170002282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e7c1482d0 0x7f6e7c0c71e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6e74007500 tx=0x7f6e74004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:21.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.298+0000 7f6e71ffb640 1 -- 172.21.15.98:0/170002282 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74009c80 con 0x7f6e7c1482d0 2024-05-09T10:50:21.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.298+0000 7f6e71ffb640 1 -- 172.21.15.98:0/170002282 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e74045070 con 0x7f6e7c1482d0 2024-05-09T10:50:21.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.298+0000 7f6e838ac640 1 -- 172.21.15.98:0/170002282 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7c0b0670 con 0x7f6e7c1482d0 2024-05-09T10:50:21.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.298+0000 7f6e71ffb640 1 -- 172.21.15.98:0/170002282 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74036a40 con 0x7f6e7c1482d0 2024-05-09T10:50:21.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.298+0000 7f6e838ac640 1 -- 172.21.15.98:0/170002282 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7c0b0c30 con 0x7f6e7c1482d0 2024-05-09T10:50:21.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.299+0000 7f6e838ac640 1 -- 172.21.15.98:0/170002282 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e7c06ba50 con 0x7f6e7c1482d0 2024-05-09T10:50:21.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.300+0000 7f6e71ffb640 1 -- 172.21.15.98:0/170002282 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e74036be0 con 0x7f6e7c1482d0 2024-05-09T10:50:21.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.300+0000 7f6e71ffb640 1 --2- 172.21.15.98:0/170002282 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6e6006b4d0 0x7f6e6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:21.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.300+0000 7f6e71ffb640 1 -- 172.21.15.98:0/170002282 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e740ab1b0 con 0x7f6e7c1482d0 2024-05-09T10:50:21.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.300+0000 7f6e81648640 1 --2- 172.21.15.98:0/170002282 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6e6006b4d0 0x7f6e6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:21.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.300+0000 7f6e81648640 1 --2- 172.21.15.98:0/170002282 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6e6006b4d0 0x7f6e6006d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6e78000c00 tx=0x7f6e780078f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:21.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.304+0000 7f6e71ffb640 1 -- 172.21.15.98:0/170002282 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e7403d070 con 0x7f6e7c1482d0 2024-05-09T10:50:21.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:21.511+0000 7f6e838ac640 1 -- 172.21.15.98:0/170002282 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6e7c09fa40 con 0x7f6e7c1482d0 2024-05-09T10:50:22.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.184+0000 7f6e71ffb640 1 -- 172.21.15.98:0/170002282 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6e74032030 con 0x7f6e7c1482d0 2024-05-09T10:50:22.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.187+0000 7f6e838ac640 1 -- 172.21.15.98:0/170002282 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6e6006b4d0 msgr2=0x7f6e6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:22.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.187+0000 7f6e838ac640 1 --2- 172.21.15.98:0/170002282 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6e6006b4d0 0x7f6e6006d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6e78000c00 tx=0x7f6e780078f0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.187+0000 7f6e838ac640 1 -- 172.21.15.98:0/170002282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e7c1482d0 msgr2=0x7f6e7c0c71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:22.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.187+0000 7f6e838ac640 1 --2- 172.21.15.98:0/170002282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e7c1482d0 0x7f6e7c0c71e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6e74007500 tx=0x7f6e74004ab0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.187+0000 7f6e838ac640 1 -- 172.21.15.98:0/170002282 shutdown_connections 2024-05-09T10:50:22.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.187+0000 7f6e838ac640 1 --2- 172.21.15.98:0/170002282 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6e6006b4d0 0x7f6e6006d990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.187+0000 7f6e838ac640 1 --2- 172.21.15.98:0/170002282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e7c1482d0 0x7f6e7c0c71e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.187+0000 7f6e838ac640 1 --2- 172.21.15.98:0/170002282 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6e7c147e50 0x7f6e7c0c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.187+0000 7f6e838ac640 1 --2- 172.21.15.98:0/170002282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e68004d40 0x7f6e7c0b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.187+0000 7f6e838ac640 1 -- 172.21.15.98:0/170002282 >> 172.21.15.98:0/170002282 conn(0x7f6e7c0d1970 msgr2=0x7f6e7c159eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:22.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.187+0000 7f6e838ac640 1 -- 172.21.15.98:0/170002282 shutdown_connections 2024-05-09T10:50:22.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.187+0000 7f6e838ac640 1 -- 172.21.15.98:0/170002282 wait complete. 2024-05-09T10:50:22.192 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-09T10:50:22.192 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T10:50:22.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.300+0000 7fbeb29b0640 1 Processor -- start 2024-05-09T10:50:22.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.300+0000 7fbeb29b0640 1 -- start start 2024-05-09T10:50:22.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.301+0000 7fbeb29b0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbeac0d0b80 0x7fbeac0b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:22.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.301+0000 7fbeb29b0640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeac0d2c30 con 0x7fbeac15de40 2024-05-09T10:50:22.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.301+0000 7fbeb29b0640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeac15e220 con 0x7fbeac0d27a0 2024-05-09T10:50:22.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.301+0000 7fbeb29b0640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac0d1790 con 0x7fbeac0d0b80 2024-05-09T10:50:22.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.301+0000 7fbeabfff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fbeac15de40 0x7fbeac0cf250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42340/0 (socket says 172.21.15.98:42340) 2024-05-09T10:50:22.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.301+0000 7fbeabfff640 1 -- 172.21.15.98:0/3068449846 learned_addr learned my addr 172.21.15.98:0/3068449846 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:22.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.301+0000 7fbeab7fe640 1 -- 172.21.15.98:0/3068449846 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3006804185 0 0) 0x7fbeac0d2c30 con 0x7fbeac15de40 2024-05-09T10:50:22.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.301+0000 7fbeb0f4d640 1 --2- 172.21.15.98:0/3068449846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbeac0d0b80 0x7fbeac0b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:22.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.301+0000 7fbeab7fe640 1 -- 172.21.15.98:0/3068449846 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe98003520 con 0x7fbeac15de40 2024-05-09T10:50:22.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.302+0000 7fbeab7fe640 1 -- 172.21.15.98:0/3068449846 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4273922532 0 0) 0x7fbe98003520 con 0x7fbeac15de40 2024-05-09T10:50:22.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.302+0000 7fbeab7fe640 1 -- 172.21.15.98:0/3068449846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbeac0d0b80 msgr2=0x7fbeac0b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:22.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.302+0000 7fbeab7fe640 1 --2- 172.21.15.98:0/3068449846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbeac0d0b80 0x7fbeac0b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.302+0000 7fbeab7fe640 1 -- 172.21.15.98:0/3068449846 >> v1:172.21.15.105:6789/0 conn(0x7fbeac0d27a0 legacy=0x7fbeac0d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:22.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.302+0000 7fbeab7fe640 1 -- 172.21.15.98:0/3068449846 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac0d19e0 con 0x7fbeac15de40 2024-05-09T10:50:22.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.302+0000 7fbeab7fe640 1 -- 172.21.15.98:0/3068449846 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7fbe9c001670 con 0x7fbeac15de40 2024-05-09T10:50:22.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.302+0000 7fbeab7fe640 1 -- 172.21.15.98:0/3068449846 >> v1:172.21.15.98:6789/0 conn(0x7fbeac15de40 legacy=0x7fbeac0cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:22.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.302+0000 7fbeab7fe640 1 --2- 172.21.15.98:0/3068449846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe980039e0 0x7fbe98003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:22.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.302+0000 7fbeab7fe640 1 --2- 172.21.15.98:0/3068449846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbeac0d0b80 0x7fbe98004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:22.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.302+0000 7fbeab7fe640 1 --2- 172.21.15.98:0/3068449846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe98004d40 0x7fbe980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:22.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.302+0000 7fbeab7fe640 1 -- 172.21.15.98:0/3068449846 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac0d1790 con 0x7fbe98004d40 2024-05-09T10:50:22.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.302+0000 7fbeab7fe640 1 -- 172.21.15.98:0/3068449846 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9800baa0 con 0x7fbe980039e0 2024-05-09T10:50:22.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.302+0000 7fbeab7fe640 1 -- 172.21.15.98:0/3068449846 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9800bca0 con 0x7fbeac0d0b80 2024-05-09T10:50:22.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.302+0000 7fbeb0f4d640 1 --2- 172.21.15.98:0/3068449846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe98004d40 0x7fbe980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:22.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.302+0000 7fbeabfff640 1 --2- 172.21.15.98:0/3068449846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe980039e0 0x7fbe98003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.302+0000 7fbea3fff640 1 --2- 172.21.15.98:0/3068449846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbeac0d0b80 0x7fbe98004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.302+0000 7fbeb0f4d640 1 -- 172.21.15.98:0/3068449846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbeac0d0b80 msgr2=0x7fbe98004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.302+0000 7fbeb0f4d640 1 --2- 172.21.15.98:0/3068449846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbeac0d0b80 0x7fbe98004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.302+0000 7fbeb0f4d640 1 -- 172.21.15.98:0/3068449846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe980039e0 msgr2=0x7fbe98003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.302+0000 7fbeb0f4d640 1 --2- 172.21.15.98:0/3068449846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe980039e0 0x7fbe98003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.302+0000 7fbeb0f4d640 1 -- 172.21.15.98:0/3068449846 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe98003520 con 0x7fbe98004d40 2024-05-09T10:50:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.303+0000 7fbeb0f4d640 1 --2- 172.21.15.98:0/3068449846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe98004d40 0x7fbe980051c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbeac1629f0 tx=0x7fbe98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e6ab2c2046ac69 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.303+0000 7fbeab7fe640 1 -- 172.21.15.98:0/3068449846 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4037070 con 0x7fbe98004d40 2024-05-09T10:50:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.303+0000 7fbeab7fe640 1 -- 172.21.15.98:0/3068449846 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea4032600 con 0x7fbe98004d40 2024-05-09T10:50:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.303+0000 7fbeab7fe640 1 -- 172.21.15.98:0/3068449846 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4009c10 con 0x7fbe98004d40 2024-05-09T10:50:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.303+0000 7fbeb29b0640 1 -- 172.21.15.98:0/3068449846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe98004d40 msgr2=0x7fbe980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.303+0000 7fbeb29b0640 1 --2- 172.21.15.98:0/3068449846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe98004d40 0x7fbe980051c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbeac1629f0 tx=0x7fbe98005830 comp rx=0 tx=0).stop 2024-05-09T10:50:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.303+0000 7fbeb0f4d640 1 -- 172.21.15.98:0/3068449846 reap_dead start 2024-05-09T10:50:22.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.304+0000 7fbeb29b0640 1 -- 172.21.15.98:0/3068449846 shutdown_connections 2024-05-09T10:50:22.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.304+0000 7fbeb29b0640 1 -- 172.21.15.98:0/3068449846 >> 172.21.15.98:0/3068449846 conn(0x7fbeac09ce90 msgr2=0x7fbeac0d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:22.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.304+0000 7fbeb29b0640 1 -- 172.21.15.98:0/3068449846 shutdown_connections 2024-05-09T10:50:22.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.304+0000 7fbeb29b0640 1 -- 172.21.15.98:0/3068449846 wait complete. 2024-05-09T10:50:22.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.305+0000 7fbeb29b0640 1 Processor -- start 2024-05-09T10:50:22.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.305+0000 7fbeb29b0640 1 -- start start 2024-05-09T10:50:22.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.305+0000 7fbeb29b0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe98004d40 0x7fbeac1582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:22.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.305+0000 7fbeb29b0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbeac155fb0 0x7fbeac155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.305+0000 7fbeb29b0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbeac0cccf0 0x7fbeac0cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.305+0000 7fbeb29b0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac0d2280 con 0x7fbe98004d40 2024-05-09T10:50:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.305+0000 7fbeb29b0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac0d1fa0 con 0x7fbeac0cccf0 2024-05-09T10:50:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.305+0000 7fbeb29b0640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac0cf960 con 0x7fbeac155fb0 2024-05-09T10:50:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.305+0000 7fbeabfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe98004d40 0x7fbeac1582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.306+0000 7fbeabfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe98004d40 0x7fbeac1582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34398/0 (socket says 172.21.15.98:34398) 2024-05-09T10:50:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.306+0000 7fbeabfff640 1 -- 172.21.15.98:0/2559908433 learned_addr learned my addr 172.21.15.98:0/2559908433 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.306+0000 7fbea3fff640 1 --2- 172.21.15.98:0/2559908433 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbeac155fb0 0x7fbeac155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.306+0000 7fbeb0f4d640 1 --2- 172.21.15.98:0/2559908433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbeac0cccf0 0x7fbeac0cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.306+0000 7fbeabfff640 1 -- 172.21.15.98:0/2559908433 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbeac155fb0 msgr2=0x7fbeac155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.306+0000 7fbeabfff640 1 --2- 172.21.15.98:0/2559908433 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbeac155fb0 0x7fbeac155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.306+0000 7fbeabfff640 1 -- 172.21.15.98:0/2559908433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbeac0cccf0 msgr2=0x7fbeac0cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.306+0000 7fbeabfff640 1 --2- 172.21.15.98:0/2559908433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbeac0cccf0 0x7fbeac0cd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.306+0000 7fbeabfff640 1 -- 172.21.15.98:0/2559908433 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac0c7750 con 0x7fbe98004d40 2024-05-09T10:50:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.306+0000 7fbeb0f4d640 1 --2- 172.21.15.98:0/2559908433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbeac0cccf0 0x7fbeac0cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.306+0000 7fbea3fff640 1 --2- 172.21.15.98:0/2559908433 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbeac155fb0 0x7fbeac155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.306+0000 7fbeabfff640 1 --2- 172.21.15.98:0/2559908433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe98004d40 0x7fbeac1582c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c000f60 tx=0x7fbe9c007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.306+0000 7fbea97fa640 1 -- 172.21.15.98:0/2559908433 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c00f960 con 0x7fbe98004d40 2024-05-09T10:50:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.306+0000 7fbea97fa640 1 -- 172.21.15.98:0/2559908433 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9c018070 con 0x7fbe98004d40 2024-05-09T10:50:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.306+0000 7fbea97fa640 1 -- 172.21.15.98:0/2559908433 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c00fd40 con 0x7fbe98004d40 2024-05-09T10:50:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.306+0000 7fbeb29b0640 1 -- 172.21.15.98:0/2559908433 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeac0c81d0 con 0x7fbe98004d40 2024-05-09T10:50:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.306+0000 7fbeb29b0640 1 -- 172.21.15.98:0/2559908433 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeac155cd0 con 0x7fbe98004d40 2024-05-09T10:50:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.307+0000 7fbeb29b0640 1 -- 172.21.15.98:0/2559908433 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeac069cb0 con 0x7fbe98004d40 2024-05-09T10:50:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.308+0000 7fbea97fa640 1 -- 172.21.15.98:0/2559908433 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe9c01d460 con 0x7fbe98004d40 2024-05-09T10:50:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.308+0000 7fbea97fa640 1 --2- 172.21.15.98:0/2559908433 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fbe8c06b4d0 0x7fbe8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.308+0000 7fbea97fa640 1 -- 172.21.15.98:0/2559908433 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe9c08a3b0 con 0x7fbe98004d40 2024-05-09T10:50:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.308+0000 7fbea3fff640 1 --2- 172.21.15.98:0/2559908433 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fbe8c06b4d0 0x7fbe8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:22.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.308+0000 7fbea3fff640 1 --2- 172.21.15.98:0/2559908433 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fbe8c06b4d0 0x7fbe8c06d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0cdc90 tx=0x7fbe94009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:22.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.312+0000 7fbea97fa640 1 -- 172.21.15.98:0/2559908433 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe9c059230 con 0x7fbe98004d40 2024-05-09T10:50:22.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.517+0000 7fbeb29b0640 1 -- 172.21.15.98:0/2559908433 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbeac0c7c30 con 0x7fbe98004d40 2024-05-09T10:50:22.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.518+0000 7fbea97fa640 1 -- 172.21.15.98:0/2559908433 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4932 (secure 0 0 0) 0x7fbe9c058b30 con 0x7fbe98004d40 2024-05-09T10:50:22.511 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:22.511 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2903091926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2903091926},{"type":"v1","addr":"172.21.15.98:6837","nonce":2903091926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/968936316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":968936316},{"type":"v1","addr":"172.21.15.105:6834","nonce":968936316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/937700297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":937700297},{"type":"v1","addr":"172.21.15.98:6835","nonce":937700297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T10:50:19.544432+0000","modified":"2024-05-09T10:50:21.512942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6835/1780062557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1780062557},{"type":"v1","addr":"172.21.15.105:6835","nonce":1780062557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T10:50:22.512 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 6 2024-05-09T10:50:22.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.520+0000 7fbeb29b0640 1 -- 172.21.15.98:0/2559908433 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fbe8c06b4d0 msgr2=0x7fbe8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:22.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.520+0000 7fbeb29b0640 1 --2- 172.21.15.98:0/2559908433 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fbe8c06b4d0 0x7fbe8c06d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0cdc90 tx=0x7fbe94009040 comp rx=0 tx=0).stop 2024-05-09T10:50:22.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.520+0000 7fbeb29b0640 1 -- 172.21.15.98:0/2559908433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe98004d40 msgr2=0x7fbeac1582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:22.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.520+0000 7fbeb29b0640 1 --2- 172.21.15.98:0/2559908433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe98004d40 0x7fbeac1582c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c000f60 tx=0x7fbe9c007720 comp rx=0 tx=0).stop 2024-05-09T10:50:22.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.520+0000 7fbeb29b0640 1 -- 172.21.15.98:0/2559908433 shutdown_connections 2024-05-09T10:50:22.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.520+0000 7fbeb29b0640 1 --2- 172.21.15.98:0/2559908433 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fbe8c06b4d0 0x7fbe8c06d990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.520+0000 7fbeb29b0640 1 --2- 172.21.15.98:0/2559908433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbeac0cccf0 0x7fbeac0cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.520+0000 7fbeb29b0640 1 --2- 172.21.15.98:0/2559908433 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbeac155fb0 0x7fbeac155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.520+0000 7fbeb29b0640 1 --2- 172.21.15.98:0/2559908433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe98004d40 0x7fbeac1582c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.520+0000 7fbeb29b0640 1 -- 172.21.15.98:0/2559908433 >> 172.21.15.98:0/2559908433 conn(0x7fbeac09ce90 msgr2=0x7fbeac0b67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:22.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.520+0000 7fbeb29b0640 1 -- 172.21.15.98:0/2559908433 shutdown_connections 2024-05-09T10:50:22.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.520+0000 7fbeb29b0640 1 -- 172.21.15.98:0/2559908433 wait complete. 2024-05-09T10:50:22.524 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:22.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.631+0000 7efcfed9e640 1 Processor -- start 2024-05-09T10:50:22.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.631+0000 7efcfed9e640 1 -- start start 2024-05-09T10:50:22.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.631+0000 7efcfed9e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7efcf815e2c0 0x7efcf806c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:22.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.631+0000 7efcfed9e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf815c190 con 0x7efcf806bbf0 2024-05-09T10:50:22.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.631+0000 7efcfed9e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf815c470 con 0x7efcf815f0e0 2024-05-09T10:50:22.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.631+0000 7efcfed9e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf815cda0 con 0x7efcf815e2c0 2024-05-09T10:50:22.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.632+0000 7efcfd33b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7efcf815e2c0 0x7efcf806c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:22.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.632+0000 7efcf7fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7efcf806bbf0 0x7efcf8152150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42342/0 (socket says 172.21.15.98:42342) 2024-05-09T10:50:22.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.632+0000 7efcf7fff640 1 -- 172.21.15.98:0/1270887162 learned_addr learned my addr 172.21.15.98:0/1270887162 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:22.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.632+0000 7efcfd33b640 1 -- 172.21.15.98:0/1270887162 >> v1:172.21.15.105:6789/0 conn(0x7efcf815f0e0 legacy=0x7efcf80cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:22.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.632+0000 7efcfd33b640 1 -- 172.21.15.98:0/1270887162 >> v1:172.21.15.98:6789/0 conn(0x7efcf806bbf0 legacy=0x7efcf8152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:22.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.632+0000 7efcf77fe640 1 -- 172.21.15.98:0/1270887162 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2041743107 0 0) 0x7efcf815c190 con 0x7efcf806bbf0 2024-05-09T10:50:22.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.632+0000 7efcfd33b640 1 -- 172.21.15.98:0/1270887162 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf815c750 con 0x7efcf815e2c0 2024-05-09T10:50:22.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.633+0000 7efcfd33b640 1 --2- 172.21.15.98:0/1270887162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcf815e2c0 0x7efcf806c070 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efce00097a0 tx=0x7efce002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8d4a40b9da8cee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:22.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.633+0000 7efcf77fe640 1 -- 172.21.15.98:0/1270887162 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce0037070 con 0x7efcf815e2c0 2024-05-09T10:50:22.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.633+0000 7efcf77fe640 1 -- 172.21.15.98:0/1270887162 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce0032440 con 0x7efcf815e2c0 2024-05-09T10:50:22.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.633+0000 7efcf77fe640 1 -- 172.21.15.98:0/1270887162 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce0032810 con 0x7efcf815e2c0 2024-05-09T10:50:22.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.633+0000 7efcfed9e640 1 -- 172.21.15.98:0/1270887162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcf815e2c0 msgr2=0x7efcf806c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:22.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.633+0000 7efcfed9e640 1 --2- 172.21.15.98:0/1270887162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcf815e2c0 0x7efcf806c070 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efce00097a0 tx=0x7efce002a230 comp rx=0 tx=0).stop 2024-05-09T10:50:22.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.633+0000 7efcfed9e640 1 -- 172.21.15.98:0/1270887162 shutdown_connections 2024-05-09T10:50:22.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.633+0000 7efcfed9e640 1 --2- 172.21.15.98:0/1270887162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcf815e2c0 0x7efcf806c070 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.633+0000 7efcfed9e640 1 -- 172.21.15.98:0/1270887162 >> 172.21.15.98:0/1270887162 conn(0x7efcf806cae0 msgr2=0x7efcf806cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:22.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.633+0000 7efcfed9e640 1 -- 172.21.15.98:0/1270887162 shutdown_connections 2024-05-09T10:50:22.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.633+0000 7efcfed9e640 1 -- 172.21.15.98:0/1270887162 wait complete. 2024-05-09T10:50:22.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.634+0000 7efcfed9e640 1 Processor -- start 2024-05-09T10:50:22.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.635+0000 7efcfed9e640 1 -- start start 2024-05-09T10:50:22.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.635+0000 7efcfed9e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efcf806bbf0 0x7efcf8148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:22.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.635+0000 7efcfed9e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcf815e2c0 0x7efcf81e41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:22.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.635+0000 7efcfed9e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcf815f0e0 0x7efcf814c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:22.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.635+0000 7efcfed9e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80afa70 con 0x7efcf815e2c0 2024-05-09T10:50:22.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.635+0000 7efcfed9e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80af8f0 con 0x7efcf815f0e0 2024-05-09T10:50:22.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.635+0000 7efcfed9e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80cfcb0 con 0x7efcf806bbf0 2024-05-09T10:50:22.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.635+0000 7efcf7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcf815e2c0 0x7efcf81e41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:22.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.636+0000 7efcf7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcf815e2c0 0x7efcf81e41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34424/0 (socket says 172.21.15.98:34424) 2024-05-09T10:50:22.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.636+0000 7efcf7fff640 1 -- 172.21.15.98:0/1489649709 learned_addr learned my addr 172.21.15.98:0/1489649709 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:22.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.636+0000 7efcfcb3a640 1 --2- 172.21.15.98:0/1489649709 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efcf806bbf0 0x7efcf8148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:22.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.636+0000 7efcfd33b640 1 --2- 172.21.15.98:0/1489649709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcf815f0e0 0x7efcf814c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:22.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.636+0000 7efcf7fff640 1 -- 172.21.15.98:0/1489649709 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efcf806bbf0 msgr2=0x7efcf8148f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:22.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.636+0000 7efcf7fff640 1 --2- 172.21.15.98:0/1489649709 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efcf806bbf0 0x7efcf8148f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.636+0000 7efcf7fff640 1 -- 172.21.15.98:0/1489649709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcf815f0e0 msgr2=0x7efcf814c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:22.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.636+0000 7efcf7fff640 1 --2- 172.21.15.98:0/1489649709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcf815f0e0 0x7efcf814c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.636+0000 7efcf7fff640 1 -- 172.21.15.98:0/1489649709 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf81e5250 con 0x7efcf815e2c0 2024-05-09T10:50:22.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.636+0000 7efcf7fff640 1 --2- 172.21.15.98:0/1489649709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcf815e2c0 0x7efcf81e41d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efce800c780 tx=0x7efce800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:22.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.636+0000 7efcf57fa640 1 -- 172.21.15.98:0/1489649709 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce8018070 con 0x7efcf815e2c0 2024-05-09T10:50:22.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.636+0000 7efcf57fa640 1 -- 172.21.15.98:0/1489649709 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce800a640 con 0x7efcf815e2c0 2024-05-09T10:50:22.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.636+0000 7efcfed9e640 1 -- 172.21.15.98:0/1489649709 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf81e4c90 con 0x7efcf815e2c0 2024-05-09T10:50:22.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.636+0000 7efcf57fa640 1 -- 172.21.15.98:0/1489649709 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce800aab0 con 0x7efcf815e2c0 2024-05-09T10:50:22.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.636+0000 7efcfed9e640 1 -- 172.21.15.98:0/1489649709 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf8149e20 con 0x7efcf815e2c0 2024-05-09T10:50:22.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.637+0000 7efcfed9e640 1 -- 172.21.15.98:0/1489649709 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf8069cb0 con 0x7efcf815e2c0 2024-05-09T10:50:22.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.637+0000 7efcf57fa640 1 -- 172.21.15.98:0/1489649709 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efce801b070 con 0x7efcf815e2c0 2024-05-09T10:50:22.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.638+0000 7efcf57fa640 1 --2- 172.21.15.98:0/1489649709 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7efcd806b4d0 0x7efcd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:22.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.638+0000 7efcfcb3a640 1 --2- 172.21.15.98:0/1489649709 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7efcd806b4d0 0x7efcd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:22.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.638+0000 7efcf57fa640 1 -- 172.21.15.98:0/1489649709 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efce8089680 con 0x7efcf815e2c0 2024-05-09T10:50:22.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.638+0000 7efcfcb3a640 1 --2- 172.21.15.98:0/1489649709 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7efcd806b4d0 0x7efcd806d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efcf0000f60 tx=0x7efcf0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:22.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.642+0000 7efcf57fa640 1 -- 172.21.15.98:0/1489649709 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efce8057f40 con 0x7efcf815e2c0 2024-05-09T10:50:22.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.812+0000 7efcfed9e640 1 -- 172.21.15.98:0/1489649709 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efcf8070ed0 con 0x7efcf815e2c0 2024-05-09T10:50:22.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.814+0000 7efcf57fa640 1 -- 172.21.15.98:0/1489649709 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7efce805bfe0 con 0x7efcf815e2c0 2024-05-09T10:50:22.807 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:22.807 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":20,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:21.176419+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:50:17.323444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T10:50:18.482315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:22.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.815+0000 7efcfed9e640 1 -- 172.21.15.98:0/1489649709 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7efcd806b4d0 msgr2=0x7efcd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:22.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.815+0000 7efcfed9e640 1 --2- 172.21.15.98:0/1489649709 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7efcd806b4d0 0x7efcd806d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efcf0000f60 tx=0x7efcf0009040 comp rx=0 tx=0).stop 2024-05-09T10:50:22.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.815+0000 7efcfed9e640 1 -- 172.21.15.98:0/1489649709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcf815e2c0 msgr2=0x7efcf81e41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:22.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.815+0000 7efcfed9e640 1 --2- 172.21.15.98:0/1489649709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcf815e2c0 0x7efcf81e41d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efce800c780 tx=0x7efce800cc40 comp rx=0 tx=0).stop 2024-05-09T10:50:22.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.816+0000 7efcfed9e640 1 -- 172.21.15.98:0/1489649709 shutdown_connections 2024-05-09T10:50:22.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.816+0000 7efcfed9e640 1 --2- 172.21.15.98:0/1489649709 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7efcd806b4d0 0x7efcd806d990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.816+0000 7efcfed9e640 1 --2- 172.21.15.98:0/1489649709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcf815f0e0 0x7efcf814c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.816+0000 7efcfed9e640 1 --2- 172.21.15.98:0/1489649709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcf815e2c0 0x7efcf81e41d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.816+0000 7efcfed9e640 1 --2- 172.21.15.98:0/1489649709 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efcf806bbf0 0x7efcf8148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.816+0000 7efcfed9e640 1 -- 172.21.15.98:0/1489649709 >> 172.21.15.98:0/1489649709 conn(0x7efcf806cae0 msgr2=0x7efcf80b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:22.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.816+0000 7efcfed9e640 1 -- 172.21.15.98:0/1489649709 shutdown_connections 2024-05-09T10:50:22.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.816+0000 7efcfed9e640 1 -- 172.21.15.98:0/1489649709 wait complete. 2024-05-09T10:50:22.820 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-09T10:50:22.820 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T10:50:22.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.926+0000 7f5328e88640 1 Processor -- start 2024-05-09T10:50:22.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.926+0000 7f5328e88640 1 -- start start 2024-05-09T10:50:22.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.927+0000 7f5328e88640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f532415bdc0 0x7f53240cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:22.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.927+0000 7f5328e88640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53240cc4c0 con 0x7f53240ceb70 2024-05-09T10:50:22.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.927+0000 7f5328e88640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53240cf0e0 con 0x7f53240b4f10 2024-05-09T10:50:22.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.927+0000 7f5328e88640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532415c1a0 con 0x7f532415bdc0 2024-05-09T10:50:22.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.927+0000 7f532259c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f532415bdc0 0x7f53240cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:22.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.927+0000 7f531bfff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f53240ceb70 0x7f53240b4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42352/0 (socket says 172.21.15.98:42352) 2024-05-09T10:50:22.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.927+0000 7f531bfff640 1 -- 172.21.15.98:0/400878367 learned_addr learned my addr 172.21.15.98:0/400878367 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:22.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.927+0000 7f532259c640 1 -- 172.21.15.98:0/400878367 >> v1:172.21.15.105:6789/0 conn(0x7f53240b4f10 legacy=0x7f53240cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:22.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.927+0000 7f532259c640 1 -- 172.21.15.98:0/400878367 >> v1:172.21.15.98:6789/0 conn(0x7f53240ceb70 legacy=0x7f53240b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:22.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.927+0000 7f532259c640 1 -- 172.21.15.98:0/400878367 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53240cc760 con 0x7f532415bdc0 2024-05-09T10:50:22.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.927+0000 7f5321d9b640 1 -- 172.21.15.98:0/400878367 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2195876447 0 0) 0x7f53240cc4c0 con 0x7f53240ceb70 2024-05-09T10:50:22.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.928+0000 7f532259c640 1 --2- 172.21.15.98:0/400878367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f532415bdc0 0x7f53240cd240 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f53140097a0 tx=0x7f531402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26728a77b9fbb09 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:22.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.928+0000 7f5321d9b640 1 -- 172.21.15.98:0/400878367 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314037070 con 0x7f532415bdc0 2024-05-09T10:50:22.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.928+0000 7f5321d9b640 1 -- 172.21.15.98:0/400878367 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5314032440 con 0x7f532415bdc0 2024-05-09T10:50:22.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.928+0000 7f5321d9b640 1 -- 172.21.15.98:0/400878367 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314032810 con 0x7f532415bdc0 2024-05-09T10:50:22.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.929+0000 7f5328e88640 1 -- 172.21.15.98:0/400878367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f532415bdc0 msgr2=0x7f53240cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:22.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.929+0000 7f5328e88640 1 --2- 172.21.15.98:0/400878367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f532415bdc0 0x7f53240cd240 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f53140097a0 tx=0x7f531402a1e0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.929+0000 7f5328e88640 1 -- 172.21.15.98:0/400878367 shutdown_connections 2024-05-09T10:50:22.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.929+0000 7f5328e88640 1 --2- 172.21.15.98:0/400878367 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f532415bdc0 0x7f53240cd240 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.929+0000 7f5328e88640 1 -- 172.21.15.98:0/400878367 >> 172.21.15.98:0/400878367 conn(0x7f53240b34e0 msgr2=0x7f53240b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:22.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.929+0000 7f5328e88640 1 -- 172.21.15.98:0/400878367 shutdown_connections 2024-05-09T10:50:22.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.929+0000 7f5328e88640 1 -- 172.21.15.98:0/400878367 wait complete. 2024-05-09T10:50:22.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.930+0000 7f5328e88640 1 Processor -- start 2024-05-09T10:50:22.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.930+0000 7f5328e88640 1 -- start start 2024-05-09T10:50:22.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.931+0000 7f5328e88640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53240b4f10 0x7f53240c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:22.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.931+0000 7f5328e88640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53240ceb70 0x7f5324159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:22.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.931+0000 7f5328e88640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f532415bdc0 0x7f53240c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:22.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.931+0000 7f5328e88640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53240ccd20 con 0x7f532415bdc0 2024-05-09T10:50:22.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.931+0000 7f5328e88640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53240cf3c0 con 0x7f53240b4f10 2024-05-09T10:50:22.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.931+0000 7f5328e88640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53240b41b0 con 0x7f53240ceb70 2024-05-09T10:50:22.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.931+0000 7f5322d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f532415bdc0 0x7f53240c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:22.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.931+0000 7f5322d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f532415bdc0 0x7f53240c7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34434/0 (socket says 172.21.15.98:34434) 2024-05-09T10:50:22.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.931+0000 7f5322d9d640 1 -- 172.21.15.98:0/315343232 learned_addr learned my addr 172.21.15.98:0/315343232 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:22.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.931+0000 7f532259c640 1 --2- 172.21.15.98:0/315343232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53240b4f10 0x7f53240c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:22.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.931+0000 7f531bfff640 1 --2- 172.21.15.98:0/315343232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53240ceb70 0x7f5324159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:22.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.931+0000 7f5322d9d640 1 -- 172.21.15.98:0/315343232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53240ceb70 msgr2=0x7f5324159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:22.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.931+0000 7f5322d9d640 1 --2- 172.21.15.98:0/315343232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53240ceb70 0x7f5324159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.931+0000 7f5322d9d640 1 -- 172.21.15.98:0/315343232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53240b4f10 msgr2=0x7f53240c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:22.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.931+0000 7f5322d9d640 1 --2- 172.21.15.98:0/315343232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53240b4f10 0x7f53240c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:22.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.931+0000 7f5322d9d640 1 -- 172.21.15.98:0/315343232 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5324159690 con 0x7f532415bdc0 2024-05-09T10:50:22.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.931+0000 7f5322d9d640 1 --2- 172.21.15.98:0/315343232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f532415bdc0 0x7f53240c7d40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f531c00c780 tx=0x7f531c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:22.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.931+0000 7f531affd640 1 -- 172.21.15.98:0/315343232 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c017070 con 0x7f532415bdc0 2024-05-09T10:50:22.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.931+0000 7f531affd640 1 -- 172.21.15.98:0/315343232 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531c003da0 con 0x7f532415bdc0 2024-05-09T10:50:22.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.931+0000 7f5328e88640 1 -- 172.21.15.98:0/315343232 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53240c6ca0 con 0x7f532415bdc0 2024-05-09T10:50:22.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.931+0000 7f531affd640 1 -- 172.21.15.98:0/315343232 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c0052b0 con 0x7f532415bdc0 2024-05-09T10:50:22.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.931+0000 7f5328e88640 1 -- 172.21.15.98:0/315343232 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53240c7260 con 0x7f532415bdc0 2024-05-09T10:50:22.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.932+0000 7f5328e88640 1 -- 172.21.15.98:0/315343232 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53240c7a30 con 0x7f532415bdc0 2024-05-09T10:50:22.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.933+0000 7f531affd640 1 -- 172.21.15.98:0/315343232 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f531c01a070 con 0x7f532415bdc0 2024-05-09T10:50:22.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.933+0000 7f531affd640 1 --2- 172.21.15.98:0/315343232 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f530406b4d0 0x7f530406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:22.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.933+0000 7f531affd640 1 -- 172.21.15.98:0/315343232 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f531c089840 con 0x7f532415bdc0 2024-05-09T10:50:22.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.933+0000 7f532259c640 1 --2- 172.21.15.98:0/315343232 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f530406b4d0 0x7f530406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:22.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.934+0000 7f532259c640 1 --2- 172.21.15.98:0/315343232 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f530406b4d0 0x7f530406d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5314034000 tx=0x7f531402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:22.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:22.937+0000 7f531affd640 1 -- 172.21.15.98:0/315343232 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f531c0586c0 con 0x7f532415bdc0 2024-05-09T10:50:23.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.105+0000 7f5328e88640 1 -- 172.21.15.98:0/315343232 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5324069cb0 con 0x7f530406b4d0 2024-05-09T10:50:23.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.110+0000 7f531affd640 1 -- 172.21.15.98:0/315343232 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237079 (secure 0 0 0) 0x7f5324069cb0 con 0x7f530406b4d0 2024-05-09T10:50:23.106 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:23.108 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-05-09T10:50:23.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.113+0000 7f5328e88640 1 -- 172.21.15.98:0/315343232 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f530406b4d0 msgr2=0x7f530406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:23.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.113+0000 7f5328e88640 1 --2- 172.21.15.98:0/315343232 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f530406b4d0 0x7f530406d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5314034000 tx=0x7f531402e000 comp rx=0 tx=0).stop 2024-05-09T10:50:23.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.113+0000 7f5328e88640 1 -- 172.21.15.98:0/315343232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f532415bdc0 msgr2=0x7f53240c7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:23.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.113+0000 7f5328e88640 1 --2- 172.21.15.98:0/315343232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f532415bdc0 0x7f53240c7d40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f531c00c780 tx=0x7f531c00cc40 comp rx=0 tx=0).stop 2024-05-09T10:50:23.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.113+0000 7f5328e88640 1 -- 172.21.15.98:0/315343232 shutdown_connections 2024-05-09T10:50:23.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.113+0000 7f5328e88640 1 --2- 172.21.15.98:0/315343232 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f530406b4d0 0x7f530406d990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.113+0000 7f5328e88640 1 --2- 172.21.15.98:0/315343232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f532415bdc0 0x7f53240c7d40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.113+0000 7f5328e88640 1 --2- 172.21.15.98:0/315343232 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f53240ceb70 0x7f5324159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.113+0000 7f5328e88640 1 --2- 172.21.15.98:0/315343232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53240b4f10 0x7f53240c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.113+0000 7f5328e88640 1 -- 172.21.15.98:0/315343232 >> 172.21.15.98:0/315343232 conn(0x7f53240b34e0 msgr2=0x7f532406fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:23.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.113+0000 7f5328e88640 1 -- 172.21.15.98:0/315343232 shutdown_connections 2024-05-09T10:50:23.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.113+0000 7f5328e88640 1 -- 172.21.15.98:0/315343232 wait complete. 2024-05-09T10:50:23.117 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-05-09T10:50:21.366002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":219132,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755588,"statfs":{"total":773094113280,"available":772869722112,"internally_reserved":0,"allocated":2433024,"data_stored":1546872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000589"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-09T10:50:09.593681+0000","last_change":"2024-05-09T10:50:08.410364+0000","last_active":"2024-05-09T10:50:09.593681+0000","last_peered":"2024-05-09T10:50:09.593681+0000","last_clean":"2024-05-09T10:50:09.593681+0000","last_became_active":"2024-05-09T10:50:08.409457+0000","last_became_peered":"2024-05-09T10:50:08.409457+0000","last_unstale":"2024-05-09T10:50:09.593681+0000","last_undegraded":"2024-05-09T10:50:09.593681+0000","last_fullsized":"2024-05-09T10:50:09.593681+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_clean_scrub_stamp":"2024-05-09T10:50:07.396510+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:03.566698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:19.147433+0000","last_change":"2024-05-09T10:50:19.147433+0000","last_active":"2024-05-09T10:50:19.147433+0000","last_peered":"2024-05-09T10:50:19.147433+0000","last_clean":"2024-05-09T10:50:19.147433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:19.147433+0000","last_undegraded":"2024-05-09T10:50:19.147433+0000","last_fullsized":"2024-05-09T10:50:19.147433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-09T10:50:18.142317+0000","last_change":"2024-05-09T10:50:18.142317+0000","last_active":"2024-05-09T10:50:18.142317+0000","last_peered":"2024-05-09T10:50:18.142317+0000","last_clean":"2024-05-09T10:50:18.142317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:50:18.142317+0000","last_undegraded":"2024-05-09T10:50:18.142317+0000","last_fullsized":"2024-05-09T10:50:18.142317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28316,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343524,"statfs":{"total":96636764160,"available":96607768576,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T10:50:23.122 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T10:50:23.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.265+0000 7fe3fcb34640 1 Processor -- start 2024-05-09T10:50:23.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.265+0000 7fe3fcb34640 1 -- start start 2024-05-09T10:50:23.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.266+0000 7fe3fcb34640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3f815f0e0 0x7fe3f80cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:23.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.266+0000 7fe3fcb34640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe3f80cfae0 0x7fe3f806bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:23.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.266+0000 7fe3fcb34640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe3f806c130 0x7fe3f8152150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:23.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.266+0000 7fe3fcb34640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f806c580 con 0x7fe3f80cfae0 2024-05-09T10:50:23.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.266+0000 7fe3fcb34640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f815cda0 con 0x7fe3f806c130 2024-05-09T10:50:23.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.266+0000 7fe3fcb34640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f815c190 con 0x7fe3f815f0e0 2024-05-09T10:50:23.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.266+0000 7fe3f5d9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe3f80cfae0 0x7fe3f806bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:23.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.266+0000 7fe3f5d9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe3f80cfae0 0x7fe3f806bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34444/0 (socket says 172.21.15.98:34444) 2024-05-09T10:50:23.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.266+0000 7fe3f5d9b640 1 -- 172.21.15.98:0/445138535 learned_addr learned my addr 172.21.15.98:0/445138535 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:23.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.266+0000 7fe3f659c640 1 --2- 172.21.15.98:0/445138535 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3f815f0e0 0x7fe3f80cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:23.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.266+0000 7fe3f6d9d640 1 --2- 172.21.15.98:0/445138535 >> v2:172.21.15.105:3300/0 conn(0x7fe3f806c130 0x7fe3f8152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:23.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.267+0000 7fe3f5d9b640 1 -- 172.21.15.98:0/445138535 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3f815f0e0 msgr2=0x7fe3f80cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:23.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.267+0000 7fe3f5d9b640 1 --2- 172.21.15.98:0/445138535 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3f815f0e0 0x7fe3f80cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.267+0000 7fe3f5d9b640 1 -- 172.21.15.98:0/445138535 >> v2:172.21.15.105:3300/0 conn(0x7fe3f806c130 msgr2=0x7fe3f8152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:23.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.267+0000 7fe3f5d9b640 1 --2- 172.21.15.98:0/445138535 >> v2:172.21.15.105:3300/0 conn(0x7fe3f806c130 0x7fe3f8152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.267+0000 7fe3f5d9b640 1 -- 172.21.15.98:0/445138535 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f815c470 con 0x7fe3f80cfae0 2024-05-09T10:50:23.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.267+0000 7fe3f5d9b640 1 --2- 172.21.15.98:0/445138535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3f80cfae0 0x7fe3f806bbf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8066f10 tx=0x7fe3ec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6475409409f21612 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:23.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.267+0000 7fe3f559a640 1 -- 172.21.15.98:0/445138535 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec037070 con 0x7fe3f80cfae0 2024-05-09T10:50:23.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.267+0000 7fe3f559a640 1 -- 172.21.15.98:0/445138535 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ec032440 con 0x7fe3f80cfae0 2024-05-09T10:50:23.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.267+0000 7fe3f559a640 1 -- 172.21.15.98:0/445138535 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec032810 con 0x7fe3f80cfae0 2024-05-09T10:50:23.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.268+0000 7fe3fcb34640 1 -- 172.21.15.98:0/445138535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3f80cfae0 msgr2=0x7fe3f806bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:23.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.268+0000 7fe3fcb34640 1 --2- 172.21.15.98:0/445138535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3f80cfae0 0x7fe3f806bbf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8066f10 tx=0x7fe3ec02a1e0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.268+0000 7fe3fcb34640 1 -- 172.21.15.98:0/445138535 shutdown_connections 2024-05-09T10:50:23.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.268+0000 7fe3fcb34640 1 --2- 172.21.15.98:0/445138535 >> v2:172.21.15.105:3300/0 conn(0x7fe3f806c130 0x7fe3f8152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.268+0000 7fe3fcb34640 1 --2- 172.21.15.98:0/445138535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3f80cfae0 0x7fe3f806bbf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.268+0000 7fe3fcb34640 1 --2- 172.21.15.98:0/445138535 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3f815f0e0 0x7fe3f80cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.268+0000 7fe3fcb34640 1 -- 172.21.15.98:0/445138535 >> 172.21.15.98:0/445138535 conn(0x7fe3f806cae0 msgr2=0x7fe3f806cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:23.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.268+0000 7fe3fcb34640 1 -- 172.21.15.98:0/445138535 shutdown_connections 2024-05-09T10:50:23.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.268+0000 7fe3fcb34640 1 -- 172.21.15.98:0/445138535 wait complete. 2024-05-09T10:50:23.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.269+0000 7fe3fcb34640 1 Processor -- start 2024-05-09T10:50:23.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.269+0000 7fe3fcb34640 1 -- start start 2024-05-09T10:50:23.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.270+0000 7fe3fcb34640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f806c130 0x7fe3f81428b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:23.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.270+0000 7fe3fcb34640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3f80cfae0 0x7fe3f80b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:23.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.270+0000 7fe3fcb34640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3f815f0e0 0x7fe3f81f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:23.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.270+0000 7fe3fcb34640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f80af8f0 con 0x7fe3f80cfae0 2024-05-09T10:50:23.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.270+0000 7fe3fcb34640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f815ca30 con 0x7fe3f806c130 2024-05-09T10:50:23.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.270+0000 7fe3fcb34640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f80afa70 con 0x7fe3f815f0e0 2024-05-09T10:50:23.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.270+0000 7fe3f5d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3f80cfae0 0x7fe3f80b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:23.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.270+0000 7fe3f5d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3f80cfae0 0x7fe3f80b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34450/0 (socket says 172.21.15.98:34450) 2024-05-09T10:50:23.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.270+0000 7fe3f5d9b640 1 -- 172.21.15.98:0/2871681472 learned_addr learned my addr 172.21.15.98:0/2871681472 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:23.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.270+0000 7fe3f5d9b640 1 -- 172.21.15.98:0/2871681472 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3f815f0e0 msgr2=0x7fe3f81f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:23.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.270+0000 7fe3f5d9b640 1 --2- 172.21.15.98:0/2871681472 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3f815f0e0 0x7fe3f81f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.270+0000 7fe3f5d9b640 1 -- 172.21.15.98:0/2871681472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f806c130 msgr2=0x7fe3f81428b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:23.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.270+0000 7fe3f5d9b640 1 --2- 172.21.15.98:0/2871681472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f806c130 0x7fe3f81428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.270+0000 7fe3f5d9b640 1 -- 172.21.15.98:0/2871681472 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f81f41e0 con 0x7fe3f80cfae0 2024-05-09T10:50:23.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.271+0000 7fe3f5d9b640 1 --2- 172.21.15.98:0/2871681472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3f80cfae0 0x7fe3f80b3b80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec02a220 tx=0x7fe3ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:23.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.271+0000 7fe3e6ffd640 1 -- 172.21.15.98:0/2871681472 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec041020 con 0x7fe3f80cfae0 2024-05-09T10:50:23.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.271+0000 7fe3e6ffd640 1 -- 172.21.15.98:0/2871681472 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ec0306c0 con 0x7fe3f80cfae0 2024-05-09T10:50:23.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.271+0000 7fe3fcb34640 1 -- 172.21.15.98:0/2871681472 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f81f44c0 con 0x7fe3f80cfae0 2024-05-09T10:50:23.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.271+0000 7fe3e6ffd640 1 -- 172.21.15.98:0/2871681472 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec037040 con 0x7fe3f80cfae0 2024-05-09T10:50:23.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.271+0000 7fe3fcb34640 1 -- 172.21.15.98:0/2871681472 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f80b3660 con 0x7fe3f80cfae0 2024-05-09T10:50:23.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.272+0000 7fe3fcb34640 1 -- 172.21.15.98:0/2871681472 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f8069cb0 con 0x7fe3f80cfae0 2024-05-09T10:50:23.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.272+0000 7fe3e6ffd640 1 -- 172.21.15.98:0/2871681472 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe3ec044020 con 0x7fe3f80cfae0 2024-05-09T10:50:23.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.273+0000 7fe3e6ffd640 1 --2- 172.21.15.98:0/2871681472 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe3d806b4d0 0x7fe3d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:23.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.273+0000 7fe3e6ffd640 1 -- 172.21.15.98:0/2871681472 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3ec0aaa00 con 0x7fe3f80cfae0 2024-05-09T10:50:23.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.273+0000 7fe3f659c640 1 --2- 172.21.15.98:0/2871681472 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe3d806b4d0 0x7fe3d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:23.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.273+0000 7fe3f659c640 1 --2- 172.21.15.98:0/2871681472 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe3d806b4d0 0x7fe3d806d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe3e80070b0 tx=0x7fe3e8007040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:23.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.277+0000 7fe3e6ffd640 1 -- 172.21.15.98:0/2871681472 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe3ec04b030 con 0x7fe3f80cfae0 2024-05-09T10:50:23.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.442+0000 7fe3fcb34640 1 -- 172.21.15.98:0/2871681472 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe3f815e000 con 0x7fe3d806b4d0 2024-05-09T10:50:23.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.448+0000 7fe3e6ffd640 1 -- 172.21.15.98:0/2871681472 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251916 (secure 0 0 0) 0x7fe3f815e000 con 0x7fe3d806b4d0 2024-05-09T10:50:23.444 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:23.445 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-05-09T10:50:23.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.451+0000 7fe3fcb34640 1 -- 172.21.15.98:0/2871681472 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe3d806b4d0 msgr2=0x7fe3d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:23.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.451+0000 7fe3fcb34640 1 --2- 172.21.15.98:0/2871681472 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe3d806b4d0 0x7fe3d806d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe3e80070b0 tx=0x7fe3e8007040 comp rx=0 tx=0).stop 2024-05-09T10:50:23.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.451+0000 7fe3fcb34640 1 -- 172.21.15.98:0/2871681472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3f80cfae0 msgr2=0x7fe3f80b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:23.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.451+0000 7fe3fcb34640 1 --2- 172.21.15.98:0/2871681472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3f80cfae0 0x7fe3f80b3b80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec02a220 tx=0x7fe3ec0046c0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.451+0000 7fe3fcb34640 1 -- 172.21.15.98:0/2871681472 shutdown_connections 2024-05-09T10:50:23.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.451+0000 7fe3fcb34640 1 --2- 172.21.15.98:0/2871681472 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe3d806b4d0 0x7fe3d806d990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.451+0000 7fe3fcb34640 1 --2- 172.21.15.98:0/2871681472 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3f815f0e0 0x7fe3f81f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.451+0000 7fe3fcb34640 1 --2- 172.21.15.98:0/2871681472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3f80cfae0 0x7fe3f80b3b80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.451+0000 7fe3fcb34640 1 --2- 172.21.15.98:0/2871681472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3f806c130 0x7fe3f81428b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.451+0000 7fe3fcb34640 1 -- 172.21.15.98:0/2871681472 >> 172.21.15.98:0/2871681472 conn(0x7fe3f806cae0 msgr2=0x7fe3f815d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:23.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.451+0000 7fe3fcb34640 1 -- 172.21.15.98:0/2871681472 shutdown_connections 2024-05-09T10:50:23.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.451+0000 7fe3fcb34640 1 -- 172.21.15.98:0/2871681472 wait complete. 2024-05-09T10:50:23.455 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-05-09T10:50:23.366299+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218940,"kb_used_data":2764,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755780,"statfs":{"total":773094113280,"available":772869918720,"internally_reserved":0,"allocated":2830336,"data_stored":1721528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.218204"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889511+0000","last_change":"2024-05-09T10:50:20.202000+0000","last_active":"2024-05-09T10:50:21.889511+0000","last_peered":"2024-05-09T10:50:21.889511+0000","last_clean":"2024-05-09T10:50:21.889511+0000","last_became_active":"2024-05-09T10:50:20.201722+0000","last_became_peered":"2024-05-09T10:50:20.201722+0000","last_unstale":"2024-05-09T10:50:21.889511+0000","last_undegraded":"2024-05-09T10:50:21.889511+0000","last_fullsized":"2024-05-09T10:50:21.889511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:51:23.834797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182607+0000","last_change":"2024-05-09T10:50:20.185947+0000","last_active":"2024-05-09T10:50:21.182607+0000","last_peered":"2024-05-09T10:50:21.182607+0000","last_clean":"2024-05-09T10:50:21.182607+0000","last_became_active":"2024-05-09T10:50:20.185672+0000","last_became_peered":"2024-05-09T10:50:20.185672+0000","last_unstale":"2024-05-09T10:50:21.182607+0000","last_undegraded":"2024-05-09T10:50:21.182607+0000","last_fullsized":"2024-05-09T10:50:21.182607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:45.067992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182555+0000","last_change":"2024-05-09T10:50:20.186891+0000","last_active":"2024-05-09T10:50:21.182555+0000","last_peered":"2024-05-09T10:50:21.182555+0000","last_clean":"2024-05-09T10:50:21.182555+0000","last_became_active":"2024-05-09T10:50:20.186605+0000","last_became_peered":"2024-05-09T10:50:20.186605+0000","last_unstale":"2024-05-09T10:50:21.182555+0000","last_undegraded":"2024-05-09T10:50:21.182555+0000","last_fullsized":"2024-05-09T10:50:21.182555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:30.799640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.202017+0000","last_change":"2024-05-09T10:50:20.195509+0000","last_active":"2024-05-09T10:50:20.202017+0000","last_peered":"2024-05-09T10:50:20.202017+0000","last_clean":"2024-05-09T10:50:20.202017+0000","last_became_active":"2024-05-09T10:50:20.195045+0000","last_became_peered":"2024-05-09T10:50:20.195045+0000","last_unstale":"2024-05-09T10:50:20.202017+0000","last_undegraded":"2024-05-09T10:50:20.202017+0000","last_fullsized":"2024-05-09T10:50:20.202017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:47.420832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889170+0000","last_change":"2024-05-09T10:50:20.198458+0000","last_active":"2024-05-09T10:50:21.889170+0000","last_peered":"2024-05-09T10:50:21.889170+0000","last_clean":"2024-05-09T10:50:21.889170+0000","last_became_active":"2024-05-09T10:50:20.195845+0000","last_became_peered":"2024-05-09T10:50:20.195845+0000","last_unstale":"2024-05-09T10:50:21.889170+0000","last_undegraded":"2024-05-09T10:50:21.889170+0000","last_fullsized":"2024-05-09T10:50:21.889170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:31:09.517362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.187385+0000","last_change":"2024-05-09T10:50:20.179665+0000","last_active":"2024-05-09T10:50:20.187385+0000","last_peered":"2024-05-09T10:50:20.187385+0000","last_clean":"2024-05-09T10:50:20.187385+0000","last_became_active":"2024-05-09T10:50:20.179379+0000","last_became_peered":"2024-05-09T10:50:20.179379+0000","last_unstale":"2024-05-09T10:50:20.187385+0000","last_undegraded":"2024-05-09T10:50:20.187385+0000","last_fullsized":"2024-05-09T10:50:20.187385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:49:06.723077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.888989+0000","last_change":"2024-05-09T10:50:20.199853+0000","last_active":"2024-05-09T10:50:21.888989+0000","last_peered":"2024-05-09T10:50:21.888989+0000","last_clean":"2024-05-09T10:50:21.888989+0000","last_became_active":"2024-05-09T10:50:20.199552+0000","last_became_peered":"2024-05-09T10:50:20.199552+0000","last_unstale":"2024-05-09T10:50:21.888989+0000","last_undegraded":"2024-05-09T10:50:21.888989+0000","last_fullsized":"2024-05-09T10:50:21.888989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:09.785658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.193129+0000","last_change":"2024-05-09T10:50:20.191734+0000","last_active":"2024-05-09T10:50:20.193129+0000","last_peered":"2024-05-09T10:50:20.193129+0000","last_clean":"2024-05-09T10:50:20.193129+0000","last_became_active":"2024-05-09T10:50:20.191484+0000","last_became_peered":"2024-05-09T10:50:20.191484+0000","last_unstale":"2024-05-09T10:50:20.193129+0000","last_undegraded":"2024-05-09T10:50:20.193129+0000","last_fullsized":"2024-05-09T10:50:20.193129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:04:46.158655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889473+0000","last_change":"2024-05-09T10:50:20.199646+0000","last_active":"2024-05-09T10:50:21.889473+0000","last_peered":"2024-05-09T10:50:21.889473+0000","last_clean":"2024-05-09T10:50:21.889473+0000","last_became_active":"2024-05-09T10:50:20.199374+0000","last_became_peered":"2024-05-09T10:50:20.199374+0000","last_unstale":"2024-05-09T10:50:21.889473+0000","last_undegraded":"2024-05-09T10:50:21.889473+0000","last_fullsized":"2024-05-09T10:50:21.889473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:27:14.733561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171869+0000","last_change":"2024-05-09T10:50:20.189694+0000","last_active":"2024-05-09T10:50:22.171869+0000","last_peered":"2024-05-09T10:50:22.171869+0000","last_clean":"2024-05-09T10:50:22.171869+0000","last_became_active":"2024-05-09T10:50:20.189429+0000","last_became_peered":"2024-05-09T10:50:20.189429+0000","last_unstale":"2024-05-09T10:50:22.171869+0000","last_undegraded":"2024-05-09T10:50:22.171869+0000","last_fullsized":"2024-05-09T10:50:22.171869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:50:14.919494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171776+0000","last_change":"2024-05-09T10:50:20.189569+0000","last_active":"2024-05-09T10:50:22.171776+0000","last_peered":"2024-05-09T10:50:22.171776+0000","last_clean":"2024-05-09T10:50:22.171776+0000","last_became_active":"2024-05-09T10:50:20.189272+0000","last_became_peered":"2024-05-09T10:50:20.189272+0000","last_unstale":"2024-05-09T10:50:22.171776+0000","last_undegraded":"2024-05-09T10:50:22.171776+0000","last_fullsized":"2024-05-09T10:50:22.171776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:47.420832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183678+0000","last_change":"2024-05-09T10:50:20.191736+0000","last_active":"2024-05-09T10:50:21.183678+0000","last_peered":"2024-05-09T10:50:21.183678+0000","last_clean":"2024-05-09T10:50:21.183678+0000","last_became_active":"2024-05-09T10:50:20.190652+0000","last_became_peered":"2024-05-09T10:50:20.190652+0000","last_unstale":"2024-05-09T10:50:21.183678+0000","last_undegraded":"2024-05-09T10:50:21.183678+0000","last_fullsized":"2024-05-09T10:50:21.183678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:08:00.575142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.194837+0000","last_change":"2024-05-09T10:50:20.192354+0000","last_active":"2024-05-09T10:50:20.194837+0000","last_peered":"2024-05-09T10:50:20.194837+0000","last_clean":"2024-05-09T10:50:20.194837+0000","last_became_active":"2024-05-09T10:50:20.191736+0000","last_became_peered":"2024-05-09T10:50:20.191736+0000","last_unstale":"2024-05-09T10:50:20.194837+0000","last_undegraded":"2024-05-09T10:50:20.194837+0000","last_fullsized":"2024-05-09T10:50:20.194837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:33:37.505180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182518+0000","last_change":"2024-05-09T10:50:20.184216+0000","last_active":"2024-05-09T10:50:21.182518+0000","last_peered":"2024-05-09T10:50:21.182518+0000","last_clean":"2024-05-09T10:50:21.182518+0000","last_became_active":"2024-05-09T10:50:20.183920+0000","last_became_peered":"2024-05-09T10:50:20.183920+0000","last_unstale":"2024-05-09T10:50:21.182518+0000","last_undegraded":"2024-05-09T10:50:21.182518+0000","last_fullsized":"2024-05-09T10:50:21.182518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:57:07.297876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183244+0000","last_change":"2024-05-09T10:50:20.189990+0000","last_active":"2024-05-09T10:50:21.183244+0000","last_peered":"2024-05-09T10:50:21.183244+0000","last_clean":"2024-05-09T10:50:21.183244+0000","last_became_active":"2024-05-09T10:50:20.189727+0000","last_became_peered":"2024-05-09T10:50:20.189727+0000","last_unstale":"2024-05-09T10:50:21.183244+0000","last_undegraded":"2024-05-09T10:50:21.183244+0000","last_fullsized":"2024-05-09T10:50:21.183244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:34:20.711637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.186934+0000","last_change":"2024-05-09T10:50:20.179735+0000","last_active":"2024-05-09T10:50:20.186934+0000","last_peered":"2024-05-09T10:50:20.186934+0000","last_clean":"2024-05-09T10:50:20.186934+0000","last_became_active":"2024-05-09T10:50:20.179440+0000","last_became_peered":"2024-05-09T10:50:20.179440+0000","last_unstale":"2024-05-09T10:50:20.186934+0000","last_undegraded":"2024-05-09T10:50:20.186934+0000","last_fullsized":"2024-05-09T10:50:20.186934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:08:00.575142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.201963+0000","last_change":"2024-05-09T10:50:20.194956+0000","last_active":"2024-05-09T10:50:20.201963+0000","last_peered":"2024-05-09T10:50:20.201963+0000","last_clean":"2024-05-09T10:50:20.201963+0000","last_became_active":"2024-05-09T10:50:20.194660+0000","last_became_peered":"2024-05-09T10:50:20.194660+0000","last_unstale":"2024-05-09T10:50:20.201963+0000","last_undegraded":"2024-05-09T10:50:20.201963+0000","last_fullsized":"2024-05-09T10:50:20.201963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:23.176619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.189494+0000","last_change":"2024-05-09T10:50:20.188140+0000","last_active":"2024-05-09T10:50:20.189494+0000","last_peered":"2024-05-09T10:50:20.189494+0000","last_clean":"2024-05-09T10:50:20.189494+0000","last_became_active":"2024-05-09T10:50:20.187855+0000","last_became_peered":"2024-05-09T10:50:20.187855+0000","last_unstale":"2024-05-09T10:50:20.189494+0000","last_undegraded":"2024-05-09T10:50:20.189494+0000","last_fullsized":"2024-05-09T10:50:20.189494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:38.767540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889478+0000","last_change":"2024-05-09T10:50:20.199567+0000","last_active":"2024-05-09T10:50:21.889478+0000","last_peered":"2024-05-09T10:50:21.889478+0000","last_clean":"2024-05-09T10:50:21.889478+0000","last_became_active":"2024-05-09T10:50:20.198671+0000","last_became_peered":"2024-05-09T10:50:20.198671+0000","last_unstale":"2024-05-09T10:50:21.889478+0000","last_undegraded":"2024-05-09T10:50:21.889478+0000","last_fullsized":"2024-05-09T10:50:21.889478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:03:14.102727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171768+0000","last_change":"2024-05-09T10:50:20.188963+0000","last_active":"2024-05-09T10:50:22.171768+0000","last_peered":"2024-05-09T10:50:22.171768+0000","last_clean":"2024-05-09T10:50:22.171768+0000","last_became_active":"2024-05-09T10:50:20.188654+0000","last_became_peered":"2024-05-09T10:50:20.188654+0000","last_unstale":"2024-05-09T10:50:22.171768+0000","last_undegraded":"2024-05-09T10:50:22.171768+0000","last_fullsized":"2024-05-09T10:50:22.171768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:23.176619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182825+0000","last_change":"2024-05-09T10:50:20.188028+0000","last_active":"2024-05-09T10:50:21.182825+0000","last_peered":"2024-05-09T10:50:21.182825+0000","last_clean":"2024-05-09T10:50:21.182825+0000","last_became_active":"2024-05-09T10:50:20.186949+0000","last_became_peered":"2024-05-09T10:50:20.186949+0000","last_unstale":"2024-05-09T10:50:21.182825+0000","last_undegraded":"2024-05-09T10:50:21.182825+0000","last_fullsized":"2024-05-09T10:50:21.182825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:00:03.184256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.193437+0000","last_change":"2024-05-09T10:50:20.187160+0000","last_active":"2024-05-09T10:50:20.193437+0000","last_peered":"2024-05-09T10:50:20.193437+0000","last_clean":"2024-05-09T10:50:20.193437+0000","last_became_active":"2024-05-09T10:50:20.186840+0000","last_became_peered":"2024-05-09T10:50:20.186840+0000","last_unstale":"2024-05-09T10:50:20.193437+0000","last_undegraded":"2024-05-09T10:50:20.193437+0000","last_fullsized":"2024-05-09T10:50:20.193437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:57:07.297876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.192016+0000","last_change":"2024-05-09T10:50:20.188532+0000","last_active":"2024-05-09T10:50:20.192016+0000","last_peered":"2024-05-09T10:50:20.192016+0000","last_clean":"2024-05-09T10:50:20.192016+0000","last_became_active":"2024-05-09T10:50:20.188264+0000","last_became_peered":"2024-05-09T10:50:20.188264+0000","last_unstale":"2024-05-09T10:50:20.192016+0000","last_undegraded":"2024-05-09T10:50:20.192016+0000","last_fullsized":"2024-05-09T10:50:20.192016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:21.666208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.201949+0000","last_change":"2024-05-09T10:50:20.194256+0000","last_active":"2024-05-09T10:50:20.201949+0000","last_peered":"2024-05-09T10:50:20.201949+0000","last_clean":"2024-05-09T10:50:20.201949+0000","last_became_active":"2024-05-09T10:50:20.193137+0000","last_became_peered":"2024-05-09T10:50:20.193137+0000","last_unstale":"2024-05-09T10:50:20.201949+0000","last_undegraded":"2024-05-09T10:50:20.201949+0000","last_fullsized":"2024-05-09T10:50:20.201949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:45.067992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.192022+0000","last_change":"2024-05-09T10:50:20.189585+0000","last_active":"2024-05-09T10:50:20.192022+0000","last_peered":"2024-05-09T10:50:20.192022+0000","last_clean":"2024-05-09T10:50:20.192022+0000","last_became_active":"2024-05-09T10:50:20.189295+0000","last_became_peered":"2024-05-09T10:50:20.189295+0000","last_unstale":"2024-05-09T10:50:20.192022+0000","last_undegraded":"2024-05-09T10:50:20.192022+0000","last_fullsized":"2024-05-09T10:50:20.192022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:49:06.723077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171966+0000","last_change":"2024-05-09T10:50:20.196303+0000","last_active":"2024-05-09T10:50:22.171966+0000","last_peered":"2024-05-09T10:50:22.171966+0000","last_clean":"2024-05-09T10:50:22.171966+0000","last_became_active":"2024-05-09T10:50:20.195986+0000","last_became_peered":"2024-05-09T10:50:20.195986+0000","last_unstale":"2024-05-09T10:50:22.171966+0000","last_undegraded":"2024-05-09T10:50:22.171966+0000","last_fullsized":"2024-05-09T10:50:22.171966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:34:20.711637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182978+0000","last_change":"2024-05-09T10:50:20.193168+0000","last_active":"2024-05-09T10:50:21.182978+0000","last_peered":"2024-05-09T10:50:21.182978+0000","last_clean":"2024-05-09T10:50:21.182978+0000","last_became_active":"2024-05-09T10:50:20.192898+0000","last_became_peered":"2024-05-09T10:50:20.192898+0000","last_unstale":"2024-05-09T10:50:21.182978+0000","last_undegraded":"2024-05-09T10:50:21.182978+0000","last_fullsized":"2024-05-09T10:50:21.182978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:50:14.919494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.192175+0000","last_change":"2024-05-09T10:50:20.188735+0000","last_active":"2024-05-09T10:50:20.192175+0000","last_peered":"2024-05-09T10:50:20.192175+0000","last_clean":"2024-05-09T10:50:20.192175+0000","last_became_active":"2024-05-09T10:50:20.188138+0000","last_became_peered":"2024-05-09T10:50:20.188138+0000","last_unstale":"2024-05-09T10:50:20.192175+0000","last_undegraded":"2024-05-09T10:50:20.192175+0000","last_fullsized":"2024-05-09T10:50:20.192175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:38.767540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182945+0000","last_change":"2024-05-09T10:50:20.194279+0000","last_active":"2024-05-09T10:50:21.182945+0000","last_peered":"2024-05-09T10:50:21.182945+0000","last_clean":"2024-05-09T10:50:21.182945+0000","last_became_active":"2024-05-09T10:50:20.193962+0000","last_became_peered":"2024-05-09T10:50:20.193962+0000","last_unstale":"2024-05-09T10:50:21.182945+0000","last_undegraded":"2024-05-09T10:50:21.182945+0000","last_fullsized":"2024-05-09T10:50:21.182945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:23.176619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.201664+0000","last_change":"2024-05-09T10:50:20.193350+0000","last_active":"2024-05-09T10:50:20.201664+0000","last_peered":"2024-05-09T10:50:20.201664+0000","last_clean":"2024-05-09T10:50:20.201664+0000","last_became_active":"2024-05-09T10:50:20.192514+0000","last_became_peered":"2024-05-09T10:50:20.192514+0000","last_unstale":"2024-05-09T10:50:20.201664+0000","last_undegraded":"2024-05-09T10:50:20.201664+0000","last_fullsized":"2024-05-09T10:50:20.201664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:32:42.671953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.187527+0000","last_change":"2024-05-09T10:50:20.175999+0000","last_active":"2024-05-09T10:50:20.187527+0000","last_peered":"2024-05-09T10:50:20.187527+0000","last_clean":"2024-05-09T10:50:20.187527+0000","last_became_active":"2024-05-09T10:50:20.175695+0000","last_became_peered":"2024-05-09T10:50:20.175695+0000","last_unstale":"2024-05-09T10:50:20.187527+0000","last_undegraded":"2024-05-09T10:50:20.187527+0000","last_fullsized":"2024-05-09T10:50:20.187527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:40.421893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171539+0000","last_change":"2024-05-09T10:50:20.196639+0000","last_active":"2024-05-09T10:50:22.171539+0000","last_peered":"2024-05-09T10:50:22.171539+0000","last_clean":"2024-05-09T10:50:22.171539+0000","last_became_active":"2024-05-09T10:50:20.196365+0000","last_became_peered":"2024-05-09T10:50:20.196365+0000","last_unstale":"2024-05-09T10:50:22.171539+0000","last_undegraded":"2024-05-09T10:50:22.171539+0000","last_fullsized":"2024-05-09T10:50:22.171539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:32:42.671953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183207+0000","last_change":"2024-05-09T10:50:20.185161+0000","last_active":"2024-05-09T10:50:21.183207+0000","last_peered":"2024-05-09T10:50:21.183207+0000","last_clean":"2024-05-09T10:50:21.183207+0000","last_became_active":"2024-05-09T10:50:20.184832+0000","last_became_peered":"2024-05-09T10:50:20.184832+0000","last_unstale":"2024-05-09T10:50:21.183207+0000","last_undegraded":"2024-05-09T10:50:21.183207+0000","last_fullsized":"2024-05-09T10:50:21.183207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:21.666208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.194594+0000","last_change":"2024-05-09T10:50:19.173345+0000","last_active":"2024-05-09T10:50:20.194594+0000","last_peered":"2024-05-09T10:50:20.194594+0000","last_clean":"2024-05-09T10:50:20.194594+0000","last_became_active":"2024-05-09T10:50:19.173059+0000","last_became_peered":"2024-05-09T10:50:19.173059+0000","last_unstale":"2024-05-09T10:50:20.194594+0000","last_undegraded":"2024-05-09T10:50:20.194594+0000","last_fullsized":"2024-05-09T10:50:20.194594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:53.042012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183068+0000","last_change":"2024-05-09T10:50:20.188071+0000","last_active":"2024-05-09T10:50:21.183068+0000","last_peered":"2024-05-09T10:50:21.183068+0000","last_clean":"2024-05-09T10:50:21.183068+0000","last_became_active":"2024-05-09T10:50:20.187766+0000","last_became_peered":"2024-05-09T10:50:20.187766+0000","last_unstale":"2024-05-09T10:50:21.183068+0000","last_undegraded":"2024-05-09T10:50:21.183068+0000","last_fullsized":"2024-05-09T10:50:21.183068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:47.420832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171634+0000","last_change":"2024-05-09T10:50:19.169316+0000","last_active":"2024-05-09T10:50:22.171634+0000","last_peered":"2024-05-09T10:50:22.171634+0000","last_clean":"2024-05-09T10:50:22.171634+0000","last_became_active":"2024-05-09T10:50:19.169040+0000","last_became_peered":"2024-05-09T10:50:19.169040+0000","last_unstale":"2024-05-09T10:50:22.171634+0000","last_undegraded":"2024-05-09T10:50:22.171634+0000","last_fullsized":"2024-05-09T10:50:22.171634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:39.416778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.179196+0000","last_change":"2024-05-09T10:50:19.167276+0000","last_active":"2024-05-09T10:50:20.179196+0000","last_peered":"2024-05-09T10:50:20.179196+0000","last_clean":"2024-05-09T10:50:20.179196+0000","last_became_active":"2024-05-09T10:50:19.166775+0000","last_became_peered":"2024-05-09T10:50:19.166775+0000","last_unstale":"2024-05-09T10:50:20.179196+0000","last_undegraded":"2024-05-09T10:50:20.179196+0000","last_fullsized":"2024-05-09T10:50:20.179196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:41:57.513979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.187906+0000","last_change":"2024-05-09T10:50:20.181449+0000","last_active":"2024-05-09T10:50:20.187906+0000","last_peered":"2024-05-09T10:50:20.187906+0000","last_clean":"2024-05-09T10:50:20.187906+0000","last_became_active":"2024-05-09T10:50:20.181137+0000","last_became_peered":"2024-05-09T10:50:20.181137+0000","last_unstale":"2024-05-09T10:50:20.187906+0000","last_undegraded":"2024-05-09T10:50:20.187906+0000","last_fullsized":"2024-05-09T10:50:20.187906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:18:18.206959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.172164+0000","last_change":"2024-05-09T10:50:19.165083+0000","last_active":"2024-05-09T10:50:20.172164+0000","last_peered":"2024-05-09T10:50:20.172164+0000","last_clean":"2024-05-09T10:50:20.172164+0000","last_became_active":"2024-05-09T10:50:19.164468+0000","last_became_peered":"2024-05-09T10:50:19.164468+0000","last_unstale":"2024-05-09T10:50:20.172164+0000","last_undegraded":"2024-05-09T10:50:20.172164+0000","last_fullsized":"2024-05-09T10:50:20.172164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:53.042012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182977+0000","last_change":"2024-05-09T10:50:20.193953+0000","last_active":"2024-05-09T10:50:21.182977+0000","last_peered":"2024-05-09T10:50:21.182977+0000","last_clean":"2024-05-09T10:50:21.182977+0000","last_became_active":"2024-05-09T10:50:20.193686+0000","last_became_peered":"2024-05-09T10:50:20.193686+0000","last_unstale":"2024-05-09T10:50:21.182977+0000","last_undegraded":"2024-05-09T10:50:21.182977+0000","last_fullsized":"2024-05-09T10:50:21.182977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:47.420832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171830+0000","last_change":"2024-05-09T10:50:19.164495+0000","last_active":"2024-05-09T10:50:22.171830+0000","last_peered":"2024-05-09T10:50:22.171830+0000","last_clean":"2024-05-09T10:50:22.171830+0000","last_became_active":"2024-05-09T10:50:19.164073+0000","last_became_peered":"2024-05-09T10:50:19.164073+0000","last_unstale":"2024-05-09T10:50:22.171830+0000","last_undegraded":"2024-05-09T10:50:22.171830+0000","last_fullsized":"2024-05-09T10:50:22.171830+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:53.042012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.188792+0000","last_change":"2024-05-09T10:50:20.186184+0000","last_active":"2024-05-09T10:50:20.188792+0000","last_peered":"2024-05-09T10:50:20.188792+0000","last_clean":"2024-05-09T10:50:20.188792+0000","last_became_active":"2024-05-09T10:50:20.185892+0000","last_became_peered":"2024-05-09T10:50:20.185892+0000","last_unstale":"2024-05-09T10:50:20.188792+0000","last_undegraded":"2024-05-09T10:50:20.188792+0000","last_fullsized":"2024-05-09T10:50:20.188792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:33:37.505180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.173508+0000","last_change":"2024-05-09T10:50:19.166015+0000","last_active":"2024-05-09T10:50:20.173508+0000","last_peered":"2024-05-09T10:50:20.173508+0000","last_clean":"2024-05-09T10:50:20.173508+0000","last_became_active":"2024-05-09T10:50:19.163525+0000","last_became_peered":"2024-05-09T10:50:19.163525+0000","last_unstale":"2024-05-09T10:50:20.173508+0000","last_undegraded":"2024-05-09T10:50:20.173508+0000","last_fullsized":"2024-05-09T10:50:20.173508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:37.762424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183206+0000","last_change":"2024-05-09T10:50:20.183493+0000","last_active":"2024-05-09T10:50:21.183206+0000","last_peered":"2024-05-09T10:50:21.183206+0000","last_clean":"2024-05-09T10:50:21.183206+0000","last_became_active":"2024-05-09T10:50:20.183222+0000","last_became_peered":"2024-05-09T10:50:20.183222+0000","last_unstale":"2024-05-09T10:50:21.183206+0000","last_undegraded":"2024-05-09T10:50:21.183206+0000","last_fullsized":"2024-05-09T10:50:21.183206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:34:20.711637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183073+0000","last_change":"2024-05-09T10:50:19.174608+0000","last_active":"2024-05-09T10:50:21.183073+0000","last_peered":"2024-05-09T10:50:21.183073+0000","last_clean":"2024-05-09T10:50:21.183073+0000","last_became_active":"2024-05-09T10:50:19.174299+0000","last_became_peered":"2024-05-09T10:50:19.174299+0000","last_unstale":"2024-05-09T10:50:21.183073+0000","last_undegraded":"2024-05-09T10:50:21.183073+0000","last_fullsized":"2024-05-09T10:50:21.183073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:49:05.717962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183616+0000","last_change":"2024-05-09T10:50:20.186118+0000","last_active":"2024-05-09T10:50:21.183616+0000","last_peered":"2024-05-09T10:50:21.183616+0000","last_clean":"2024-05-09T10:50:21.183616+0000","last_became_active":"2024-05-09T10:50:20.185818+0000","last_became_peered":"2024-05-09T10:50:20.185818+0000","last_unstale":"2024-05-09T10:50:21.183616+0000","last_undegraded":"2024-05-09T10:50:21.183616+0000","last_fullsized":"2024-05-09T10:50:21.183616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:49:06.723077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171706+0000","last_change":"2024-05-09T10:50:19.170262+0000","last_active":"2024-05-09T10:50:22.171706+0000","last_peered":"2024-05-09T10:50:22.171706+0000","last_clean":"2024-05-09T10:50:22.171706+0000","last_became_active":"2024-05-09T10:50:19.169908+0000","last_became_peered":"2024-05-09T10:50:19.169908+0000","last_unstale":"2024-05-09T10:50:22.171706+0000","last_undegraded":"2024-05-09T10:50:22.171706+0000","last_fullsized":"2024-05-09T10:50:22.171706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:37.762424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889300+0000","last_change":"2024-05-09T10:50:20.199969+0000","last_active":"2024-05-09T10:50:21.889300+0000","last_peered":"2024-05-09T10:50:21.889300+0000","last_clean":"2024-05-09T10:50:21.889300+0000","last_became_active":"2024-05-09T10:50:20.199060+0000","last_became_peered":"2024-05-09T10:50:20.199060+0000","last_unstale":"2024-05-09T10:50:21.889300+0000","last_undegraded":"2024-05-09T10:50:21.889300+0000","last_fullsized":"2024-05-09T10:50:21.889300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:35.361899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183103+0000","last_change":"2024-05-09T10:50:19.176037+0000","last_active":"2024-05-09T10:50:21.183103+0000","last_peered":"2024-05-09T10:50:21.183103+0000","last_clean":"2024-05-09T10:50:21.183103+0000","last_became_active":"2024-05-09T10:50:19.170706+0000","last_became_peered":"2024-05-09T10:50:19.170706+0000","last_unstale":"2024-05-09T10:50:21.183103+0000","last_undegraded":"2024-05-09T10:50:21.183103+0000","last_fullsized":"2024-05-09T10:50:21.183103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:54:23.668155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182651+0000","last_change":"2024-05-09T10:50:20.194758+0000","last_active":"2024-05-09T10:50:21.182651+0000","last_peered":"2024-05-09T10:50:21.182651+0000","last_clean":"2024-05-09T10:50:21.182651+0000","last_became_active":"2024-05-09T10:50:20.194509+0000","last_became_peered":"2024-05-09T10:50:20.194509+0000","last_unstale":"2024-05-09T10:50:21.182651+0000","last_undegraded":"2024-05-09T10:50:21.182651+0000","last_fullsized":"2024-05-09T10:50:21.182651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:32:42.671953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.173258+0000","last_change":"2024-05-09T10:50:19.167487+0000","last_active":"2024-05-09T10:50:20.173258+0000","last_peered":"2024-05-09T10:50:20.173258+0000","last_clean":"2024-05-09T10:50:20.173258+0000","last_became_active":"2024-05-09T10:50:19.167139+0000","last_became_peered":"2024-05-09T10:50:19.167139+0000","last_unstale":"2024-05-09T10:50:20.173258+0000","last_undegraded":"2024-05-09T10:50:20.173258+0000","last_fullsized":"2024-05-09T10:50:20.173258+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:20.661093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889232+0000","last_change":"2024-05-09T10:50:20.198511+0000","last_active":"2024-05-09T10:50:21.889232+0000","last_peered":"2024-05-09T10:50:21.889232+0000","last_clean":"2024-05-09T10:50:21.889232+0000","last_became_active":"2024-05-09T10:50:20.198165+0000","last_became_peered":"2024-05-09T10:50:20.198165+0000","last_unstale":"2024-05-09T10:50:21.889232+0000","last_undegraded":"2024-05-09T10:50:21.889232+0000","last_fullsized":"2024-05-09T10:50:21.889232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:47:01.454064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171521+0000","last_change":"2024-05-09T10:50:19.173828+0000","last_active":"2024-05-09T10:50:22.171521+0000","last_peered":"2024-05-09T10:50:22.171521+0000","last_clean":"2024-05-09T10:50:22.171521+0000","last_became_active":"2024-05-09T10:50:19.173524+0000","last_became_peered":"2024-05-09T10:50:19.173524+0000","last_unstale":"2024-05-09T10:50:22.171521+0000","last_undegraded":"2024-05-09T10:50:22.171521+0000","last_fullsized":"2024-05-09T10:50:22.171521+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:18:17.201844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171530+0000","last_change":"2024-05-09T10:50:20.196631+0000","last_active":"2024-05-09T10:50:22.171530+0000","last_peered":"2024-05-09T10:50:22.171530+0000","last_clean":"2024-05-09T10:50:22.171530+0000","last_became_active":"2024-05-09T10:50:20.196252+0000","last_became_peered":"2024-05-09T10:50:20.196252+0000","last_unstale":"2024-05-09T10:50:22.171530+0000","last_undegraded":"2024-05-09T10:50:22.171530+0000","last_fullsized":"2024-05-09T10:50:22.171530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:00:03.184256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.187578+0000","last_change":"2024-05-09T10:50:19.173070+0000","last_active":"2024-05-09T10:50:20.187578+0000","last_peered":"2024-05-09T10:50:20.187578+0000","last_clean":"2024-05-09T10:50:20.187578+0000","last_became_active":"2024-05-09T10:50:19.172711+0000","last_became_peered":"2024-05-09T10:50:19.172711+0000","last_unstale":"2024-05-09T10:50:20.187578+0000","last_undegraded":"2024-05-09T10:50:20.187578+0000","last_fullsized":"2024-05-09T10:50:20.187578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:53.042012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889056+0000","last_change":"2024-05-09T10:50:20.197201+0000","last_active":"2024-05-09T10:50:21.889056+0000","last_peered":"2024-05-09T10:50:21.889056+0000","last_clean":"2024-05-09T10:50:21.889056+0000","last_became_active":"2024-05-09T10:50:20.196926+0000","last_became_peered":"2024-05-09T10:50:20.196926+0000","last_unstale":"2024-05-09T10:50:21.889056+0000","last_undegraded":"2024-05-09T10:50:21.889056+0000","last_fullsized":"2024-05-09T10:50:21.889056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:39.852439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183547+0000","last_change":"2024-05-09T10:50:19.179678+0000","last_active":"2024-05-09T10:50:21.183547+0000","last_peered":"2024-05-09T10:50:21.183547+0000","last_clean":"2024-05-09T10:50:21.183547+0000","last_became_active":"2024-05-09T10:50:19.179419+0000","last_became_peered":"2024-05-09T10:50:19.179419+0000","last_unstale":"2024-05-09T10:50:21.183547+0000","last_undegraded":"2024-05-09T10:50:21.183547+0000","last_fullsized":"2024-05-09T10:50:21.183547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:38:13.651797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889436+0000","last_change":"2024-05-09T10:50:20.197970+0000","last_active":"2024-05-09T10:50:21.889436+0000","last_peered":"2024-05-09T10:50:21.889436+0000","last_clean":"2024-05-09T10:50:21.889436+0000","last_became_active":"2024-05-09T10:50:20.197653+0000","last_became_peered":"2024-05-09T10:50:20.197653+0000","last_unstale":"2024-05-09T10:50:21.889436+0000","last_undegraded":"2024-05-09T10:50:21.889436+0000","last_fullsized":"2024-05-09T10:50:21.889436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:07:48.935082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.186916+0000","last_change":"2024-05-09T10:50:19.174326+0000","last_active":"2024-05-09T10:50:20.186916+0000","last_peered":"2024-05-09T10:50:20.186916+0000","last_clean":"2024-05-09T10:50:20.186916+0000","last_became_active":"2024-05-09T10:50:19.173957+0000","last_became_peered":"2024-05-09T10:50:19.173957+0000","last_unstale":"2024-05-09T10:50:20.186916+0000","last_undegraded":"2024-05-09T10:50:20.186916+0000","last_fullsized":"2024-05-09T10:50:20.186916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:20.661093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889578+0000","last_change":"2024-05-09T10:50:20.199478+0000","last_active":"2024-05-09T10:50:21.889578+0000","last_peered":"2024-05-09T10:50:21.889578+0000","last_clean":"2024-05-09T10:50:21.889578+0000","last_became_active":"2024-05-09T10:50:20.199083+0000","last_became_peered":"2024-05-09T10:50:20.199083+0000","last_unstale":"2024-05-09T10:50:21.889578+0000","last_undegraded":"2024-05-09T10:50:21.889578+0000","last_fullsized":"2024-05-09T10:50:21.889578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:34:17.252316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183186+0000","last_change":"2024-05-09T10:50:19.173709+0000","last_active":"2024-05-09T10:50:21.183186+0000","last_peered":"2024-05-09T10:50:21.183186+0000","last_clean":"2024-05-09T10:50:21.183186+0000","last_became_active":"2024-05-09T10:50:19.173453+0000","last_became_peered":"2024-05-09T10:50:19.173453+0000","last_unstale":"2024-05-09T10:50:21.183186+0000","last_undegraded":"2024-05-09T10:50:21.183186+0000","last_fullsized":"2024-05-09T10:50:21.183186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:07:59.570026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.201722+0000","last_change":"2024-05-09T10:50:20.193882+0000","last_active":"2024-05-09T10:50:20.201722+0000","last_peered":"2024-05-09T10:50:20.201722+0000","last_clean":"2024-05-09T10:50:20.201722+0000","last_became_active":"2024-05-09T10:50:20.193186+0000","last_became_peered":"2024-05-09T10:50:20.193186+0000","last_unstale":"2024-05-09T10:50:20.201722+0000","last_undegraded":"2024-05-09T10:50:20.201722+0000","last_fullsized":"2024-05-09T10:50:20.201722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:00:03.184256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.169440+0000","last_change":"2024-05-09T10:50:19.163763+0000","last_active":"2024-05-09T10:50:20.169440+0000","last_peered":"2024-05-09T10:50:20.169440+0000","last_clean":"2024-05-09T10:50:20.169440+0000","last_became_active":"2024-05-09T10:50:19.163429+0000","last_became_peered":"2024-05-09T10:50:19.163429+0000","last_unstale":"2024-05-09T10:50:20.169440+0000","last_undegraded":"2024-05-09T10:50:20.169440+0000","last_fullsized":"2024-05-09T10:50:20.169440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:41:57.513979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889302+0000","last_change":"2024-05-09T10:50:20.200705+0000","last_active":"2024-05-09T10:50:21.889302+0000","last_peered":"2024-05-09T10:50:21.889302+0000","last_clean":"2024-05-09T10:50:21.889302+0000","last_became_active":"2024-05-09T10:50:20.200416+0000","last_became_peered":"2024-05-09T10:50:20.200416+0000","last_unstale":"2024-05-09T10:50:21.889302+0000","last_undegraded":"2024-05-09T10:50:21.889302+0000","last_fullsized":"2024-05-09T10:50:21.889302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:15.043180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.888962+0000","last_change":"2024-05-09T10:50:08.410364+0000","last_active":"2024-05-09T10:50:21.888962+0000","last_peered":"2024-05-09T10:50:21.888962+0000","last_clean":"2024-05-09T10:50:21.888962+0000","last_became_active":"2024-05-09T10:50:08.409457+0000","last_became_peered":"2024-05-09T10:50:08.409457+0000","last_unstale":"2024-05-09T10:50:21.888962+0000","last_undegraded":"2024-05-09T10:50:21.888962+0000","last_fullsized":"2024-05-09T10:50:21.888962+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_clean_scrub_stamp":"2024-05-09T10:50:07.396510+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:03.566698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.171374+0000","last_change":"2024-05-09T10:50:19.158709+0000","last_active":"2024-05-09T10:50:20.171374+0000","last_peered":"2024-05-09T10:50:20.171374+0000","last_clean":"2024-05-09T10:50:20.171374+0000","last_became_active":"2024-05-09T10:50:19.158385+0000","last_became_peered":"2024-05-09T10:50:19.158385+0000","last_unstale":"2024-05-09T10:50:20.171374+0000","last_undegraded":"2024-05-09T10:50:20.171374+0000","last_fullsized":"2024-05-09T10:50:20.171374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:18:17.201844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.192390+0000","last_change":"2024-05-09T10:50:20.190695+0000","last_active":"2024-05-09T10:50:20.192390+0000","last_peered":"2024-05-09T10:50:20.192390+0000","last_clean":"2024-05-09T10:50:20.192390+0000","last_became_active":"2024-05-09T10:50:20.190446+0000","last_became_peered":"2024-05-09T10:50:20.190446+0000","last_unstale":"2024-05-09T10:50:20.192390+0000","last_undegraded":"2024-05-09T10:50:20.192390+0000","last_fullsized":"2024-05-09T10:50:20.192390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:16:53.166077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889057+0000","last_change":"2024-05-09T10:50:19.173033+0000","last_active":"2024-05-09T10:50:21.889057+0000","last_peered":"2024-05-09T10:50:21.889057+0000","last_clean":"2024-05-09T10:50:21.889057+0000","last_became_active":"2024-05-09T10:50:19.172691+0000","last_became_peered":"2024-05-09T10:50:19.172691+0000","last_unstale":"2024-05-09T10:50:21.889057+0000","last_undegraded":"2024-05-09T10:50:21.889057+0000","last_fullsized":"2024-05-09T10:50:21.889057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:49:05.717962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182430+0000","last_change":"2024-05-09T10:50:20.186680+0000","last_active":"2024-05-09T10:50:21.182430+0000","last_peered":"2024-05-09T10:50:21.182430+0000","last_clean":"2024-05-09T10:50:21.182430+0000","last_became_active":"2024-05-09T10:50:20.186413+0000","last_became_peered":"2024-05-09T10:50:20.186413+0000","last_unstale":"2024-05-09T10:50:21.182430+0000","last_undegraded":"2024-05-09T10:50:21.182430+0000","last_fullsized":"2024-05-09T10:50:21.182430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:33:37.505180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.169724+0000","last_change":"2024-05-09T10:50:19.166130+0000","last_active":"2024-05-09T10:50:20.169724+0000","last_peered":"2024-05-09T10:50:20.169724+0000","last_clean":"2024-05-09T10:50:20.169724+0000","last_became_active":"2024-05-09T10:50:19.165820+0000","last_became_peered":"2024-05-09T10:50:19.165820+0000","last_unstale":"2024-05-09T10:50:20.169724+0000","last_undegraded":"2024-05-09T10:50:20.169724+0000","last_fullsized":"2024-05-09T10:50:20.169724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:18:17.201844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182893+0000","last_change":"2024-05-09T10:50:20.190864+0000","last_active":"2024-05-09T10:50:21.182893+0000","last_peered":"2024-05-09T10:50:21.182893+0000","last_clean":"2024-05-09T10:50:21.182893+0000","last_became_active":"2024-05-09T10:50:20.190613+0000","last_became_peered":"2024-05-09T10:50:20.190613+0000","last_unstale":"2024-05-09T10:50:21.182893+0000","last_undegraded":"2024-05-09T10:50:21.182893+0000","last_fullsized":"2024-05-09T10:50:21.182893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:23.176619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.173434+0000","last_change":"2024-05-09T10:50:19.173006+0000","last_active":"2024-05-09T10:50:20.173434+0000","last_peered":"2024-05-09T10:50:20.173434+0000","last_clean":"2024-05-09T10:50:20.173434+0000","last_became_active":"2024-05-09T10:50:19.172607+0000","last_became_peered":"2024-05-09T10:50:19.172607+0000","last_unstale":"2024-05-09T10:50:20.173434+0000","last_undegraded":"2024-05-09T10:50:20.173434+0000","last_fullsized":"2024-05-09T10:50:20.173434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:39.416778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183468+0000","last_change":"2024-05-09T10:50:20.192361+0000","last_active":"2024-05-09T10:50:21.183468+0000","last_peered":"2024-05-09T10:50:21.183468+0000","last_clean":"2024-05-09T10:50:21.183468+0000","last_became_active":"2024-05-09T10:50:20.192116+0000","last_became_peered":"2024-05-09T10:50:20.192116+0000","last_unstale":"2024-05-09T10:50:21.183468+0000","last_undegraded":"2024-05-09T10:50:21.183468+0000","last_fullsized":"2024-05-09T10:50:21.183468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:38.767540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182627+0000","last_change":"2024-05-09T10:50:19.176926+0000","last_active":"2024-05-09T10:50:21.182627+0000","last_peered":"2024-05-09T10:50:21.182627+0000","last_clean":"2024-05-09T10:50:21.182627+0000","last_became_active":"2024-05-09T10:50:19.176627+0000","last_became_peered":"2024-05-09T10:50:19.176627+0000","last_unstale":"2024-05-09T10:50:21.182627+0000","last_undegraded":"2024-05-09T10:50:21.182627+0000","last_fullsized":"2024-05-09T10:50:21.182627+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:37.762424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.201641+0000","last_change":"2024-05-09T10:50:20.193387+0000","last_active":"2024-05-09T10:50:20.201641+0000","last_peered":"2024-05-09T10:50:20.201641+0000","last_clean":"2024-05-09T10:50:20.201641+0000","last_became_active":"2024-05-09T10:50:20.193115+0000","last_became_peered":"2024-05-09T10:50:20.193115+0000","last_unstale":"2024-05-09T10:50:20.201641+0000","last_undegraded":"2024-05-09T10:50:20.201641+0000","last_fullsized":"2024-05-09T10:50:20.201641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:30.799640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889536+0000","last_change":"2024-05-09T10:50:19.170801+0000","last_active":"2024-05-09T10:50:21.889536+0000","last_peered":"2024-05-09T10:50:21.889536+0000","last_clean":"2024-05-09T10:50:21.889536+0000","last_became_active":"2024-05-09T10:50:19.170130+0000","last_became_peered":"2024-05-09T10:50:19.170130+0000","last_unstale":"2024-05-09T10:50:21.889536+0000","last_undegraded":"2024-05-09T10:50:21.889536+0000","last_fullsized":"2024-05-09T10:50:21.889536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:04:45.153539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182544+0000","last_change":"2024-05-09T10:50:20.189228+0000","last_active":"2024-05-09T10:50:21.182544+0000","last_peered":"2024-05-09T10:50:21.182544+0000","last_clean":"2024-05-09T10:50:21.182544+0000","last_became_active":"2024-05-09T10:50:20.188969+0000","last_became_peered":"2024-05-09T10:50:20.188969+0000","last_unstale":"2024-05-09T10:50:21.182544+0000","last_undegraded":"2024-05-09T10:50:21.182544+0000","last_fullsized":"2024-05-09T10:50:21.182544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:00:03.184256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.170973+0000","last_change":"2024-05-09T10:50:19.171062+0000","last_active":"2024-05-09T10:50:22.170973+0000","last_peered":"2024-05-09T10:50:22.170973+0000","last_clean":"2024-05-09T10:50:22.170973+0000","last_became_active":"2024-05-09T10:50:19.170760+0000","last_became_peered":"2024-05-09T10:50:19.170760+0000","last_unstale":"2024-05-09T10:50:22.170973+0000","last_undegraded":"2024-05-09T10:50:22.170973+0000","last_fullsized":"2024-05-09T10:50:22.170973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:39.416778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.194442+0000","last_change":"2024-05-09T10:50:20.193239+0000","last_active":"2024-05-09T10:50:20.194442+0000","last_peered":"2024-05-09T10:50:20.194442+0000","last_clean":"2024-05-09T10:50:20.194442+0000","last_became_active":"2024-05-09T10:50:20.193014+0000","last_became_peered":"2024-05-09T10:50:20.193014+0000","last_unstale":"2024-05-09T10:50:20.194442+0000","last_undegraded":"2024-05-09T10:50:20.194442+0000","last_fullsized":"2024-05-09T10:50:20.194442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:30.799640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171972+0000","last_change":"2024-05-09T10:50:19.171256+0000","last_active":"2024-05-09T10:50:22.171972+0000","last_peered":"2024-05-09T10:50:22.171972+0000","last_clean":"2024-05-09T10:50:22.171972+0000","last_became_active":"2024-05-09T10:50:19.170964+0000","last_became_peered":"2024-05-09T10:50:19.170964+0000","last_unstale":"2024-05-09T10:50:22.171972+0000","last_undegraded":"2024-05-09T10:50:22.171972+0000","last_fullsized":"2024-05-09T10:50:22.171972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:20.661093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182872+0000","last_change":"2024-05-09T10:50:20.184272+0000","last_active":"2024-05-09T10:50:21.182872+0000","last_peered":"2024-05-09T10:50:21.182872+0000","last_clean":"2024-05-09T10:50:21.182872+0000","last_became_active":"2024-05-09T10:50:20.183978+0000","last_became_peered":"2024-05-09T10:50:20.183978+0000","last_unstale":"2024-05-09T10:50:21.182872+0000","last_undegraded":"2024-05-09T10:50:21.182872+0000","last_fullsized":"2024-05-09T10:50:21.182872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:45.067992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.888875+0000","last_change":"2024-05-09T10:50:19.171403+0000","last_active":"2024-05-09T10:50:21.888875+0000","last_peered":"2024-05-09T10:50:21.888875+0000","last_clean":"2024-05-09T10:50:21.888875+0000","last_became_active":"2024-05-09T10:50:19.171106+0000","last_became_peered":"2024-05-09T10:50:19.171106+0000","last_unstale":"2024-05-09T10:50:21.888875+0000","last_undegraded":"2024-05-09T10:50:21.888875+0000","last_fullsized":"2024-05-09T10:50:21.888875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:39.416778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183306+0000","last_change":"2024-05-09T10:50:20.189694+0000","last_active":"2024-05-09T10:50:21.183306+0000","last_peered":"2024-05-09T10:50:21.183306+0000","last_clean":"2024-05-09T10:50:21.183306+0000","last_became_active":"2024-05-09T10:50:20.189423+0000","last_became_peered":"2024-05-09T10:50:20.189423+0000","last_unstale":"2024-05-09T10:50:21.183306+0000","last_undegraded":"2024-05-09T10:50:21.183306+0000","last_fullsized":"2024-05-09T10:50:21.183306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:40.421893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183073+0000","last_change":"2024-05-09T10:50:19.174912+0000","last_active":"2024-05-09T10:50:21.183073+0000","last_peered":"2024-05-09T10:50:21.183073+0000","last_clean":"2024-05-09T10:50:21.183073+0000","last_became_active":"2024-05-09T10:50:19.174640+0000","last_became_peered":"2024-05-09T10:50:19.174640+0000","last_unstale":"2024-05-09T10:50:21.183073+0000","last_undegraded":"2024-05-09T10:50:21.183073+0000","last_fullsized":"2024-05-09T10:50:21.183073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:53.042012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.192123+0000","last_change":"2024-05-09T10:50:20.190046+0000","last_active":"2024-05-09T10:50:20.192123+0000","last_peered":"2024-05-09T10:50:20.192123+0000","last_clean":"2024-05-09T10:50:20.192123+0000","last_became_active":"2024-05-09T10:50:20.189778+0000","last_became_peered":"2024-05-09T10:50:20.189778+0000","last_unstale":"2024-05-09T10:50:20.192123+0000","last_undegraded":"2024-05-09T10:50:20.192123+0000","last_fullsized":"2024-05-09T10:50:20.192123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:08:00.575142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182837+0000","last_change":"2024-05-09T10:50:19.172738+0000","last_active":"2024-05-09T10:50:21.182837+0000","last_peered":"2024-05-09T10:50:21.182837+0000","last_clean":"2024-05-09T10:50:21.182837+0000","last_became_active":"2024-05-09T10:50:19.172449+0000","last_became_peered":"2024-05-09T10:50:19.172449+0000","last_unstale":"2024-05-09T10:50:21.182837+0000","last_undegraded":"2024-05-09T10:50:21.182837+0000","last_fullsized":"2024-05-09T10:50:21.182837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:20.661093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.194855+0000","last_change":"2024-05-09T10:50:20.193228+0000","last_active":"2024-05-09T10:50:20.194855+0000","last_peered":"2024-05-09T10:50:20.194855+0000","last_clean":"2024-05-09T10:50:20.194855+0000","last_became_active":"2024-05-09T10:50:20.192454+0000","last_became_peered":"2024-05-09T10:50:20.192454+0000","last_unstale":"2024-05-09T10:50:20.194855+0000","last_undegraded":"2024-05-09T10:50:20.194855+0000","last_fullsized":"2024-05-09T10:50:20.194855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:57:07.297876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889326+0000","last_change":"2024-05-09T10:50:19.174967+0000","last_active":"2024-05-09T10:50:21.889326+0000","last_peered":"2024-05-09T10:50:21.889326+0000","last_clean":"2024-05-09T10:50:21.889326+0000","last_became_active":"2024-05-09T10:50:19.173603+0000","last_became_peered":"2024-05-09T10:50:19.173603+0000","last_unstale":"2024-05-09T10:50:21.889326+0000","last_undegraded":"2024-05-09T10:50:21.889326+0000","last_fullsized":"2024-05-09T10:50:21.889326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:16:52.160962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171552+0000","last_change":"2024-05-09T10:50:20.187976+0000","last_active":"2024-05-09T10:50:22.171552+0000","last_peered":"2024-05-09T10:50:22.171552+0000","last_clean":"2024-05-09T10:50:22.171552+0000","last_became_active":"2024-05-09T10:50:20.187661+0000","last_became_peered":"2024-05-09T10:50:20.187661+0000","last_unstale":"2024-05-09T10:50:22.171552+0000","last_undegraded":"2024-05-09T10:50:22.171552+0000","last_fullsized":"2024-05-09T10:50:22.171552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:45.067992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.188550+0000","last_change":"2024-05-09T10:50:20.181807+0000","last_active":"2024-05-09T10:50:20.188550+0000","last_peered":"2024-05-09T10:50:20.188550+0000","last_clean":"2024-05-09T10:50:20.188550+0000","last_became_active":"2024-05-09T10:50:20.181212+0000","last_became_peered":"2024-05-09T10:50:20.181212+0000","last_unstale":"2024-05-09T10:50:20.188550+0000","last_undegraded":"2024-05-09T10:50:20.188550+0000","last_fullsized":"2024-05-09T10:50:20.188550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:16:53.166077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889187+0000","last_change":"2024-05-09T10:50:19.175398+0000","last_active":"2024-05-09T10:50:21.889187+0000","last_peered":"2024-05-09T10:50:21.889187+0000","last_clean":"2024-05-09T10:50:21.889187+0000","last_became_active":"2024-05-09T10:50:19.174893+0000","last_became_peered":"2024-05-09T10:50:19.174893+0000","last_unstale":"2024-05-09T10:50:21.889187+0000","last_undegraded":"2024-05-09T10:50:21.889187+0000","last_fullsized":"2024-05-09T10:50:21.889187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:07:59.570026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.188596+0000","last_change":"2024-05-09T10:50:20.183971+0000","last_active":"2024-05-09T10:50:20.188596+0000","last_peered":"2024-05-09T10:50:20.188596+0000","last_clean":"2024-05-09T10:50:20.188596+0000","last_became_active":"2024-05-09T10:50:20.183666+0000","last_became_peered":"2024-05-09T10:50:20.183666+0000","last_unstale":"2024-05-09T10:50:20.188596+0000","last_undegraded":"2024-05-09T10:50:20.188596+0000","last_fullsized":"2024-05-09T10:50:20.188596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:04:46.158655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889464+0000","last_change":"2024-05-09T10:50:19.169800+0000","last_active":"2024-05-09T10:50:21.889464+0000","last_peered":"2024-05-09T10:50:21.889464+0000","last_clean":"2024-05-09T10:50:21.889464+0000","last_became_active":"2024-05-09T10:50:19.169521+0000","last_became_peered":"2024-05-09T10:50:19.169521+0000","last_unstale":"2024-05-09T10:50:21.889464+0000","last_undegraded":"2024-05-09T10:50:21.889464+0000","last_fullsized":"2024-05-09T10:50:21.889464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:37.762424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183081+0000","last_change":"2024-05-09T10:50:20.181912+0000","last_active":"2024-05-09T10:50:21.183081+0000","last_peered":"2024-05-09T10:50:21.183081+0000","last_clean":"2024-05-09T10:50:21.183081+0000","last_became_active":"2024-05-09T10:50:20.181615+0000","last_became_peered":"2024-05-09T10:50:20.181615+0000","last_unstale":"2024-05-09T10:50:21.183081+0000","last_undegraded":"2024-05-09T10:50:21.183081+0000","last_fullsized":"2024-05-09T10:50:21.183081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:50:14.919494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.192167+0000","last_change":"2024-05-09T10:50:19.165320+0000","last_active":"2024-05-09T10:50:20.192167+0000","last_peered":"2024-05-09T10:50:20.192167+0000","last_clean":"2024-05-09T10:50:20.192167+0000","last_became_active":"2024-05-09T10:50:19.165033+0000","last_became_peered":"2024-05-09T10:50:19.165033+0000","last_unstale":"2024-05-09T10:50:20.192167+0000","last_undegraded":"2024-05-09T10:50:20.192167+0000","last_fullsized":"2024-05-09T10:50:20.192167+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:11:05.623356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T10:50:20.191946+0000","last_change":"2024-05-09T10:50:20.188606+0000","last_active":"2024-05-09T10:50:20.191946+0000","last_peered":"2024-05-09T10:50:20.191946+0000","last_clean":"2024-05-09T10:50:20.191946+0000","last_became_active":"2024-05-09T10:50:20.188300+0000","last_became_peered":"2024-05-09T10:50:20.188300+0000","last_unstale":"2024-05-09T10:50:20.191946+0000","last_undegraded":"2024-05-09T10:50:20.191946+0000","last_fullsized":"2024-05-09T10:50:20.191946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:40.421893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183175+0000","last_change":"2024-05-09T10:50:19.168206+0000","last_active":"2024-05-09T10:50:21.183175+0000","last_peered":"2024-05-09T10:50:21.183175+0000","last_clean":"2024-05-09T10:50:21.183175+0000","last_became_active":"2024-05-09T10:50:19.167924+0000","last_became_peered":"2024-05-09T10:50:19.167924+0000","last_unstale":"2024-05-09T10:50:21.183175+0000","last_undegraded":"2024-05-09T10:50:21.183175+0000","last_fullsized":"2024-05-09T10:50:21.183175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:11:05.623356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":64724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T10:50:23.460 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-09T10:50:23.460 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-09T10:50:23.460 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-09T10:50:23.460 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-09T10:50:23.461 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-09T10:50:23.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.602+0000 7f98022e4640 1 Processor -- start 2024-05-09T10:50:23.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.602+0000 7f98022e4640 1 -- start start 2024-05-09T10:50:23.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.603+0000 7f98022e4640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97fc0ceb70 0x7f97fc0b4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:23.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.603+0000 7f98022e4640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f97fc0b4d40 0x7f97fc0cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:23.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.603+0000 7f98022e4640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97fc0cc4c0 con 0x7f97fc15bdc0 2024-05-09T10:50:23.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.603+0000 7f98022e4640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97fc15c1a0 con 0x7f97fc0b4d40 2024-05-09T10:50:23.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.603+0000 7f98022e4640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc0cf3c0 con 0x7f97fc0ceb70 2024-05-09T10:50:23.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.603+0000 7f9800881640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f97fc0b4d40 0x7f97fc0cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:23.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.603+0000 7f9800881640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f97fc0b4d40 0x7f97fc0cfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41840/0 (socket says 172.21.15.98:41840) 2024-05-09T10:50:23.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.603+0000 7f9800881640 1 -- 172.21.15.98:0/4048642601 learned_addr learned my addr 172.21.15.98:0/4048642601 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:23.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.603+0000 7f97fb7fe640 1 --2- 172.21.15.98:0/4048642601 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97fc0ceb70 0x7f97fc0b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:23.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.603+0000 7f9800881640 1 -- 172.21.15.98:0/4048642601 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97fc0ceb70 msgr2=0x7f97fc0b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:23.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.603+0000 7f9800881640 1 --2- 172.21.15.98:0/4048642601 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97fc0ceb70 0x7f97fc0b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.603+0000 7f9800881640 1 -- 172.21.15.98:0/4048642601 >> v1:172.21.15.105:6789/0 conn(0x7f97fc15bdc0 legacy=0x7f97fc0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:23.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.603+0000 7f9800881640 1 -- 172.21.15.98:0/4048642601 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97fc0cf0e0 con 0x7f97fc0b4d40 2024-05-09T10:50:23.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.604+0000 7f9800881640 1 --2- 172.21.15.98:0/4048642601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97fc0b4d40 0x7f97fc0cfa30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009e50 tx=0x7f97f402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4c1431d256dde3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:23.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.604+0000 7f97faffd640 1 -- 172.21.15.98:0/4048642601 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f4036070 con 0x7f97fc0b4d40 2024-05-09T10:50:23.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.604+0000 7f97faffd640 1 -- 172.21.15.98:0/4048642601 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f4031600 con 0x7f97fc0b4d40 2024-05-09T10:50:23.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.604+0000 7f97faffd640 1 -- 172.21.15.98:0/4048642601 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f40319d0 con 0x7f97fc0b4d40 2024-05-09T10:50:23.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.605+0000 7f98022e4640 1 -- 172.21.15.98:0/4048642601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97fc0b4d40 msgr2=0x7f97fc0cfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:23.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.605+0000 7f98022e4640 1 --2- 172.21.15.98:0/4048642601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97fc0b4d40 0x7f97fc0cfa30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009e50 tx=0x7f97f402a710 comp rx=0 tx=0).stop 2024-05-09T10:50:23.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.605+0000 7f98022e4640 1 -- 172.21.15.98:0/4048642601 shutdown_connections 2024-05-09T10:50:23.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.605+0000 7f98022e4640 1 --2- 172.21.15.98:0/4048642601 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97fc0ceb70 0x7f97fc0b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.605+0000 7f98022e4640 1 --2- 172.21.15.98:0/4048642601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97fc0b4d40 0x7f97fc0cfa30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.605+0000 7f98022e4640 1 -- 172.21.15.98:0/4048642601 >> 172.21.15.98:0/4048642601 conn(0x7f97fc0b34e0 msgr2=0x7f97fc0b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:23.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.605+0000 7f98022e4640 1 -- 172.21.15.98:0/4048642601 shutdown_connections 2024-05-09T10:50:23.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.605+0000 7f98022e4640 1 -- 172.21.15.98:0/4048642601 wait complete. 2024-05-09T10:50:23.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.606+0000 7f98022e4640 1 Processor -- start 2024-05-09T10:50:23.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.606+0000 7f98022e4640 1 -- start start 2024-05-09T10:50:23.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.607+0000 7f98022e4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97fc0b4d40 0x7f97fc156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:23.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.607+0000 7f98022e4640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97fc0ceb70 0x7f97fc1e5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:23.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.607+0000 7f98022e4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97fc15bdc0 0x7f97fc0b6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:23.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.607+0000 7f98022e4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc0ccd20 con 0x7f97fc0b4d40 2024-05-09T10:50:23.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.607+0000 7f98022e4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc0cca40 con 0x7f97fc15bdc0 2024-05-09T10:50:23.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.607+0000 7f98022e4640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc0b51d0 con 0x7f97fc0ceb70 2024-05-09T10:50:23.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.607+0000 7f97fbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97fc0b4d40 0x7f97fc156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:23.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.607+0000 7f97fbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97fc0b4d40 0x7f97fc156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41854/0 (socket says 172.21.15.98:41854) 2024-05-09T10:50:23.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.607+0000 7f97fbfff640 1 -- 172.21.15.98:0/1445085463 learned_addr learned my addr 172.21.15.98:0/1445085463 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:23.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.607+0000 7f97fbfff640 1 -- 172.21.15.98:0/1445085463 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97fc0ceb70 msgr2=0x7f97fc1e5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:23.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.607+0000 7f97fbfff640 1 --2- 172.21.15.98:0/1445085463 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97fc0ceb70 0x7f97fc1e5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.607+0000 7f97fbfff640 1 -- 172.21.15.98:0/1445085463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97fc15bdc0 msgr2=0x7f97fc0b6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:23.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.607+0000 7f97fbfff640 1 --2- 172.21.15.98:0/1445085463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97fc15bdc0 0x7f97fc0b6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.607+0000 7f97fbfff640 1 -- 172.21.15.98:0/1445085463 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97fc1e5f90 con 0x7f97fc0b4d40 2024-05-09T10:50:23.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.607+0000 7f97fbfff640 1 --2- 172.21.15.98:0/1445085463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97fc0b4d40 0x7f97fc156ea0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00c400 tx=0x7f97ec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:23.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.608+0000 7f97f8ff9640 1 -- 172.21.15.98:0/1445085463 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec0052d0 con 0x7f97fc0b4d40 2024-05-09T10:50:23.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.608+0000 7f97f8ff9640 1 -- 172.21.15.98:0/1445085463 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ec005470 con 0x7f97fc0b4d40 2024-05-09T10:50:23.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.608+0000 7f98022e4640 1 -- 172.21.15.98:0/1445085463 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97fc1e6270 con 0x7f97fc0b4d40 2024-05-09T10:50:23.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.608+0000 7f97f8ff9640 1 -- 172.21.15.98:0/1445085463 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec007260 con 0x7f97fc0b4d40 2024-05-09T10:50:23.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.608+0000 7f98022e4640 1 -- 172.21.15.98:0/1445085463 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97fc0c63d0 con 0x7f97fc0b4d40 2024-05-09T10:50:23.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.609+0000 7f98022e4640 1 -- 172.21.15.98:0/1445085463 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97fc0c65d0 con 0x7f97fc0b4d40 2024-05-09T10:50:23.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.609+0000 7f97f8ff9640 1 -- 172.21.15.98:0/1445085463 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f97ec007400 con 0x7f97fc0b4d40 2024-05-09T10:50:23.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.610+0000 7f97f8ff9640 1 --2- 172.21.15.98:0/1445085463 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f97e806b4d0 0x7f97e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:23.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.610+0000 7f97fb7fe640 1 --2- 172.21.15.98:0/1445085463 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f97e806b4d0 0x7f97e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:23.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.610+0000 7f97f8ff9640 1 -- 172.21.15.98:0/1445085463 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97ec089710 con 0x7f97fc0b4d40 2024-05-09T10:50:23.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.610+0000 7f97fb7fe640 1 --2- 172.21.15.98:0/1445085463 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f97e806b4d0 0x7f97e806d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f97fc1f50b0 tx=0x7f97f0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:23.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.614+0000 7f97f8ff9640 1 -- 172.21.15.98:0/1445085463 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97fc0c65d0 con 0x7f97fc0b4d40 2024-05-09T10:50:23.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.830+0000 7f98022e4640 1 -- 172.21.15.98:0/1445085463 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f97fc069cb0 con 0x7f97fc0b4d40 2024-05-09T10:50:23.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.833+0000 7f97f8ff9640 1 -- 172.21.15.98:0/1445085463 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171782 (secure 0 0 0) 0x7f97ec058590 con 0x7f97fc0b4d40 2024-05-09T10:50:23.828 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:23.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.835+0000 7f98022e4640 1 -- 172.21.15.98:0/1445085463 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f97e806b4d0 msgr2=0x7f97e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:23.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.835+0000 7f98022e4640 1 --2- 172.21.15.98:0/1445085463 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f97e806b4d0 0x7f97e806d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f97fc1f50b0 tx=0x7f97f0008040 comp rx=0 tx=0).stop 2024-05-09T10:50:23.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.835+0000 7f98022e4640 1 -- 172.21.15.98:0/1445085463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97fc0b4d40 msgr2=0x7f97fc156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:23.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.835+0000 7f98022e4640 1 --2- 172.21.15.98:0/1445085463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97fc0b4d40 0x7f97fc156ea0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00c400 tx=0x7f97ec00c8c0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.835+0000 7f98022e4640 1 -- 172.21.15.98:0/1445085463 shutdown_connections 2024-05-09T10:50:23.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.835+0000 7f98022e4640 1 --2- 172.21.15.98:0/1445085463 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f97e806b4d0 0x7f97e806d990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.835+0000 7f98022e4640 1 --2- 172.21.15.98:0/1445085463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97fc15bdc0 0x7f97fc0b6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.835+0000 7f98022e4640 1 --2- 172.21.15.98:0/1445085463 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97fc0ceb70 0x7f97fc1e5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.835+0000 7f98022e4640 1 --2- 172.21.15.98:0/1445085463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97fc0b4d40 0x7f97fc156ea0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.835+0000 7f98022e4640 1 -- 172.21.15.98:0/1445085463 >> 172.21.15.98:0/1445085463 conn(0x7f97fc0b34e0 msgr2=0x7f97fc06eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:23.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.835+0000 7f98022e4640 1 -- 172.21.15.98:0/1445085463 shutdown_connections 2024-05-09T10:50:23.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.835+0000 7f98022e4640 1 -- 172.21.15.98:0/1445085463 wait complete. 2024-05-09T10:50:23.839 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":5,"flags":0,"active_gid":4110,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":800971426},{"type":"v1","addr":"172.21.15.98:6833","nonce":800971426}]},"active_addr":"172.21.15.98:6833/800971426","active_change":"2024-05-09T10:49:59.355011+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.98:0","nonce":3031147356}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.98:0","nonce":3733137378}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.98:0","nonce":1177398271}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.98:0","nonce":3652209367}]}]} 2024-05-09T10:50:23.843 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-09T10:50:23.843 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-09T10:50:23.843 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:23.924 INFO:tasks.ceph.mds.d.smithi105.stderr:2024-05-09T10:50:23.931+0000 7f7576662640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T10:50:23.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.949+0000 7f630b0d8640 1 Processor -- start 2024-05-09T10:50:23.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.949+0000 7f630b0d8640 1 -- start start 2024-05-09T10:50:23.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.950+0000 7f630b0d8640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f63040d27a0 0x7f63040d0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:23.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.950+0000 7f630b0d8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63040d09b0 0x7f63040b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:23.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.950+0000 7f630b0d8640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63040d2c30 con 0x7f630415de40 2024-05-09T10:50:23.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.950+0000 7f630b0d8640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63040d0e20 con 0x7f63040d27a0 2024-05-09T10:50:23.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.950+0000 7f630b0d8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63040d1790 con 0x7f63040d09b0 2024-05-09T10:50:23.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.950+0000 7f6303fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f63040d27a0 0x7f63040d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:23.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.950+0000 7f6308e74640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f630415de40 0x7f63040cf250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55964/0 (socket says 172.21.15.98:55964) 2024-05-09T10:50:23.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.950+0000 7f6308e74640 1 -- 172.21.15.98:0/1895812904 learned_addr learned my addr 172.21.15.98:0/1895812904 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:23.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.950+0000 7f6309675640 1 --2- 172.21.15.98:0/1895812904 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63040d09b0 0x7f63040b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:23.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.950+0000 7f6303fff640 1 -- 172.21.15.98:0/1895812904 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63040d09b0 msgr2=0x7f63040b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:23.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.950+0000 7f6303fff640 1 --2- 172.21.15.98:0/1895812904 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63040d09b0 0x7f63040b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.950+0000 7f6303fff640 1 -- 172.21.15.98:0/1895812904 >> v1:172.21.15.98:6789/0 conn(0x7f630415de40 legacy=0x7f63040cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:23.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.950+0000 7f6303fff640 1 -- 172.21.15.98:0/1895812904 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630415e220 con 0x7f63040d27a0 2024-05-09T10:50:23.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.950+0000 7f63037fe640 1 -- 172.21.15.98:0/1895812904 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 191977901 0 0) 0x7f63040d2c30 con 0x7f630415de40 2024-05-09T10:50:23.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.951+0000 7f6303fff640 1 --2- 172.21.15.98:0/1895812904 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63040d27a0 0x7f63040d0470 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f63040d1910 tx=0x7f62f802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c162302726f8a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:23.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.951+0000 7f63037fe640 1 -- 172.21.15.98:0/1895812904 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f8037070 con 0x7f63040d27a0 2024-05-09T10:50:23.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.951+0000 7f63037fe640 1 -- 172.21.15.98:0/1895812904 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f8032440 con 0x7f63040d27a0 2024-05-09T10:50:23.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.951+0000 7f63037fe640 1 -- 172.21.15.98:0/1895812904 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f8032810 con 0x7f63040d27a0 2024-05-09T10:50:23.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.952+0000 7f630b0d8640 1 -- 172.21.15.98:0/1895812904 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63040d27a0 msgr2=0x7f63040d0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:23.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.952+0000 7f630b0d8640 1 --2- 172.21.15.98:0/1895812904 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63040d27a0 0x7f63040d0470 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f63040d1910 tx=0x7f62f802a180 comp rx=0 tx=0).stop 2024-05-09T10:50:23.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.952+0000 7f630b0d8640 1 -- 172.21.15.98:0/1895812904 shutdown_connections 2024-05-09T10:50:23.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.952+0000 7f630b0d8640 1 --2- 172.21.15.98:0/1895812904 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63040d09b0 0x7f63040b6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.952+0000 7f630b0d8640 1 --2- 172.21.15.98:0/1895812904 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63040d27a0 0x7f63040d0470 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.952+0000 7f630b0d8640 1 -- 172.21.15.98:0/1895812904 >> 172.21.15.98:0/1895812904 conn(0x7f630409ce90 msgr2=0x7f63040d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:23.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.952+0000 7f630b0d8640 1 -- 172.21.15.98:0/1895812904 shutdown_connections 2024-05-09T10:50:23.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.952+0000 7f630b0d8640 1 -- 172.21.15.98:0/1895812904 wait complete. 2024-05-09T10:50:23.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.953+0000 7f630b0d8640 1 Processor -- start 2024-05-09T10:50:23.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.953+0000 7f630b0d8640 1 -- start start 2024-05-09T10:50:23.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.954+0000 7f630b0d8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63040d09b0 0x7f6304150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:23.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.954+0000 7f630b0d8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63040d27a0 0x7f63041f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:23.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.954+0000 7f630b0d8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f630415de40 0x7f63040cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:23.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.954+0000 7f630b0d8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63040d1fa0 con 0x7f630415de40 2024-05-09T10:50:23.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.954+0000 7f630b0d8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63040d1cc0 con 0x7f63040d09b0 2024-05-09T10:50:23.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.954+0000 7f630b0d8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63040d2280 con 0x7f63040d27a0 2024-05-09T10:50:23.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.954+0000 7f6309675640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f630415de40 0x7f63040cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:23.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.954+0000 7f6309675640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f630415de40 0x7f63040cc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41884/0 (socket says 172.21.15.98:41884) 2024-05-09T10:50:23.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.954+0000 7f6309675640 1 -- 172.21.15.98:0/3515883749 learned_addr learned my addr 172.21.15.98:0/3515883749 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:23.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.954+0000 7f6303fff640 1 --2- 172.21.15.98:0/3515883749 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63040d27a0 0x7f63041f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:23.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.954+0000 7f6309675640 1 -- 172.21.15.98:0/3515883749 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63040d27a0 msgr2=0x7f63041f3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:23.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.954+0000 7f6309675640 1 --2- 172.21.15.98:0/3515883749 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63040d27a0 0x7f63041f3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.954+0000 7f6309675640 1 -- 172.21.15.98:0/3515883749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63040d09b0 msgr2=0x7f6304150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:23.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.954+0000 7f6309675640 1 --2- 172.21.15.98:0/3515883749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63040d09b0 0x7f6304150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:23.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.954+0000 7f6309675640 1 -- 172.21.15.98:0/3515883749 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6304158690 con 0x7f630415de40 2024-05-09T10:50:23.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.954+0000 7f6309675640 1 --2- 172.21.15.98:0/3515883749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f630415de40 0x7f63040cc220 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f62fc007e40 tx=0x7f62fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:23.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.954+0000 7f63017fa640 1 -- 172.21.15.98:0/3515883749 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc014620 con 0x7f630415de40 2024-05-09T10:50:23.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.954+0000 7f63017fa640 1 -- 172.21.15.98:0/3515883749 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62fc019070 con 0x7f630415de40 2024-05-09T10:50:23.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.955+0000 7f630b0d8640 1 -- 172.21.15.98:0/3515883749 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6304158970 con 0x7f630415de40 2024-05-09T10:50:23.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.955+0000 7f63017fa640 1 -- 172.21.15.98:0/3515883749 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc014a20 con 0x7f630415de40 2024-05-09T10:50:23.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.955+0000 7f630b0d8640 1 -- 172.21.15.98:0/3515883749 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6304159210 con 0x7f630415de40 2024-05-09T10:50:23.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.955+0000 7f630b0d8640 1 -- 172.21.15.98:0/3515883749 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6304069cb0 con 0x7f630415de40 2024-05-09T10:50:23.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.956+0000 7f63017fa640 1 -- 172.21.15.98:0/3515883749 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f62fc014bc0 con 0x7f630415de40 2024-05-09T10:50:23.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.956+0000 7f63017fa640 1 --2- 172.21.15.98:0/3515883749 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f62f006b4d0 0x7f62f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:23.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.956+0000 7f6308e74640 1 --2- 172.21.15.98:0/3515883749 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f62f006b4d0 0x7f62f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:23.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.956+0000 7f63017fa640 1 -- 172.21.15.98:0/3515883749 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62fc08ab00 con 0x7f630415de40 2024-05-09T10:50:23.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.957+0000 7f6308e74640 1 --2- 172.21.15.98:0/3515883749 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f62f006b4d0 0x7f62f006d990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f62f4007fd0 tx=0x7f62f4007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:23.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:23.961+0000 7f63017fa640 1 -- 172.21.15.98:0/3515883749 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f62fc059a10 con 0x7f630415de40 2024-05-09T10:50:24.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.133+0000 7f630b0d8640 1 -- 172.21.15.98:0/3515883749 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63040d42d0 con 0x7f630415de40 2024-05-09T10:50:24.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.134+0000 7f63017fa640 1 -- 172.21.15.98:0/3515883749 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f62fc059310 con 0x7f630415de40 2024-05-09T10:50:24.128 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:24.128 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":20,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:21.176419+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:50:17.323444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T10:50:18.482315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:24.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.136+0000 7f630b0d8640 1 -- 172.21.15.98:0/3515883749 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f62f006b4d0 msgr2=0x7f62f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.136+0000 7f630b0d8640 1 --2- 172.21.15.98:0/3515883749 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f62f006b4d0 0x7f62f006d990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f62f4007fd0 tx=0x7f62f4007960 comp rx=0 tx=0).stop 2024-05-09T10:50:24.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.136+0000 7f630b0d8640 1 -- 172.21.15.98:0/3515883749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f630415de40 msgr2=0x7f63040cc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.136+0000 7f630b0d8640 1 --2- 172.21.15.98:0/3515883749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f630415de40 0x7f63040cc220 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f62fc007e40 tx=0x7f62fc00e4b0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.136+0000 7f630b0d8640 1 -- 172.21.15.98:0/3515883749 shutdown_connections 2024-05-09T10:50:24.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.136+0000 7f630b0d8640 1 --2- 172.21.15.98:0/3515883749 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f62f006b4d0 0x7f62f006d990 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.136+0000 7f630b0d8640 1 --2- 172.21.15.98:0/3515883749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f630415de40 0x7f63040cc220 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.136+0000 7f630b0d8640 1 --2- 172.21.15.98:0/3515883749 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63040d27a0 0x7f63041f3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.136+0000 7f630b0d8640 1 --2- 172.21.15.98:0/3515883749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63040d09b0 0x7f6304150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.136+0000 7f630b0d8640 1 -- 172.21.15.98:0/3515883749 >> 172.21.15.98:0/3515883749 conn(0x7f630409ce90 msgr2=0x7f630406b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.136+0000 7f630b0d8640 1 -- 172.21.15.98:0/3515883749 shutdown_connections 2024-05-09T10:50:24.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.136+0000 7f630b0d8640 1 -- 172.21.15.98:0/3515883749 wait complete. 2024-05-09T10:50:24.141 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-09T10:50:24.141 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:24.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.248+0000 7fb1e93cf640 1 Processor -- start 2024-05-09T10:50:24.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.248+0000 7fb1e93cf640 1 -- start start 2024-05-09T10:50:24.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.249+0000 7fb1e93cf640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb1e406eb30 0x7fb1e40b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.249+0000 7fb1e93cf640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e40d07a0 con 0x7fb1e406d570 2024-05-09T10:50:24.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.249+0000 7fb1e93cf640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e406f9b0 con 0x7fb1e415ddc0 2024-05-09T10:50:24.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.249+0000 7fb1e93cf640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e40d0db0 con 0x7fb1e406eb30 2024-05-09T10:50:24.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.249+0000 7fb1e27fc640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb1e406d570 0x7fb1e406ef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55966/0 (socket says 172.21.15.98:55966) 2024-05-09T10:50:24.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.249+0000 7fb1e27fc640 1 -- 172.21.15.98:0/2918577362 learned_addr learned my addr 172.21.15.98:0/2918577362 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:24.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.249+0000 7fb1e1ffb640 1 -- 172.21.15.98:0/2918577362 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1428717328 0 0) 0x7fb1e40d07a0 con 0x7fb1e406d570 2024-05-09T10:50:24.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.249+0000 7fb1e1ffb640 1 -- 172.21.15.98:0/2918577362 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1d0003520 con 0x7fb1e406d570 2024-05-09T10:50:24.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.249+0000 7fb1e2ffd640 1 --2- 172.21.15.98:0/2918577362 >> v2:172.21.15.105:3300/0 conn(0x7fb1e406eb30 0x7fb1e40b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.250+0000 7fb1e1ffb640 1 -- 172.21.15.98:0/2918577362 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1982390245 0 0) 0x7fb1d0003520 con 0x7fb1e406d570 2024-05-09T10:50:24.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.250+0000 7fb1e1ffb640 1 -- 172.21.15.98:0/2918577362 >> v2:172.21.15.105:3300/0 conn(0x7fb1e406eb30 msgr2=0x7fb1e40b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:24.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.250+0000 7fb1e1ffb640 1 --2- 172.21.15.98:0/2918577362 >> v2:172.21.15.105:3300/0 conn(0x7fb1e406eb30 0x7fb1e40b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.250+0000 7fb1e1ffb640 1 -- 172.21.15.98:0/2918577362 >> v1:172.21.15.105:6789/0 conn(0x7fb1e415ddc0 legacy=0x7fb1e415e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.250+0000 7fb1e1ffb640 1 -- 172.21.15.98:0/2918577362 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e406f6d0 con 0x7fb1e406d570 2024-05-09T10:50:24.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.250+0000 7fb1e1ffb640 1 -- 172.21.15.98:0/2918577362 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7fb1cc001670 con 0x7fb1e406d570 2024-05-09T10:50:24.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.250+0000 7fb1e1ffb640 1 -- 172.21.15.98:0/2918577362 >> v1:172.21.15.98:6789/0 conn(0x7fb1e406d570 legacy=0x7fb1e406ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.250+0000 7fb1e1ffb640 1 --2- 172.21.15.98:0/2918577362 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1d00039b0 0x7fb1d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.250+0000 7fb1e1ffb640 1 --2- 172.21.15.98:0/2918577362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1d00043b0 0x7fb1d0004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.250+0000 7fb1e1ffb640 1 --2- 172.21.15.98:0/2918577362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d0005120 0x7fb1d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.250+0000 7fb1e1ffb640 1 -- 172.21.15.98:0/2918577362 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40d0db0 con 0x7fb1d00043b0 2024-05-09T10:50:24.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.250+0000 7fb1e1ffb640 1 -- 172.21.15.98:0/2918577362 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d000c060 con 0x7fb1d0005120 2024-05-09T10:50:24.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.250+0000 7fb1e1ffb640 1 -- 172.21.15.98:0/2918577362 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d000c260 con 0x7fb1d00039b0 2024-05-09T10:50:24.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.250+0000 7fb1e27fc640 1 --2- 172.21.15.98:0/2918577362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1d00043b0 0x7fb1d0004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.250+0000 7fb1e37fe640 1 --2- 172.21.15.98:0/2918577362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d0005120 0x7fb1d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.250+0000 7fb1e2ffd640 1 --2- 172.21.15.98:0/2918577362 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1d00039b0 0x7fb1d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.250+0000 7fb1e27fc640 1 -- 172.21.15.98:0/2918577362 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1d00039b0 msgr2=0x7fb1d0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:24.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.250+0000 7fb1e27fc640 1 --2- 172.21.15.98:0/2918577362 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1d00039b0 0x7fb1d0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.250+0000 7fb1e27fc640 1 -- 172.21.15.98:0/2918577362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d0005120 msgr2=0x7fb1d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:24.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.250+0000 7fb1e27fc640 1 --2- 172.21.15.98:0/2918577362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d0005120 0x7fb1d000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.250+0000 7fb1e27fc640 1 -- 172.21.15.98:0/2918577362 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d0003520 con 0x7fb1d00043b0 2024-05-09T10:50:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.250+0000 7fb1e37fe640 1 --2- 172.21.15.98:0/2918577362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d0005120 0x7fb1d000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.250+0000 7fb1e37fe640 1 -- 172.21.15.98:0/2918577362 reap_dead start 2024-05-09T10:50:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.251+0000 7fb1e27fc640 1 --2- 172.21.15.98:0/2918577362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1d00043b0 0x7fb1d0004880 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00c240 tx=0x7fb1d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df38ec08a74c9a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.251+0000 7fb1e1ffb640 1 -- 172.21.15.98:0/2918577362 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc007d30 con 0x7fb1d00043b0 2024-05-09T10:50:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.251+0000 7fb1e1ffb640 1 -- 172.21.15.98:0/2918577362 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1cc005af0 con 0x7fb1d00043b0 2024-05-09T10:50:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.251+0000 7fb1e1ffb640 1 -- 172.21.15.98:0/2918577362 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc007070 con 0x7fb1d00043b0 2024-05-09T10:50:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.251+0000 7fb1e93cf640 1 -- 172.21.15.98:0/2918577362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1d00043b0 msgr2=0x7fb1d0004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.251+0000 7fb1e93cf640 1 --2- 172.21.15.98:0/2918577362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1d00043b0 0x7fb1d0004880 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00c240 tx=0x7fb1d0005830 comp rx=0 tx=0).stop 2024-05-09T10:50:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.252+0000 7fb1e93cf640 1 -- 172.21.15.98:0/2918577362 shutdown_connections 2024-05-09T10:50:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.252+0000 7fb1e93cf640 1 --2- 172.21.15.98:0/2918577362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1d00043b0 0x7fb1d0004880 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.252+0000 7fb1e93cf640 1 -- 172.21.15.98:0/2918577362 >> 172.21.15.98:0/2918577362 conn(0x7fb1e40b34e0 msgr2=0x7fb1e40b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.252+0000 7fb1e93cf640 1 -- 172.21.15.98:0/2918577362 shutdown_connections 2024-05-09T10:50:24.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.252+0000 7fb1e93cf640 1 -- 172.21.15.98:0/2918577362 wait complete. 2024-05-09T10:50:24.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.253+0000 7fb1e93cf640 1 Processor -- start 2024-05-09T10:50:24.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.253+0000 7fb1e93cf640 1 -- start start 2024-05-09T10:50:24.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1e93cf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d00043b0 0x7fb1e414cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1e93cf640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1e40c8070 0x7fb1e40b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1e93cf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1e40b0a00 0x7fb1e41f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1e93cf640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40b4180 con 0x7fb1e40b0a00 2024-05-09T10:50:24.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1e93cf640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40b4000 con 0x7fb1d00043b0 2024-05-09T10:50:24.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1e93cf640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e40b4300 con 0x7fb1e40c8070 2024-05-09T10:50:24.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1e37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1e40b0a00 0x7fb1e41f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1e37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1e40b0a00 0x7fb1e41f37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41904/0 (socket says 172.21.15.98:41904) 2024-05-09T10:50:24.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1e37fe640 1 -- 172.21.15.98:0/1403341012 learned_addr learned my addr 172.21.15.98:0/1403341012 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1e27fc640 1 --2- 172.21.15.98:0/1403341012 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1e40c8070 0x7fb1e40b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1e2ffd640 1 --2- 172.21.15.98:0/1403341012 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d00043b0 0x7fb1e414cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1e37fe640 1 -- 172.21.15.98:0/1403341012 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1e40c8070 msgr2=0x7fb1e40b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1e37fe640 1 --2- 172.21.15.98:0/1403341012 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1e40c8070 0x7fb1e40b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1e37fe640 1 -- 172.21.15.98:0/1403341012 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d00043b0 msgr2=0x7fb1e414cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1e37fe640 1 --2- 172.21.15.98:0/1403341012 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d00043b0 0x7fb1e414cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1e37fe640 1 -- 172.21.15.98:0/1403341012 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e40c6fd0 con 0x7fb1e40b0a00 2024-05-09T10:50:24.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1e27fc640 1 --2- 172.21.15.98:0/1403341012 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1e40c8070 0x7fb1e40b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:24.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1e2ffd640 1 --2- 172.21.15.98:0/1403341012 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d00043b0 0x7fb1e414cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:24.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1e37fe640 1 --2- 172.21.15.98:0/1403341012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1e40b0a00 0x7fb1e41f37d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc007ac0 tx=0x7fb1dc00e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1db7fe640 1 -- 172.21.15.98:0/1403341012 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc019070 con 0x7fb1e40b0a00 2024-05-09T10:50:24.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1db7fe640 1 -- 172.21.15.98:0/1403341012 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1dc0052e0 con 0x7fb1e40b0a00 2024-05-09T10:50:24.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1e93cf640 1 -- 172.21.15.98:0/1403341012 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e40c72b0 con 0x7fb1e40b0a00 2024-05-09T10:50:24.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1db7fe640 1 -- 172.21.15.98:0/1403341012 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc007250 con 0x7fb1e40b0a00 2024-05-09T10:50:24.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.254+0000 7fb1e93cf640 1 -- 172.21.15.98:0/1403341012 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e40c7870 con 0x7fb1e40b0a00 2024-05-09T10:50:24.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.255+0000 7fb1e93cf640 1 -- 172.21.15.98:0/1403341012 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e40b3df0 con 0x7fb1e40b0a00 2024-05-09T10:50:24.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.256+0000 7fb1db7fe640 1 -- 172.21.15.98:0/1403341012 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb1dc01c070 con 0x7fb1e40b0a00 2024-05-09T10:50:24.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.256+0000 7fb1db7fe640 1 --2- 172.21.15.98:0/1403341012 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb1c406b4d0 0x7fb1c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.256+0000 7fb1db7fe640 1 -- 172.21.15.98:0/1403341012 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1dc089000 con 0x7fb1e40b0a00 2024-05-09T10:50:24.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.256+0000 7fb1e2ffd640 1 --2- 172.21.15.98:0/1403341012 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb1c406b4d0 0x7fb1c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.257+0000 7fb1e2ffd640 1 --2- 172.21.15.98:0/1403341012 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb1c406b4d0 0x7fb1c406d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4007440 tx=0x7fb1d400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.260+0000 7fb1db7fe640 1 -- 172.21.15.98:0/1403341012 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb1e40b3df0 con 0x7fb1e40b0a00 2024-05-09T10:50:24.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.434+0000 7fb1e93cf640 1 -- 172.21.15.98:0/1403341012 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1e406f9b0 con 0x7fb1e40b0a00 2024-05-09T10:50:24.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.435+0000 7fb1db7fe640 1 -- 172.21.15.98:0/1403341012 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7fb1dc057e80 con 0x7fb1e40b0a00 2024-05-09T10:50:24.428 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:24.429 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":20,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:21.176419+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:50:17.323444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T10:50:18.482315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.437+0000 7fb1e93cf640 1 -- 172.21.15.98:0/1403341012 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb1c406b4d0 msgr2=0x7fb1c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.437+0000 7fb1e93cf640 1 --2- 172.21.15.98:0/1403341012 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb1c406b4d0 0x7fb1c406d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4007440 tx=0x7fb1d400b040 comp rx=0 tx=0).stop 2024-05-09T10:50:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.437+0000 7fb1e93cf640 1 -- 172.21.15.98:0/1403341012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1e40b0a00 msgr2=0x7fb1e41f37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.437+0000 7fb1e93cf640 1 --2- 172.21.15.98:0/1403341012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1e40b0a00 0x7fb1e41f37d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc007ac0 tx=0x7fb1dc00e820 comp rx=0 tx=0).stop 2024-05-09T10:50:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.437+0000 7fb1e93cf640 1 -- 172.21.15.98:0/1403341012 shutdown_connections 2024-05-09T10:50:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.437+0000 7fb1e93cf640 1 --2- 172.21.15.98:0/1403341012 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb1c406b4d0 0x7fb1c406d990 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.437+0000 7fb1e93cf640 1 --2- 172.21.15.98:0/1403341012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1e40b0a00 0x7fb1e41f37d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.437+0000 7fb1e93cf640 1 --2- 172.21.15.98:0/1403341012 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1e40c8070 0x7fb1e40b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.437+0000 7fb1e93cf640 1 --2- 172.21.15.98:0/1403341012 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1d00043b0 0x7fb1e414cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.437+0000 7fb1e93cf640 1 -- 172.21.15.98:0/1403341012 >> 172.21.15.98:0/1403341012 conn(0x7fb1e40b34e0 msgr2=0x7fb1e415d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.437+0000 7fb1e93cf640 1 -- 172.21.15.98:0/1403341012 shutdown_connections 2024-05-09T10:50:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.437+0000 7fb1e93cf640 1 -- 172.21.15.98:0/1403341012 wait complete. 2024-05-09T10:50:24.442 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-09T10:50:24.443 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-09T10:50:24.443 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-09T10:50:24.444 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-09T10:50:24.444 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-09T10:50:24.444 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-09T10:50:24.445 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-09T10:50:24.445 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-09T10:50:24.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.622+0000 7fce98c6b640 1 Processor -- start 2024-05-09T10:50:24.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.622+0000 7fce98c6b640 1 -- start start 2024-05-09T10:50:24.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.623+0000 7f92c663a640 1 Processor -- start 2024-05-09T10:50:24.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.624+0000 7fce98c6b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce9415a190 0x7fce940668f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.624+0000 7fce98c6b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fce9406c300 0x7fce940b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.624+0000 7fce98c6b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce940b5140 con 0x7fce940b4680 2024-05-09T10:50:24.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.624+0000 7fce98c6b640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce9415a990 con 0x7fce9406c300 2024-05-09T10:50:24.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.624+0000 7fce98c6b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9415ac70 con 0x7fce9415a190 2024-05-09T10:50:24.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.624+0000 7fb2da84a640 1 Processor -- start 2024-05-09T10:50:24.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.624+0000 7fce92d9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fce9406c300 0x7fce940b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.624+0000 7fce9259c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce9415a190 0x7fce940668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.624+0000 7fce9259c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce9415a190 0x7fce940668f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:53086/0 (socket says 172.21.15.98:53086) 2024-05-09T10:50:24.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.624+0000 7fce9259c640 1 -- 172.21.15.98:0/1069721617 learned_addr learned my addr 172.21.15.98:0/1069721617 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.625+0000 7ffa4ad8b640 1 Processor -- start 2024-05-09T10:50:24.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.625+0000 7ffa4ad8b640 1 -- start start 2024-05-09T10:50:24.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.625+0000 7f29b9c93640 1 Processor -- start 2024-05-09T10:50:24.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7f29b9c93640 1 -- start start 2024-05-09T10:50:24.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.624+0000 7fb2da84a640 1 -- start start 2024-05-09T10:50:24.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7fb2da84a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2cc0e12b0 0x7fb2cc0534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7fb2da84a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb2cc054b50 0x7fb2cc053a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7fb2da84a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2cc054fe0 con 0x7fb2cc0e4d30 2024-05-09T10:50:24.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7fb2da84a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2cc0552c0 con 0x7fb2cc054b50 2024-05-09T10:50:24.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7fb2da84a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc0555a0 con 0x7fb2cc0e12b0 2024-05-09T10:50:24.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7fb2d9848640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2cc0e12b0 0x7fb2cc0534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7fb2d9848640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2cc0e12b0 0x7fb2cc0534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:53106/0 (socket says 172.21.15.98:53106) 2024-05-09T10:50:24.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7fb2d9848640 1 -- 172.21.15.98:0/1059571630 learned_addr learned my addr 172.21.15.98:0/1059571630 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.625+0000 7fce9259c640 1 -- 172.21.15.98:0/1069721617 >> v2:172.21.15.98:3300/0 conn(0x7fce9406c300 msgr2=0x7fce940b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:24.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.625+0000 7fce9259c640 1 --2- 172.21.15.98:0/1069721617 >> v2:172.21.15.98:3300/0 conn(0x7fce9406c300 0x7fce940b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.625+0000 7fce9259c640 1 -- 172.21.15.98:0/1069721617 >> v1:172.21.15.105:6789/0 conn(0x7fce940b4680 legacy=0x7fce9406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:24.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.625+0000 7fce9259c640 1 -- 172.21.15.98:0/1069721617 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9415af50 con 0x7fce9415a190 2024-05-09T10:50:24.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.625+0000 7fce92d9d640 1 --2- 172.21.15.98:0/1069721617 >> v2:172.21.15.98:3300/0 conn(0x7fce9406c300 0x7fce940b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:50:24.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7fce9259c640 1 --2- 172.21.15.98:0/1069721617 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce9415a190 0x7fce940668f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fce88009870 tx=0x7fce8802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feea6818230a7833 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7fce9159a640 1 -- 172.21.15.98:0/1069721617 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce88037070 con 0x7fce9415a190 2024-05-09T10:50:24.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.624+0000 7f92c663a640 1 -- start start 2024-05-09T10:50:24.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.625+0000 7f92c663a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f92b80e0090 0x7f92b8034fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.625+0000 7f92c663a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f92b8050420 0x7f92b8038dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.625+0000 7f92c663a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92b8039310 0x7f92b8008ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.625+0000 7f92c663a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b8050920 con 0x7f92b80e0090 2024-05-09T10:50:24.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.625+0000 7f92bffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f92b80e0090 0x7f92b8034fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.625+0000 7f92c663a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b8034d00 con 0x7f92b8050420 2024-05-09T10:50:24.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.625+0000 7f92c663a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b8007d20 con 0x7f92b8039310 2024-05-09T10:50:24.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.625+0000 7f92bffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f92b80e0090 0x7f92b8034fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41926/0 (socket says 172.21.15.98:41926) 2024-05-09T10:50:24.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.625+0000 7f92bffff640 1 -- 172.21.15.98:0/619546514 learned_addr learned my addr 172.21.15.98:0/619546514 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.625+0000 7f08ab9f4640 1 Processor -- start 2024-05-09T10:50:24.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.625+0000 7f62453b7640 1 Processor -- start 2024-05-09T10:50:24.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.624+0000 7f6dc4ce1640 1 Processor -- start 2024-05-09T10:50:24.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.624+0000 7f6dc4ce1640 1 -- start start 2024-05-09T10:50:24.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7f6dc4ce1640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6dc0165550 0x7f6dc00cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7f6dc4ce1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6dc0165930 0x7f6dc00d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7ffa4ad8b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa440b7520 0x7ffa44165d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7ffa4ad8b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffa440baec0 0x7ffa440d6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7ffa4ad8b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa441625d0 con 0x7ffa440d8690 2024-05-09T10:50:24.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7ffa4ad8b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa4406df10 con 0x7ffa440baec0 2024-05-09T10:50:24.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7fce9159a640 1 -- 172.21.15.98:0/1069721617 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce88032440 con 0x7fce9415a190 2024-05-09T10:50:24.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7ffa4ad8b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa440b7900 con 0x7ffa440b7520 2024-05-09T10:50:24.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7f6dbe59c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6dc0165550 0x7f6dc00cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7f6dbdd9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6dc0165930 0x7f6dc00d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7f6dbdd9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6dc0165930 0x7f6dc00d0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.98:35128/0 (socket says 172.21.15.98:35128) 2024-05-09T10:50:24.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7f6dbe59c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6dc0165550 0x7f6dc00cfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:53118/0 (socket says 172.21.15.98:53118) 2024-05-09T10:50:24.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7f6dbdd9b640 1 -- 172.21.15.98:0/3726393882 learned_addr learned my addr 172.21.15.98:0/3726393882 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7ffa48b27640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa440b7520 0x7ffa44165d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7ffa48b27640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa440b7520 0x7ffa44165d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:53130/0 (socket says 172.21.15.98:53130) 2024-05-09T10:50:24.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7ffa48b27640 1 -- 172.21.15.98:0/66061445 learned_addr learned my addr 172.21.15.98:0/66061445 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7ffa43fff640 1 --2- 172.21.15.98:0/66061445 >> v2:172.21.15.105:3300/0 conn(0x7ffa440baec0 0x7ffa440d6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7f6dc4ce1640 1 --2- 172.21.15.98:0/3726393882 >> v2:172.21.15.98:3300/0 conn(0x7f6dc00b46e0 0x7f6dc00b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f6dc4ce1640 1 -- 172.21.15.98:0/3726393882 --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc00b2f40 con 0x7f6dc00b46e0 2024-05-09T10:50:24.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f6dc4ce1640 1 -- 172.21.15.98:0/3726393882 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc0166b60 con 0x7f6dc0165930 2024-05-09T10:50:24.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f6dc4ce1640 1 -- 172.21.15.98:0/3726393882 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc00b50b0 con 0x7f6dc0165550 2024-05-09T10:50:24.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7fce9159a640 1 -- 172.21.15.98:0/1069721617 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce88032810 con 0x7fce9415a190 2024-05-09T10:50:24.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7fb2d9848640 1 -- 172.21.15.98:0/1059571630 >> v2:172.21.15.105:3300/0 conn(0x7fb2cc054b50 msgr2=0x7fb2cc053a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T10:50:24.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7fb2d9848640 1 --2- 172.21.15.98:0/1059571630 >> v2:172.21.15.105:3300/0 conn(0x7fb2cc054b50 0x7fb2cc053a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7fb2d9848640 1 -- 172.21.15.98:0/1059571630 >> v1:172.21.15.105:6789/0 conn(0x7fb2cc0e4d30 legacy=0x7fb2cc0e1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7fb2d9848640 1 -- 172.21.15.98:0/1059571630 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2cc055880 con 0x7fb2cc0e12b0 2024-05-09T10:50:24.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.625+0000 7f08ab9f4640 1 -- start start 2024-05-09T10:50:24.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f08ab9f4640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f08a415c770 0x7f08a4163330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f08ab9f4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f08a415cb50 0x7f08a40cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f08ab9f4640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08a40cfbc0 0x7f08a40ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f08ab9f4640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a40d0010 con 0x7f08a415c770 2024-05-09T10:50:24.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f08ab9f4640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a4163870 con 0x7f08a415cb50 2024-05-09T10:50:24.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f08ab9f4640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a4166240 con 0x7f08a40cfbc0 2024-05-09T10:50:24.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f08a8f8f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f08a415cb50 0x7f08a40cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f29b9c93640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f29b40cf050 0x7f29b415a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f29b9c93640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b415a950 con 0x7f29b40cf430 2024-05-09T10:50:24.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f29b9c93640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b415ac30 con 0x7f29b406dac0 2024-05-09T10:50:24.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f29b9c93640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b415af10 con 0x7f29b40cf050 2024-05-09T10:50:24.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.625+0000 7f92c4bd7640 1 --2- 172.21.15.98:0/619546514 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92b8039310 0x7f92b8008ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.625+0000 7f92bffff640 1 -- 172.21.15.98:0/619546514 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92b8039310 msgr2=0x7f92b8008ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:24.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.625+0000 7f92bffff640 1 --2- 172.21.15.98:0/619546514 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92b8039310 0x7f92b8008ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7f92bffff640 1 -- 172.21.15.98:0/619546514 >> v2:172.21.15.105:3300/0 conn(0x7f92b8050420 msgr2=0x7f92b8038dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T10:50:24.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7f92bffff640 1 --2- 172.21.15.98:0/619546514 >> v2:172.21.15.105:3300/0 conn(0x7f92b8050420 0x7f92b8038dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7f92bffff640 1 -- 172.21.15.98:0/619546514 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b8008000 con 0x7f92b80e0090 2024-05-09T10:50:24.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7f92c4bd7640 1 --2- 172.21.15.98:0/619546514 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92b8039310 0x7f92b8008ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:24.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f08a8f8f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f08a415cb50 0x7f08a40cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.98:35136/0 (socket says 172.21.15.98:35136) 2024-05-09T10:50:24.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f08a8f8f640 1 -- 172.21.15.98:0/942925801 learned_addr learned my addr 172.21.15.98:0/942925801 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f08a9f91640 1 --2- 172.21.15.98:0/942925801 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08a40cfbc0 0x7f08a40ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f29b2ffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f29b406dac0 0x7f29b4071400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.98:41246/0 (socket says 172.21.15.98:41246) 2024-05-09T10:50:24.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f29b2ffd640 1 -- 172.21.15.98:0/2006738179 learned_addr learned my addr 172.21.15.98:0/2006738179 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:24.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.626+0000 7f62453b7640 1 -- start start 2024-05-09T10:50:24.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7ffa48b27640 1 -- 172.21.15.98:0/66061445 >> v2:172.21.15.105:3300/0 conn(0x7ffa440baec0 msgr2=0x7ffa440d6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:24.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7ffa48b27640 1 --2- 172.21.15.98:0/66061445 >> v2:172.21.15.105:3300/0 conn(0x7ffa440baec0 0x7ffa440d6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7ffa48b27640 1 -- 172.21.15.98:0/66061445 >> v1:172.21.15.105:6789/0 conn(0x7ffa440d8690 legacy=0x7ffa44071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:24.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7ffa48b27640 1 -- 172.21.15.98:0/66061445 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa441663b0 con 0x7ffa440b7520 2024-05-09T10:50:24.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f62453b7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f62400afe60 0x7f6240165e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f62453b7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f624006e8a0 0x7f624006ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f08a9790640 1 --2- 172.21.15.98:0/942925801 >> v2:172.21.15.98:3300/0 conn(0x7f08a415c770 0x7f08a4163330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f62453b7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62400cd320 0x7f62400cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f62453b7640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62401667b0 con 0x7f62400afe60 2024-05-09T10:50:24.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f62453b7640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6240166a90 con 0x7f624006e8a0 2024-05-09T10:50:24.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f62453b7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62400ce9b0 con 0x7f62400cd320 2024-05-09T10:50:24.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f08a8f8f640 1 -- 172.21.15.98:0/942925801 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08a40cfbc0 msgr2=0x7f08a40ae4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:24.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f08a8f8f640 1 --2- 172.21.15.98:0/942925801 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08a40cfbc0 0x7f08a40ae4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f08a8f8f640 1 -- 172.21.15.98:0/942925801 >> v2:172.21.15.98:3300/0 conn(0x7f08a415c770 msgr2=0x7f08a4163330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:24.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f08a8f8f640 1 --2- 172.21.15.98:0/942925801 >> v2:172.21.15.98:3300/0 conn(0x7f08a415c770 0x7f08a4163330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f08a8f8f640 1 -- 172.21.15.98:0/942925801 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a4166520 con 0x7f08a415cb50 2024-05-09T10:50:24.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7f623e7fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f624006e8a0 0x7f624006ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7f92bffff640 1 --2- 172.21.15.98:0/619546514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92b80e0090 0x7f92b8034fb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f92b8053ac0 tx=0x7f92b402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4d4dfe11449776 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7f08a9790640 1 --2- 172.21.15.98:0/942925801 >> v2:172.21.15.98:3300/0 conn(0x7f08a415c770 0x7f08a4163330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:24.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f6dbdd9b640 1 -- 172.21.15.98:0/3726393882 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6dc0165550 msgr2=0x7f6dc00cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:24.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f6dbdd9b640 1 --2- 172.21.15.98:0/3726393882 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6dc0165550 0x7f6dc00cfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f6dbdd9b640 1 -- 172.21.15.98:0/3726393882 >> v2:172.21.15.98:3300/0 conn(0x7f6dc00b46e0 msgr2=0x7f6dc00b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:24.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f6dbdd9b640 1 --2- 172.21.15.98:0/3726393882 >> v2:172.21.15.98:3300/0 conn(0x7f6dc00b46e0 0x7f6dc00b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.627+0000 7f6dbdd9b640 1 -- 172.21.15.98:0/3726393882 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc00acd40 con 0x7f6dc0165930 2024-05-09T10:50:24.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7f623e7fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f624006e8a0 0x7f624006ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.98:35140/0 (socket says 172.21.15.98:35140) 2024-05-09T10:50:24.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7f29b37fe640 1 --2- 172.21.15.98:0/2006738179 >> v2:172.21.15.98:3300/0 conn(0x7f29b40cf050 0x7f29b415a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7fb2d9848640 1 --2- 172.21.15.98:0/1059571630 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2cc0e12b0 0x7fb2cc0534f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000f9b0 tx=0x7fb2d002e570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae437a61672404a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7f623e7fc640 1 -- 172.21.15.98:0/522190869 learned_addr learned my addr 172.21.15.98:0/522190869 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7f623ffff640 1 --2- 172.21.15.98:0/522190869 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62400cd320 0x7f62400cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7f623effd640 1 --2- 172.21.15.98:0/522190869 >> v2:172.21.15.98:3300/0 conn(0x7f62400afe60 0x7f6240165e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7f29b27fc640 1 -- 172.21.15.98:0/2006738179 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2013984795 0 0) 0x7f29b415ac30 con 0x7f29b406dac0 2024-05-09T10:50:24.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7f29b27fc640 1 -- 172.21.15.98:0/2006738179 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f299c005730 con 0x7f29b406dac0 2024-05-09T10:50:24.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7f29b27fc640 1 -- 172.21.15.98:0/2006738179 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 806327671 0 0) 0x7f29b415a950 con 0x7f29b40cf430 2024-05-09T10:50:24.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7f29b27fc640 1 -- 172.21.15.98:0/2006738179 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29b415ac30 con 0x7f29b40cf430 2024-05-09T10:50:24.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7f92beffd640 1 -- 172.21.15.98:0/619546514 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4037070 con 0x7f92b80e0090 2024-05-09T10:50:24.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7f92beffd640 1 -- 172.21.15.98:0/619546514 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b4032440 con 0x7f92b80e0090 2024-05-09T10:50:24.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7f623e7fc640 1 -- 172.21.15.98:0/522190869 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62400cd320 msgr2=0x7f62400cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:24.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7f623e7fc640 1 --2- 172.21.15.98:0/522190869 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62400cd320 0x7f62400cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7f623e7fc640 1 -- 172.21.15.98:0/522190869 >> v2:172.21.15.98:3300/0 conn(0x7f62400afe60 msgr2=0x7f6240165e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:24.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7f623e7fc640 1 --2- 172.21.15.98:0/522190869 >> v2:172.21.15.98:3300/0 conn(0x7f62400afe60 0x7f6240165e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7f623e7fc640 1 -- 172.21.15.98:0/522190869 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62400cec90 con 0x7f624006e8a0 2024-05-09T10:50:24.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7f29b37fe640 1 -- 172.21.15.98:0/2006738179 >> v1:172.21.15.105:6789/0 conn(0x7f29b406dac0 legacy=0x7f29b4071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7f29b37fe640 1 -- 172.21.15.98:0/2006738179 >> v1:172.21.15.98:6789/0 conn(0x7f29b40cf430 legacy=0x7f29b40cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7f29b37fe640 1 -- 172.21.15.98:0/2006738179 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b415b1f0 con 0x7f29b40cf050 2024-05-09T10:50:24.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7fce98c6b640 1 -- 172.21.15.98:0/1069721617 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce9415a190 msgr2=0x7fce940668f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.628+0000 7fce98c6b640 1 --2- 172.21.15.98:0/1069721617 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce9415a190 0x7fce940668f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fce88009870 tx=0x7fce8802a210 comp rx=0 tx=0).stop 2024-05-09T10:50:24.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7fce98c6b640 1 -- 172.21.15.98:0/1069721617 shutdown_connections 2024-05-09T10:50:24.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7fce98c6b640 1 --2- 172.21.15.98:0/1069721617 >> v2:172.21.15.98:3300/0 conn(0x7fce9406c300 0x7fce940b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7fce98c6b640 1 --2- 172.21.15.98:0/1069721617 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce9415a190 0x7fce940668f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7fce98c6b640 1 -- 172.21.15.98:0/1069721617 >> 172.21.15.98:0/1069721617 conn(0x7fce9406cae0 msgr2=0x7fce9406cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7fce98c6b640 1 -- 172.21.15.98:0/1069721617 shutdown_connections 2024-05-09T10:50:24.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7ffa48b27640 1 --2- 172.21.15.98:0/66061445 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa440b7520 0x7ffa44165d60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009e50 tx=0x7ffa3c02a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16657ecd693a2887 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7fce98c6b640 1 -- 172.21.15.98:0/1069721617 wait complete. 2024-05-09T10:50:24.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7f92beffd640 1 -- 172.21.15.98:0/619546514 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4032810 con 0x7f92b80e0090 2024-05-09T10:50:24.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7f08a8f8f640 1 --2- 172.21.15.98:0/942925801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08a415cb50 0x7f08a40cf680 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f08a406e350 tx=0x7f08a002a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4ce5d65dd74f823 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7fb2d8846640 1 -- 172.21.15.98:0/1059571630 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d003a070 con 0x7fb2cc0e12b0 2024-05-09T10:50:24.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7fb2d8846640 1 -- 172.21.15.98:0/1059571630 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d0033710 con 0x7fb2cc0e12b0 2024-05-09T10:50:24.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7fb2d8846640 1 -- 172.21.15.98:0/1059571630 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0033ae0 con 0x7fb2cc0e12b0 2024-05-09T10:50:24.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7f92c663a640 1 -- 172.21.15.98:0/619546514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92b80e0090 msgr2=0x7f92b8034fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7f92c663a640 1 --2- 172.21.15.98:0/619546514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92b80e0090 0x7f92b8034fb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f92b8053ac0 tx=0x7f92b402a160 comp rx=0 tx=0).stop 2024-05-09T10:50:24.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7f92c663a640 1 -- 172.21.15.98:0/619546514 shutdown_connections 2024-05-09T10:50:24.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7f92c663a640 1 --2- 172.21.15.98:0/619546514 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92b8039310 0x7f92b8008ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7f92c663a640 1 --2- 172.21.15.98:0/619546514 >> v2:172.21.15.105:3300/0 conn(0x7f92b8050420 0x7f92b8038dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7f92c663a640 1 --2- 172.21.15.98:0/619546514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92b80e0090 0x7f92b8034fb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7ffa437fe640 1 -- 172.21.15.98:0/66061445 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c036070 con 0x7ffa440b7520 2024-05-09T10:50:24.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7ffa437fe640 1 -- 172.21.15.98:0/66061445 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3c031600 con 0x7ffa440b7520 2024-05-09T10:50:24.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7ffa437fe640 1 -- 172.21.15.98:0/66061445 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c0319d0 con 0x7ffa440b7520 2024-05-09T10:50:24.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7f92c663a640 1 -- 172.21.15.98:0/619546514 >> 172.21.15.98:0/619546514 conn(0x7f92b8055bc0 msgr2=0x7f92b8055fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7f6dbdd9b640 1 --2- 172.21.15.98:0/3726393882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6dc0165930 0x7f6dc00d0400 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6dc006bb40 tx=0x7f6db402a1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc39f8d133184163 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7f92c663a640 1 -- 172.21.15.98:0/619546514 shutdown_connections 2024-05-09T10:50:24.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7f089bfff640 1 -- 172.21.15.98:0/942925801 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0037070 con 0x7f08a415cb50 2024-05-09T10:50:24.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7f089bfff640 1 -- 172.21.15.98:0/942925801 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a0032440 con 0x7f08a415cb50 2024-05-09T10:50:24.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7f089bfff640 1 -- 172.21.15.98:0/942925801 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0032810 con 0x7f08a415cb50 2024-05-09T10:50:24.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7f623e7fc640 1 --2- 172.21.15.98:0/522190869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624006e8a0 0x7f624006ed00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6240070dd0 tx=0x7f623402a140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33cee4f758f98e7c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7f92c663a640 1 -- 172.21.15.98:0/619546514 wait complete. 2024-05-09T10:50:24.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.630+0000 7f623dffb640 1 -- 172.21.15.98:0/522190869 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6234037070 con 0x7f624006e8a0 2024-05-09T10:50:24.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.629+0000 7f29b37fe640 1 --2- 172.21.15.98:0/2006738179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29b40cf050 0x7f29b415a370 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f29a40097a0 tx=0x7f29a402a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0bd1bf9dc4fd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.630+0000 7f623dffb640 1 -- 172.21.15.98:0/522190869 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6234032440 con 0x7f624006e8a0 2024-05-09T10:50:24.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.630+0000 7f29b27fc640 1 -- 172.21.15.98:0/2006738179 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a4037070 con 0x7f29b40cf050 2024-05-09T10:50:24.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.630+0000 7f29b27fc640 1 -- 172.21.15.98:0/2006738179 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a4032640 con 0x7f29b40cf050 2024-05-09T10:50:24.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.630+0000 7f29b27fc640 1 -- 172.21.15.98:0/2006738179 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a4032a10 con 0x7f29b40cf050 2024-05-09T10:50:24.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.630+0000 7fb2da84a640 1 -- 172.21.15.98:0/1059571630 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2cc0e12b0 msgr2=0x7fb2cc0534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.630+0000 7fb2da84a640 1 --2- 172.21.15.98:0/1059571630 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2cc0e12b0 0x7fb2cc0534f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000f9b0 tx=0x7fb2d002e570 comp rx=0 tx=0).stop 2024-05-09T10:50:24.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.630+0000 7fb2da84a640 1 -- 172.21.15.98:0/1059571630 shutdown_connections 2024-05-09T10:50:24.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.630+0000 7f08ab9f4640 1 -- 172.21.15.98:0/942925801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08a415cb50 msgr2=0x7f08a40cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.630+0000 7f08ab9f4640 1 --2- 172.21.15.98:0/942925801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08a415cb50 0x7f08a40cf680 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f08a406e350 tx=0x7f08a002a1e0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.630+0000 7fb2da84a640 1 --2- 172.21.15.98:0/1059571630 >> v2:172.21.15.105:3300/0 conn(0x7fb2cc054b50 0x7fb2cc053a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.630+0000 7fb2da84a640 1 --2- 172.21.15.98:0/1059571630 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2cc0e12b0 0x7fb2cc0534f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.630+0000 7fb2da84a640 1 -- 172.21.15.98:0/1059571630 >> 172.21.15.98:0/1059571630 conn(0x7fb2cc020040 msgr2=0x7fb2cc056d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.630+0000 7fce98c6b640 1 Processor -- start 2024-05-09T10:50:24.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.630+0000 7fce98c6b640 1 -- start start 2024-05-09T10:50:24.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7f08ab9f4640 1 -- 172.21.15.98:0/942925801 shutdown_connections 2024-05-09T10:50:24.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7f08ab9f4640 1 --2- 172.21.15.98:0/942925801 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08a40cfbc0 0x7f08a40ae4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7f08ab9f4640 1 --2- 172.21.15.98:0/942925801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08a415cb50 0x7f08a40cf680 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7f08ab9f4640 1 --2- 172.21.15.98:0/942925801 >> v2:172.21.15.98:3300/0 conn(0x7f08a415c770 0x7f08a4163330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7f08ab9f4640 1 -- 172.21.15.98:0/942925801 >> 172.21.15.98:0/942925801 conn(0x7f08a406abe0 msgr2=0x7f08a406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.630+0000 7ffa4ad8b640 1 -- 172.21.15.98:0/66061445 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa440b7520 msgr2=0x7ffa44165d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.630+0000 7ffa4ad8b640 1 --2- 172.21.15.98:0/66061445 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa440b7520 0x7ffa44165d60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009e50 tx=0x7ffa3c02a640 comp rx=0 tx=0).stop 2024-05-09T10:50:24.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.630+0000 7ffa4ad8b640 1 -- 172.21.15.98:0/66061445 shutdown_connections 2024-05-09T10:50:24.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.630+0000 7ffa4ad8b640 1 --2- 172.21.15.98:0/66061445 >> v2:172.21.15.105:3300/0 conn(0x7ffa440baec0 0x7ffa440d6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.630+0000 7ffa4ad8b640 1 --2- 172.21.15.98:0/66061445 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa440b7520 0x7ffa44165d60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.630+0000 7ffa4ad8b640 1 -- 172.21.15.98:0/66061445 >> 172.21.15.98:0/66061445 conn(0x7ffa44066f20 msgr2=0x7ffa4406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.630+0000 7ffa4ad8b640 1 -- 172.21.15.98:0/66061445 shutdown_connections 2024-05-09T10:50:24.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.630+0000 7ffa4ad8b640 1 -- 172.21.15.98:0/66061445 wait complete. 2024-05-09T10:50:24.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7fb2da84a640 1 -- 172.21.15.98:0/1059571630 shutdown_connections 2024-05-09T10:50:24.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7fb2da84a640 1 -- 172.21.15.98:0/1059571630 wait complete. 2024-05-09T10:50:24.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7f08ab9f4640 1 -- 172.21.15.98:0/942925801 shutdown_connections 2024-05-09T10:50:24.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7f08ab9f4640 1 -- 172.21.15.98:0/942925801 wait complete. 2024-05-09T10:50:24.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7f29b9c93640 1 -- 172.21.15.98:0/2006738179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29b40cf050 msgr2=0x7f29b415a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7f29b9c93640 1 --2- 172.21.15.98:0/2006738179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29b40cf050 0x7f29b415a370 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f29a40097a0 tx=0x7f29a402a330 comp rx=0 tx=0).stop 2024-05-09T10:50:24.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7f92c663a640 1 Processor -- start 2024-05-09T10:50:24.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7f623dffb640 1 -- 172.21.15.98:0/522190869 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6234032810 con 0x7f624006e8a0 2024-05-09T10:50:24.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7f62453b7640 1 -- 172.21.15.98:0/522190869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624006e8a0 msgr2=0x7f624006ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7f62453b7640 1 --2- 172.21.15.98:0/522190869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624006e8a0 0x7f624006ed00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6240070dd0 tx=0x7f623402a140 comp rx=0 tx=0).stop 2024-05-09T10:50:24.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7f29b9c93640 1 -- 172.21.15.98:0/2006738179 shutdown_connections 2024-05-09T10:50:24.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7f29b9c93640 1 --2- 172.21.15.98:0/2006738179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29b40cf050 0x7f29b415a370 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7f29b9c93640 1 -- 172.21.15.98:0/2006738179 >> 172.21.15.98:0/2006738179 conn(0x7f29b406abe0 msgr2=0x7f29b406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7f6dbd59a640 1 -- 172.21.15.98:0/3726393882 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db4037070 con 0x7f6dc0165930 2024-05-09T10:50:24.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7f6dbd59a640 1 -- 172.21.15.98:0/3726393882 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db4032440 con 0x7f6dc0165930 2024-05-09T10:50:24.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7f29b9c93640 1 -- 172.21.15.98:0/2006738179 shutdown_connections 2024-05-09T10:50:24.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7f92c663a640 1 -- start start 2024-05-09T10:50:24.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7f6dbd59a640 1 -- 172.21.15.98:0/3726393882 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db4032810 con 0x7f6dc0165930 2024-05-09T10:50:24.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7fce98c6b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce9406c300 0x7fce9415b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7fce98c6b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce940b4680 0x7fce9415b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7fce98c6b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9415a190 0x7fce9415bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7fce98c6b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce94066fb0 con 0x7fce9406c300 2024-05-09T10:50:24.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7fce98c6b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce94066e30 con 0x7fce9415a190 2024-05-09T10:50:24.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7fce98c6b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9415a570 con 0x7fce940b4680 2024-05-09T10:50:24.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.632+0000 7fce9259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce9406c300 0x7fce9415b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.632+0000 7fce92d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9415a190 0x7fce9415bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.632+0000 7fce9259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce9406c300 0x7fce9415b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41962/0 (socket says 172.21.15.98:41962) 2024-05-09T10:50:24.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.632+0000 7fce92d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9415a190 0x7fce9415bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.98:35144/0 (socket says 172.21.15.98:35144) 2024-05-09T10:50:24.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.632+0000 7fce9259c640 1 -- 172.21.15.98:0/2779381409 learned_addr learned my addr 172.21.15.98:0/2779381409 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.632+0000 7fce91d9b640 1 --2- 172.21.15.98:0/2779381409 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce940b4680 0x7fce9415b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.631+0000 7f29b9c93640 1 -- 172.21.15.98:0/2006738179 wait complete. 2024-05-09T10:50:24.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.632+0000 7fce92d9d640 1 -- 172.21.15.98:0/2779381409 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce940b4680 msgr2=0x7fce9415b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.632+0000 7fce92d9d640 1 --2- 172.21.15.98:0/2779381409 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce940b4680 0x7fce9415b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.632+0000 7fce92d9d640 1 -- 172.21.15.98:0/2779381409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce9406c300 msgr2=0x7fce9415b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.632+0000 7fce92d9d640 1 --2- 172.21.15.98:0/2779381409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce9406c300 0x7fce9415b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.632+0000 7fce92d9d640 1 -- 172.21.15.98:0/2779381409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9413d9a0 con 0x7fce9415a190 2024-05-09T10:50:24.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.632+0000 7fce9259c640 1 --2- 172.21.15.98:0/2779381409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce9406c300 0x7fce9415b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:50:24.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.632+0000 7f62453b7640 1 -- 172.21.15.98:0/522190869 shutdown_connections 2024-05-09T10:50:24.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.632+0000 7f62453b7640 1 --2- 172.21.15.98:0/522190869 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62400cd320 0x7f62400cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.632+0000 7f6dc4ce1640 1 -- 172.21.15.98:0/3726393882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6dc0165930 msgr2=0x7f6dc00d0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.632+0000 7f6dc4ce1640 1 --2- 172.21.15.98:0/3726393882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6dc0165930 0x7f6dc00d0400 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6dc006bb40 tx=0x7f6db402a1d0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.632+0000 7f62453b7640 1 --2- 172.21.15.98:0/522190869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624006e8a0 0x7f624006ed00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.632+0000 7f62453b7640 1 --2- 172.21.15.98:0/522190869 >> v2:172.21.15.98:3300/0 conn(0x7f62400afe60 0x7f6240165e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.632+0000 7fce92d9d640 1 --2- 172.21.15.98:0/2779381409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9415a190 0x7fce9415bef0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fce940b1d90 tx=0x7fce8c00ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7fb2da84a640 1 Processor -- start 2024-05-09T10:50:24.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.632+0000 7f62453b7640 1 -- 172.21.15.98:0/522190869 >> 172.21.15.98:0/522190869 conn(0x7f62400b2530 msgr2=0x7f62400b2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7f62453b7640 1 -- 172.21.15.98:0/522190869 shutdown_connections 2024-05-09T10:50:24.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7f62453b7640 1 -- 172.21.15.98:0/522190869 wait complete. 2024-05-09T10:50:24.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7ffa4ad8b640 1 Processor -- start 2024-05-09T10:50:24.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7fce82ffd640 1 -- 172.21.15.98:0/2779381409 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c018030 con 0x7fce9415a190 2024-05-09T10:50:24.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7fce82ffd640 1 -- 172.21.15.98:0/2779381409 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8c00ccc0 con 0x7fce9415a190 2024-05-09T10:50:24.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7f92c663a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92b8039310 0x7f92b80e2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7fce82ffd640 1 -- 172.21.15.98:0/2779381409 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c003bb0 con 0x7fce9415a190 2024-05-09T10:50:24.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7fce98c6b640 1 -- 172.21.15.98:0/2779381409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce94138930 con 0x7fce9415a190 2024-05-09T10:50:24.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7fce98c6b640 1 -- 172.21.15.98:0/2779381409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce9413e480 con 0x7fce9415a190 2024-05-09T10:50:24.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7ffa4ad8b640 1 -- start start 2024-05-09T10:50:24.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7f6dc4ce1640 1 -- 172.21.15.98:0/3726393882 shutdown_connections 2024-05-09T10:50:24.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7f6dc4ce1640 1 --2- 172.21.15.98:0/3726393882 >> v2:172.21.15.98:3300/0 conn(0x7f6dc00b46e0 0x7f6dc00b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7f6dc4ce1640 1 --2- 172.21.15.98:0/3726393882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6dc0165930 0x7f6dc00d0400 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7f6dc4ce1640 1 --2- 172.21.15.98:0/3726393882 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6dc0165550 0x7f6dc00cfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7f6dc4ce1640 1 -- 172.21.15.98:0/3726393882 >> 172.21.15.98:0/3726393882 conn(0x7f6dc006adc0 msgr2=0x7f6dc006b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7f29b9c93640 1 Processor -- start 2024-05-09T10:50:24.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.632+0000 7f08ab9f4640 1 Processor -- start 2024-05-09T10:50:24.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7f08ab9f4640 1 -- start start 2024-05-09T10:50:24.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7f92c663a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92b8050420 0x7f92b8054730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7f92c663a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92b80e0090 0x7f92b80dd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7f92c663a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b8009020 con 0x7f92b8039310 2024-05-09T10:50:24.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7f92bffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92b8039310 0x7f92b80e2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7f92bffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92b8039310 0x7f92b80e2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41968/0 (socket says 172.21.15.98:41968) 2024-05-09T10:50:24.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7f6dc4ce1640 1 -- 172.21.15.98:0/3726393882 shutdown_connections 2024-05-09T10:50:24.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7f92bffff640 1 -- 172.21.15.98:0/3896757311 learned_addr learned my addr 172.21.15.98:0/3896757311 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7f92c663a640 1 -- 172.21.15.98:0/3896757311 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b80085c0 con 0x7f92b80e0090 2024-05-09T10:50:24.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7f92c663a640 1 -- 172.21.15.98:0/3896757311 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b80091a0 con 0x7f92b8050420 2024-05-09T10:50:24.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.634+0000 7f08ab9f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08a40cfbc0 0x7f08a414f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.634+0000 7f92bf7fe640 1 --2- 172.21.15.98:0/3896757311 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92b8050420 0x7f92b8054730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.634+0000 7f92bffff640 1 -- 172.21.15.98:0/3896757311 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92b8050420 msgr2=0x7f92b8054730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.634+0000 7f92bffff640 1 --2- 172.21.15.98:0/3896757311 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92b8050420 0x7f92b8054730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.634+0000 7f92bffff640 1 -- 172.21.15.98:0/3896757311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92b80e0090 msgr2=0x7f92b80dd540 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T10:50:24.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.634+0000 7f92bffff640 1 --2- 172.21.15.98:0/3896757311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92b80e0090 0x7f92b80dd540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.634+0000 7f92bffff640 1 -- 172.21.15.98:0/3896757311 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b80e2540 con 0x7f92b8039310 2024-05-09T10:50:24.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.634+0000 7f08ab9f4640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08a415c770 0x7f08a415d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.634+0000 7f29b9c93640 1 -- start start 2024-05-09T10:50:24.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.634+0000 7f08ab9f4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08a415cb50 0x7f08a415d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.633+0000 7fb2da84a640 1 -- start start 2024-05-09T10:50:24.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.634+0000 7fb2da84a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2cc0e12b0 0x7fb2cc04bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.634+0000 7fb2da84a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2cc0e4d30 0x7fb2cc04cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.634+0000 7fb2da84a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2cc04de40 0x7fb2cc17d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.634+0000 7fb2da84a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc057820 con 0x7fb2cc0e12b0 2024-05-09T10:50:24.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.634+0000 7fb2da84a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc0576a0 con 0x7fb2cc0e4d30 2024-05-09T10:50:24.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.634+0000 7f92bffff640 1 --2- 172.21.15.98:0/3896757311 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92b8039310 0x7f92b80e2000 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f92b4000c00 tx=0x7f92b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.634+0000 7fb2da84a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc0e1da0 con 0x7fb2cc04de40 2024-05-09T10:50:24.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.634+0000 7f08ab9f4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a40ce2f0 con 0x7f08a415cb50 2024-05-09T10:50:24.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.634+0000 7f08ab9f4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a4166ae0 con 0x7f08a40cfbc0 2024-05-09T10:50:24.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.634+0000 7f08ab9f4640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a40ce470 con 0x7f08a415c770 2024-05-09T10:50:24.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7ffa4ad8b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa440b7520 0x7ffa440b3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7ffa4ad8b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa440baec0 0x7ffa441ebae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7ffa4ad8b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa440d8690 0x7ffa44164ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7ffa4ad8b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa44166c50 con 0x7ffa440baec0 2024-05-09T10:50:24.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7ffa4ad8b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa44166970 con 0x7ffa440d8690 2024-05-09T10:50:24.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7ffa4ad8b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa440d6f90 con 0x7ffa440b7520 2024-05-09T10:50:24.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7fb2da049640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2cc04de40 0x7fb2cc17d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7fb2da049640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2cc04de40 0x7fb2cc17d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:53174/0 (socket says 172.21.15.98:53174) 2024-05-09T10:50:24.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7fb2da049640 1 -- 172.21.15.98:0/2783980682 learned_addr learned my addr 172.21.15.98:0/2783980682 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7f62453b7640 1 Processor -- start 2024-05-09T10:50:24.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7f62453b7640 1 -- start start 2024-05-09T10:50:24.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7ffa48b27640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa440b7520 0x7ffa440b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7f29b9c93640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29b406dac0 0x7f29b41fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7f29b9c93640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f29b40cf050 0x7f29b41ed690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7fb2d9848640 1 --2- 172.21.15.98:0/2783980682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2cc0e12b0 0x7fb2cc04bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7f92bcff9640 1 -- 172.21.15.98:0/3896757311 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4048070 con 0x7f92b8039310 2024-05-09T10:50:24.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7f92bcff9640 1 -- 172.21.15.98:0/3896757311 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b40304c0 con 0x7f92b8039310 2024-05-09T10:50:24.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7f92c663a640 1 -- 172.21.15.98:0/3896757311 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b8168c70 con 0x7f92b8039310 2024-05-09T10:50:24.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.634+0000 7f6dc4ce1640 1 -- 172.21.15.98:0/3726393882 wait complete. 2024-05-09T10:50:24.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7ffa48b27640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa440b7520 0x7ffa440b3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:53180/0 (socket says 172.21.15.98:53180) 2024-05-09T10:50:24.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7ffa48b27640 1 -- 172.21.15.98:0/1360316151 learned_addr learned my addr 172.21.15.98:0/1360316151 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7f29b9c93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29b40cf430 0x7f29b41edbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7fb2da049640 1 -- 172.21.15.98:0/2783980682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2cc0e4d30 msgr2=0x7fb2cc04cd10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T10:50:24.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7ffa49328640 1 --2- 172.21.15.98:0/1360316151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa440d8690 0x7ffa44164ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7fb2da049640 1 --2- 172.21.15.98:0/2783980682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2cc0e4d30 0x7fb2cc04cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7fb2da049640 1 -- 172.21.15.98:0/2783980682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2cc0e12b0 msgr2=0x7fb2cc04bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7fb2da049640 1 --2- 172.21.15.98:0/2783980682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2cc0e12b0 0x7fb2cc04bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7fce82ffd640 1 -- 172.21.15.98:0/2779381409 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce8c005d10 con 0x7fce9415a190 2024-05-09T10:50:24.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7fb2da049640 1 -- 172.21.15.98:0/2783980682 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2cc04d560 con 0x7fb2cc04de40 2024-05-09T10:50:24.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7f29b2ffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f29b40cf050 0x7f29b41ed690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7f29b2ffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f29b40cf050 0x7f29b41ed690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:53190/0 (socket says 172.21.15.98:53190) 2024-05-09T10:50:24.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7f29b2ffd640 1 -- 172.21.15.98:0/1655145821 learned_addr learned my addr 172.21.15.98:0/1655145821 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f29b3fff640 1 --2- 172.21.15.98:0/1655145821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29b40cf430 0x7f29b41edbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f29b9c93640 1 -- 172.21.15.98:0/1655145821 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b406e0c0 con 0x7f29b406dac0 2024-05-09T10:50:24.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f29b9c93640 1 -- 172.21.15.98:0/1655145821 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b406df40 con 0x7f29b40cf430 2024-05-09T10:50:24.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f29b9c93640 1 -- 172.21.15.98:0/1655145821 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b406e3b0 con 0x7f29b40cf050 2024-05-09T10:50:24.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7fb2d9848640 1 --2- 172.21.15.98:0/2783980682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2cc0e12b0 0x7fb2cc04bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:24.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7f08a9790640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08a40cfbc0 0x7f08a414f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7f08a9790640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08a40cfbc0 0x7f08a414f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.98:35176/0 (socket says 172.21.15.98:35176) 2024-05-09T10:50:24.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.635+0000 7f08a9790640 1 -- 172.21.15.98:0/294168447 learned_addr learned my addr 172.21.15.98:0/294168447 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f08a9790640 1 -- 172.21.15.98:0/294168447 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08a415c770 msgr2=0x7f08a415d050 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T10:50:24.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f08a9790640 1 --2- 172.21.15.98:0/294168447 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08a415c770 0x7f08a415d050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f08a9790640 1 -- 172.21.15.98:0/294168447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08a415cb50 msgr2=0x7f08a415d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f08a9790640 1 --2- 172.21.15.98:0/294168447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08a415cb50 0x7f08a415d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f08a9790640 1 -- 172.21.15.98:0/294168447 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a4149950 con 0x7f08a40cfbc0 2024-05-09T10:50:24.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7ffa43fff640 1 --2- 172.21.15.98:0/1360316151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa440baec0 0x7ffa441ebae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7ffa48b27640 1 -- 172.21.15.98:0/1360316151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa440d8690 msgr2=0x7ffa44164ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7ffa48b27640 1 --2- 172.21.15.98:0/1360316151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa440d8690 0x7ffa44164ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7ffa48b27640 1 -- 172.21.15.98:0/1360316151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa440baec0 msgr2=0x7ffa441ebae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7ffa48b27640 1 --2- 172.21.15.98:0/1360316151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa440baec0 0x7ffa441ebae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f92c663a640 1 -- 172.21.15.98:0/3896757311 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b8169230 con 0x7f92b8039310 2024-05-09T10:50:24.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7ffa48b27640 1 -- 172.21.15.98:0/1360316151 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa440b3990 con 0x7ffa440b7520 2024-05-09T10:50:24.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7fb2da049640 1 --2- 172.21.15.98:0/2783980682 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2cc04de40 0x7fb2cc17d3c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400c3f0 tx=0x7fb2c400c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7ffa43fff640 1 --2- 172.21.15.98:0/1360316151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa440baec0 0x7ffa441ebae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:24.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f62453b7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624006e8a0 0x7f62401f8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f6dc4ce1640 1 Processor -- start 2024-05-09T10:50:24.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f08a9790640 1 --2- 172.21.15.98:0/294168447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08a40cfbc0 0x7f08a414f4f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f089c00cf60 tx=0x7f089c00cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f62453b7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62400afe60 0x7f62401629a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f6dc4ce1640 1 -- start start 2024-05-09T10:50:24.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f29b37fe640 1 --2- 172.21.15.98:0/1655145821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29b406dac0 0x7f29b41fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f29b2ffd640 1 -- 172.21.15.98:0/1655145821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29b40cf430 msgr2=0x7f29b41edbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f29b2ffd640 1 --2- 172.21.15.98:0/1655145821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29b40cf430 0x7f29b41edbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f29b2ffd640 1 -- 172.21.15.98:0/1655145821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29b406dac0 msgr2=0x7f29b41fbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f29b2ffd640 1 --2- 172.21.15.98:0/1655145821 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29b406dac0 0x7f29b41fbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f29b2ffd640 1 -- 172.21.15.98:0/1655145821 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b41f88f0 con 0x7f29b40cf050 2024-05-09T10:50:24.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f62453b7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62400cd320 0x7f6240161640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f62453b7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6240166380 con 0x7f62400afe60 2024-05-09T10:50:24.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f62453b7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624006f240 con 0x7f624006e8a0 2024-05-09T10:50:24.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f62453b7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6240166500 con 0x7f62400cd320 2024-05-09T10:50:24.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f623e7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62400afe60 0x7f62401629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f623effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624006e8a0 0x7f62401f8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f623e7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62400afe60 0x7f62401629a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42016/0 (socket says 172.21.15.98:42016) 2024-05-09T10:50:24.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f623e7fc640 1 -- 172.21.15.98:0/3183400844 learned_addr learned my addr 172.21.15.98:0/3183400844 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f29b3fff640 1 --2- 172.21.15.98:0/1655145821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29b40cf430 0x7f29b41edbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:24.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f623ffff640 1 --2- 172.21.15.98:0/3183400844 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62400cd320 0x7f6240161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7fce82ffd640 1 --2- 172.21.15.98:0/2779381409 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fce7406b5a0 0x7fce7406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7fce82ffd640 1 -- 172.21.15.98:0/2779381409 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce8c08a880 con 0x7fce9415a190 2024-05-09T10:50:24.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7ffa48b27640 1 --2- 172.21.15.98:0/1360316151 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa440b7520 0x7ffa440b3450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009f70 tx=0x7ffa3c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.636+0000 7f0899ffb640 1 -- 172.21.15.98:0/294168447 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c01a070 con 0x7f08a40cfbc0 2024-05-09T10:50:24.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.637+0000 7f0899ffb640 1 -- 172.21.15.98:0/294168447 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089c00d840 con 0x7f08a40cfbc0 2024-05-09T10:50:24.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.637+0000 7f0899ffb640 1 -- 172.21.15.98:0/294168447 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c01a070 con 0x7f08a40cfbc0 2024-05-09T10:50:24.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.637+0000 7f08ab9f4640 1 -- 172.21.15.98:0/294168447 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a415dad0 con 0x7f08a40cfbc0 2024-05-09T10:50:24.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.637+0000 7fb2ca7fc640 1 -- 172.21.15.98:0/2783980682 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c40052d0 con 0x7fb2cc04de40 2024-05-09T10:50:24.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.637+0000 7fce9259c640 1 --2- 172.21.15.98:0/2779381409 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fce7406b5a0 0x7fce7406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.637+0000 7fb2ca7fc640 1 -- 172.21.15.98:0/2783980682 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c4005470 con 0x7fb2cc04de40 2024-05-09T10:50:24.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.637+0000 7fb2ca7fc640 1 -- 172.21.15.98:0/2783980682 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c40052d0 con 0x7fb2cc04de40 2024-05-09T10:50:24.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.637+0000 7f623effd640 1 -- 172.21.15.98:0/3183400844 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62400cd320 msgr2=0x7f6240161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.637+0000 7f623effd640 1 --2- 172.21.15.98:0/3183400844 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62400cd320 0x7f6240161640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.637+0000 7f623effd640 1 -- 172.21.15.98:0/3183400844 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62400afe60 msgr2=0x7f62401629a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.637+0000 7f623effd640 1 --2- 172.21.15.98:0/3183400844 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62400afe60 0x7f62401629a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.637+0000 7f623effd640 1 -- 172.21.15.98:0/3183400844 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6240165660 con 0x7f624006e8a0 2024-05-09T10:50:24.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.637+0000 7f08ab9f4640 1 -- 172.21.15.98:0/294168447 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a414c0d0 con 0x7f08a40cfbc0 2024-05-09T10:50:24.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.637+0000 7fce98c6b640 1 --2- 172.21.15.98:0/2779381409 >> [v2:172.21.15.98:6808/2633494785,v1:172.21.15.98:6809/2633494785] conn(0x7fce78001650 0x7fce78003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7f6dc4ce1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dc00b46e0 0x7f6dc00adc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.637+0000 7f29b2ffd640 1 --2- 172.21.15.98:0/1655145821 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f29b40cf050 0x7f29b41ed690 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f29a800d570 tx=0x7f29a800da30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.637+0000 7f299bfff640 1 -- 172.21.15.98:0/1655145821 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a8014070 con 0x7f29b40cf050 2024-05-09T10:50:24.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.637+0000 7f299bfff640 1 -- 172.21.15.98:0/1655145821 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a8009040 con 0x7f29b40cf050 2024-05-09T10:50:24.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7f6dc4ce1640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6dc0165550 0x7f6dc00c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7f623effd640 1 --2- 172.21.15.98:0/3183400844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624006e8a0 0x7f62401f8180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f623000d5f0 tx=0x7f623000dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7ffa417fa640 1 -- 172.21.15.98:0/1360316151 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c041020 con 0x7ffa440b7520 2024-05-09T10:50:24.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7fce91d9b640 1 --2- 172.21.15.98:0/2779381409 >> [v2:172.21.15.98:6808/2633494785,v1:172.21.15.98:6809/2633494785] conn(0x7fce78001650 0x7fce78003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7f299bfff640 1 -- 172.21.15.98:0/1655145821 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a800b8d0 con 0x7f29b40cf050 2024-05-09T10:50:24.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7ffa4ad8b640 1 -- 172.21.15.98:0/1360316151 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa441ec3e0 con 0x7ffa440b7520 2024-05-09T10:50:24.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7ffa4ad8b640 1 -- 172.21.15.98:0/1360316151 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa441ec9a0 con 0x7ffa440b7520 2024-05-09T10:50:24.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7ffa417fa640 1 -- 172.21.15.98:0/1360316151 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3c008790 con 0x7ffa440b7520 2024-05-09T10:50:24.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7f29b9c93640 1 -- 172.21.15.98:0/1655145821 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b41f8bd0 con 0x7f29b40cf050 2024-05-09T10:50:24.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7fb2da84a640 1 -- 172.21.15.98:0/2783980682 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2cc0e2870 con 0x7fb2cc04de40 2024-05-09T10:50:24.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7fb2da84a640 1 -- 172.21.15.98:0/2783980682 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2cc0e2e30 con 0x7fb2cc04de40 2024-05-09T10:50:24.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7f08877fe640 1 -- 172.21.15.98:0/294168447 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f08a4069cb0 con 0x7f08a40cfbc0 2024-05-09T10:50:24.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7f29b9c93640 1 -- 172.21.15.98:0/1655145821 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b41fb980 con 0x7f29b40cf050 2024-05-09T10:50:24.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7f92bcff9640 1 -- 172.21.15.98:0/3896757311 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4037040 con 0x7f92b8039310 2024-05-09T10:50:24.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7f62277fe640 1 -- 172.21.15.98:0/3183400844 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62300125f0 con 0x7f624006e8a0 2024-05-09T10:50:24.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7f62277fe640 1 -- 172.21.15.98:0/3183400844 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6230019070 con 0x7f624006e8a0 2024-05-09T10:50:24.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7f62277fe640 1 -- 172.21.15.98:0/3183400844 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62300129d0 con 0x7f624006e8a0 2024-05-09T10:50:24.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.639+0000 7f62453b7640 1 -- 172.21.15.98:0/3183400844 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62401ec030 con 0x7f624006e8a0 2024-05-09T10:50:24.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7f6dbe59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dc00b46e0 0x7f6dc00adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7f6dbe59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dc00b46e0 0x7f6dc00adc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42024/0 (socket says 172.21.15.98:42024) 2024-05-09T10:50:24.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7f6dc4ce1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6dc0165930 0x7f6dc014d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7f6dc4ce1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc0166510 con 0x7f6dc00b46e0 2024-05-09T10:50:24.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7f6dc4ce1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc0166230 con 0x7f6dc0165930 2024-05-09T10:50:24.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7f6dc4ce1640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc01667f0 con 0x7f6dc0165550 2024-05-09T10:50:24.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.639+0000 7f6dbdd9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6dc0165550 0x7f6dc00c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.639+0000 7f6dbdd9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6dc0165550 0x7f6dc00c4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:53208/0 (socket says 172.21.15.98:53208) 2024-05-09T10:50:24.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.639+0000 7f6dbdd9b640 1 -- 172.21.15.98:0/1170321421 learned_addr learned my addr 172.21.15.98:0/1170321421 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.638+0000 7fce98c6b640 1 -- 172.21.15.98:0/2779381409 --> [v2:172.21.15.98:6808/2633494785,v1:172.21.15.98:6809/2633494785] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fce78006bb0 con 0x7fce78001650 2024-05-09T10:50:24.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.639+0000 7fce9259c640 1 --2- 172.21.15.98:0/2779381409 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fce7406b5a0 0x7fce7406da60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fce88035040 tx=0x7fce8802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.639+0000 7f62453b7640 1 -- 172.21.15.98:0/3183400844 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62401ec5f0 con 0x7f624006e8a0 2024-05-09T10:50:24.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.639+0000 7f6dbdd9b640 1 -- 172.21.15.98:0/1170321421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6dc0165930 msgr2=0x7f6dc014d880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T10:50:24.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.639+0000 7f6dbdd9b640 1 --2- 172.21.15.98:0/1170321421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6dc0165930 0x7f6dc014d880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.639+0000 7f6dbdd9b640 1 -- 172.21.15.98:0/1170321421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dc00b46e0 msgr2=0x7f6dc00adc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.639+0000 7f6dbdd9b640 1 --2- 172.21.15.98:0/1170321421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dc00b46e0 0x7f6dc00adc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.639+0000 7f6dbdd9b640 1 -- 172.21.15.98:0/1170321421 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc00c50a0 con 0x7f6dc0165550 2024-05-09T10:50:24.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.639+0000 7ffa417fa640 1 -- 172.21.15.98:0/1360316151 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c036040 con 0x7ffa440b7520 2024-05-09T10:50:24.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.639+0000 7f29b9c93640 1 -- 172.21.15.98:0/1655145821 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f29b41f90f0 con 0x7f29b40cf050 2024-05-09T10:50:24.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.639+0000 7f6dbe59c640 1 --2- 172.21.15.98:0/1170321421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dc00b46e0 0x7f6dc00adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-09T10:50:24.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.639+0000 7fb2da84a640 1 -- 172.21.15.98:0/2783980682 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb2cc0057e0 con 0x7fb2cc04de40 2024-05-09T10:50:24.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.639+0000 7f92aa7fc640 1 -- 172.21.15.98:0/3896757311 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9298001030 con 0x7f92b8039310 2024-05-09T10:50:24.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.639+0000 7f92bcff9640 1 -- 172.21.15.98:0/3896757311 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f92b404b070 con 0x7f92b8039310 2024-05-09T10:50:24.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.640+0000 7f62453b7640 1 -- 172.21.15.98:0/3183400844 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f624006de80 con 0x7f624006e8a0 2024-05-09T10:50:24.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.640+0000 7f92bcff9640 1 --2- 172.21.15.98:0/3896757311 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f92a406b5a0 0x7f92a406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.640+0000 7fce91d9b640 1 --2- 172.21.15.98:0/2779381409 >> [v2:172.21.15.98:6808/2633494785,v1:172.21.15.98:6809/2633494785] conn(0x7fce78001650 0x7fce78003b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.640+0000 7f92bcff9640 1 -- 172.21.15.98:0/3896757311 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92b40b3250 con 0x7f92b8039310 2024-05-09T10:50:24.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.640+0000 7ffa4ad8b640 1 -- 172.21.15.98:0/1360316151 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffa24001030 con 0x7ffa440b7520 2024-05-09T10:50:24.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.640+0000 7f92bf7fe640 1 --2- 172.21.15.98:0/3896757311 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f92a406b5a0 0x7f92a406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.640+0000 7ffa417fa640 1 -- 172.21.15.98:0/1360316151 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa3c044020 con 0x7ffa440b7520 2024-05-09T10:50:24.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.640+0000 7f92bcff9640 1 --2- 172.21.15.98:0/3896757311 >> [v2:172.21.15.105:6808/3698267721,v1:172.21.15.105:6809/3698267721] conn(0x7f92a40737c0 0x7f92a4075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.641+0000 7f0899ffb640 1 -- 172.21.15.98:0/294168447 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f089c01d070 con 0x7f08a40cfbc0 2024-05-09T10:50:24.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.641+0000 7f92bcff9640 1 -- 172.21.15.98:0/3896757311 --> [v2:172.21.15.105:6808/3698267721,v1:172.21.15.105:6809/3698267721] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f92b400af30 con 0x7f92a40737c0 2024-05-09T10:50:24.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.641+0000 7f92bcff9640 1 -- 172.21.15.98:0/3896757311 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92b40ba050 con 0x7f92b8039310 2024-05-09T10:50:24.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.641+0000 7f6dbdd9b640 1 --2- 172.21.15.98:0/1170321421 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6dc0165550 0x7f6dc00c4b60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6db4009960 tx=0x7f6db40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.641+0000 7f299bfff640 1 -- 172.21.15.98:0/1655145821 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f29a8017070 con 0x7f29b40cf050 2024-05-09T10:50:24.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.641+0000 7f92c4bd7640 1 --2- 172.21.15.98:0/3896757311 >> [v2:172.21.15.105:6808/3698267721,v1:172.21.15.105:6809/3698267721] conn(0x7f92a40737c0 0x7f92a4075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.641+0000 7ffa417fa640 1 --2- 172.21.15.98:0/1360316151 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ffa3006b5d0 0x7ffa3006da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.641+0000 7f0899ffb640 1 --2- 172.21.15.98:0/294168447 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f088806b6d0 0x7f088806db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.641+0000 7f92bf7fe640 1 --2- 172.21.15.98:0/3896757311 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f92a406b5a0 0x7f92a406da60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f92b80e3980 tx=0x7f92ac0090e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.641+0000 7f0899ffb640 1 -- 172.21.15.98:0/294168447 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f089c08b1b0 con 0x7f08a40cfbc0 2024-05-09T10:50:24.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.641+0000 7ffa417fa640 1 -- 172.21.15.98:0/1360316151 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa3c0ab0c0 con 0x7ffa440b7520 2024-05-09T10:50:24.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7f0899ffb640 1 --2- 172.21.15.98:0/294168447 >> [v2:172.21.15.105:6800/2110551679,v1:172.21.15.105:6801/2110551679] conn(0x7f0888073870 0x7f0888075c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7f0899ffb640 1 -- 172.21.15.98:0/294168447 --> [v2:172.21.15.105:6800/2110551679,v1:172.21.15.105:6801/2110551679] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0888076340 con 0x7f0888073870 2024-05-09T10:50:24.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.641+0000 7fb2ca7fc640 1 -- 172.21.15.98:0/2783980682 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2c40067d0 con 0x7fb2cc04de40 2024-05-09T10:50:24.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7fb2ca7fc640 1 --2- 172.21.15.98:0/2783980682 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb2b806b8d0 0x7fb2b806dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7f0899ffb640 1 -- 172.21.15.98:0/294168447 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f089c08b5a0 con 0x7f08a40cfbc0 2024-05-09T10:50:24.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.641+0000 7f6daeffd640 1 -- 172.21.15.98:0/1170321421 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db4037050 con 0x7f6dc0165550 2024-05-09T10:50:24.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.641+0000 7f6daeffd640 1 -- 172.21.15.98:0/1170321421 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db4043070 con 0x7f6dc0165550 2024-05-09T10:50:24.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.641+0000 7f6daeffd640 1 -- 172.21.15.98:0/1170321421 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db4030d80 con 0x7f6dc0165550 2024-05-09T10:50:24.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7f6dc4ce1640 1 -- 172.21.15.98:0/1170321421 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc014e300 con 0x7f6dc0165550 2024-05-09T10:50:24.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.641+0000 7f299bfff640 1 --2- 172.21.15.98:0/1655145821 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f29a006b5a0 0x7f29a006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7f6dc4ce1640 1 -- 172.21.15.98:0/1170321421 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc014e8c0 con 0x7f6dc0165550 2024-05-09T10:50:24.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7f299bfff640 1 -- 172.21.15.98:0/1655145821 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29a808a770 con 0x7f29b40cf050 2024-05-09T10:50:24.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7f299bfff640 1 --2- 172.21.15.98:0/1655145821 >> [v2:172.21.15.98:6816/1902516290,v1:172.21.15.98:6817/1902516290] conn(0x7f29a00737c0 0x7f29a0075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7f299bfff640 1 -- 172.21.15.98:0/1655145821 --> [v2:172.21.15.98:6816/1902516290,v1:172.21.15.98:6817/1902516290] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f29a8021e50 con 0x7f29a00737c0 2024-05-09T10:50:24.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7f299bfff640 1 -- 172.21.15.98:0/1655145821 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29a808ab60 con 0x7f29b40cf050 2024-05-09T10:50:24.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7fce82ffd640 1 -- 172.21.15.98:0/2779381409 <== osd.3 v2:172.21.15.98:6808/2633494785 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fce78006bb0 con 0x7fce78001650 2024-05-09T10:50:24.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7f08a8f8f640 1 --2- 172.21.15.98:0/294168447 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f088806b6d0 0x7f088806db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7f62277fe640 1 -- 172.21.15.98:0/3183400844 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6230012b70 con 0x7f624006e8a0 2024-05-09T10:50:24.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7f29b3fff640 1 --2- 172.21.15.98:0/1655145821 >> [v2:172.21.15.98:6816/1902516290,v1:172.21.15.98:6817/1902516290] conn(0x7f29a00737c0 0x7f29a0075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7ffa417fa640 1 --2- 172.21.15.98:0/1360316151 >> [v2:172.21.15.98:6800/3255566819,v1:172.21.15.98:6801/3255566819] conn(0x7ffa300737f0 0x7ffa30075c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7ffa417fa640 1 -- 172.21.15.98:0/1360316151 --> [v2:172.21.15.98:6800/3255566819,v1:172.21.15.98:6801/3255566819] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa3c031600 con 0x7ffa300737f0 2024-05-09T10:50:24.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7ffa417fa640 1 -- 172.21.15.98:0/1360316151 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa3c0b2050 con 0x7ffa440b7520 2024-05-09T10:50:24.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7ffa43fff640 1 --2- 172.21.15.98:0/1360316151 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ffa3006b5d0 0x7ffa3006da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7ffa49328640 1 --2- 172.21.15.98:0/1360316151 >> [v2:172.21.15.98:6800/3255566819,v1:172.21.15.98:6801/3255566819] conn(0x7ffa300737f0 0x7ffa30075c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7f29b37fe640 1 --2- 172.21.15.98:0/1655145821 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f29a006b5a0 0x7f29a006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7f08a9f91640 1 --2- 172.21.15.98:0/294168447 >> [v2:172.21.15.105:6800/2110551679,v1:172.21.15.105:6801/2110551679] conn(0x7f0888073870 0x7f0888075c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.643+0000 7f29b3fff640 1 --2- 172.21.15.98:0/1655145821 >> [v2:172.21.15.98:6816/1902516290,v1:172.21.15.98:6817/1902516290] conn(0x7f29a00737c0 0x7f29a0075be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.643+0000 7ffa43fff640 1 --2- 172.21.15.98:0/1360316151 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ffa3006b5d0 0x7ffa3006da90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffa440704a0 tx=0x7ffa34008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7fb2d9848640 1 --2- 172.21.15.98:0/2783980682 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb2b806b8d0 0x7fb2b806dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7fb2ca7fc640 1 -- 172.21.15.98:0/2783980682 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2c40050d0 con 0x7fb2cc04de40 2024-05-09T10:50:24.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7fb2ca7fc640 1 --2- 172.21.15.98:0/2783980682 >> [v2:172.21.15.98:6824/4137049320,v1:172.21.15.98:6825/4137049320] conn(0x7fb2b80746d0 0x7fb2b8076b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.642+0000 7fb2d9047640 1 --2- 172.21.15.98:0/2783980682 >> [v2:172.21.15.98:6824/4137049320,v1:172.21.15.98:6825/4137049320] conn(0x7fb2b80746d0 0x7fb2b8076b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.643+0000 7fb2d9047640 1 --2- 172.21.15.98:0/2783980682 >> [v2:172.21.15.98:6824/4137049320,v1:172.21.15.98:6825/4137049320] conn(0x7fb2b80746d0 0x7fb2b8076b70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.643+0000 7fb2ca7fc640 1 -- 172.21.15.98:0/2783980682 --> [v2:172.21.15.98:6824/4137049320,v1:172.21.15.98:6825/4137049320] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2b8077240 con 0x7fb2b80746d0 2024-05-09T10:50:24.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.643+0000 7f08a8f8f640 1 --2- 172.21.15.98:0/294168447 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f088806b6d0 0x7f088806db90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f08a0034000 tx=0x7f08a002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.643+0000 7fb2ca7fc640 1 -- 172.21.15.98:0/2783980682 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2c40549d0 con 0x7fb2cc04de40 2024-05-09T10:50:24.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.643+0000 7f62277fe640 1 --2- 172.21.15.98:0/3183400844 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f622c06b5a0 0x7f622c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.643+0000 7f62277fe640 1 -- 172.21.15.98:0/3183400844 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f623008a5f0 con 0x7f624006e8a0 2024-05-09T10:50:24.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.643+0000 7f62277fe640 1 --2- 172.21.15.98:0/3183400844 >> [v2:172.21.15.105:6816/721380547,v1:172.21.15.105:6817/721380547] conn(0x7f622c0737c0 0x7f622c075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.643+0000 7f62277fe640 1 -- 172.21.15.98:0/3183400844 --> [v2:172.21.15.105:6816/721380547,v1:172.21.15.105:6817/721380547] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6230001d30 con 0x7f622c0737c0 2024-05-09T10:50:24.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.643+0000 7f62277fe640 1 -- 172.21.15.98:0/3183400844 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6230091050 con 0x7f624006e8a0 2024-05-09T10:50:24.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.643+0000 7f623e7fc640 1 --2- 172.21.15.98:0/3183400844 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f622c06b5a0 0x7f622c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.643+0000 7ffa49328640 1 --2- 172.21.15.98:0/1360316151 >> [v2:172.21.15.98:6800/3255566819,v1:172.21.15.98:6801/3255566819] conn(0x7ffa300737f0 0x7ffa30075c10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.644+0000 7f29b37fe640 1 --2- 172.21.15.98:0/1655145821 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f29a006b5a0 0x7f29a006da60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f29a4034000 tx=0x7f29a402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.644+0000 7f299bfff640 1 -- 172.21.15.98:0/1655145821 <== osd.1 v2:172.21.15.98:6816/1902516290 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f29a8021e50 con 0x7f29a00737c0 2024-05-09T10:50:24.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.643+0000 7fb2d9848640 1 --2- 172.21.15.98:0/2783980682 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb2b806b8d0 0x7fb2b806dd90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0037000 tx=0x7fb2d0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.644+0000 7fb2ca7fc640 1 -- 172.21.15.98:0/2783980682 <== osd.2 v2:172.21.15.98:6824/4137049320 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb2b8077240 con 0x7fb2b80746d0 2024-05-09T10:50:24.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.643+0000 7f92c4bd7640 1 --2- 172.21.15.98:0/3896757311 >> [v2:172.21.15.105:6808/3698267721,v1:172.21.15.105:6809/3698267721] conn(0x7f92a40737c0 0x7f92a4075be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.644+0000 7f92bcff9640 1 -- 172.21.15.98:0/3896757311 <== osd.4 v2:172.21.15.105:6808/3698267721 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f92b400af30 con 0x7f92a40737c0 2024-05-09T10:50:24.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.644+0000 7f08a9f91640 1 --2- 172.21.15.98:0/294168447 >> [v2:172.21.15.105:6800/2110551679,v1:172.21.15.105:6801/2110551679] conn(0x7f0888073870 0x7f0888075c90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.644+0000 7f623e7fc640 1 --2- 172.21.15.98:0/3183400844 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f622c06b5a0 0x7f622c06da60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6240068150 tx=0x7f623402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.644+0000 7f623ffff640 1 --2- 172.21.15.98:0/3183400844 >> [v2:172.21.15.105:6816/721380547,v1:172.21.15.105:6817/721380547] conn(0x7f622c0737c0 0x7f622c075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.643+0000 7f6dacff9640 1 -- 172.21.15.98:0/1170321421 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6dc0069cb0 con 0x7f6dc0165550 2024-05-09T10:50:24.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.644+0000 7f623ffff640 1 --2- 172.21.15.98:0/3183400844 >> [v2:172.21.15.105:6816/721380547,v1:172.21.15.105:6817/721380547] conn(0x7f622c0737c0 0x7f622c075be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.645+0000 7ffa417fa640 1 -- 172.21.15.98:0/1360316151 <== osd.0 v2:172.21.15.98:6800/3255566819 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ffa3c031600 con 0x7ffa300737f0 2024-05-09T10:50:24.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.645+0000 7f0899ffb640 1 -- 172.21.15.98:0/294168447 <== osd.5 v2:172.21.15.105:6800/2110551679 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0888076340 con 0x7f0888073870 2024-05-09T10:50:24.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.644+0000 7f6daeffd640 1 -- 172.21.15.98:0/1170321421 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6db4048460 con 0x7f6dc0165550 2024-05-09T10:50:24.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.645+0000 7f6daeffd640 1 --2- 172.21.15.98:0/1170321421 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6da006b5a0 0x7f6da006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.645+0000 7f6daeffd640 1 -- 172.21.15.98:0/1170321421 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6db40ab1a0 con 0x7f6dc0165550 2024-05-09T10:50:24.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.645+0000 7f62277fe640 1 -- 172.21.15.98:0/3183400844 <== osd.6 v2:172.21.15.105:6816/721380547 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6230001d30 con 0x7f622c0737c0 2024-05-09T10:50:24.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.645+0000 7f6daeffd640 1 --2- 172.21.15.98:0/1170321421 >> [v2:172.21.15.105:6824/3129650451,v1:172.21.15.105:6825/3129650451] conn(0x7f6da0073740 0x7f6da0075b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.645+0000 7f6daeffd640 1 -- 172.21.15.98:0/1170321421 --> [v2:172.21.15.105:6824/3129650451,v1:172.21.15.105:6825/3129650451] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6db4032440 con 0x7f6da0073740 2024-05-09T10:50:24.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.645+0000 7f6daeffd640 1 -- 172.21.15.98:0/1170321421 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6db40b2050 con 0x7f6dc0165550 2024-05-09T10:50:24.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.645+0000 7f6dbe59c640 1 --2- 172.21.15.98:0/1170321421 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6da006b5a0 0x7f6da006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.646+0000 7f6dbe59c640 1 --2- 172.21.15.98:0/1170321421 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6da006b5a0 0x7f6da006da60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6db0006fd0 tx=0x7f6db0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.646+0000 7f6dbed9d640 1 --2- 172.21.15.98:0/1170321421 >> [v2:172.21.15.105:6824/3129650451,v1:172.21.15.105:6825/3129650451] conn(0x7f6da0073740 0x7f6da0075b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.647+0000 7f6dbed9d640 1 --2- 172.21.15.98:0/1170321421 >> [v2:172.21.15.105:6824/3129650451,v1:172.21.15.105:6825/3129650451] conn(0x7f6da0073740 0x7f6da0075b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.648+0000 7f6daeffd640 1 -- 172.21.15.98:0/1170321421 <== osd.7 v2:172.21.15.105:6824/3129650451 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6db4032440 con 0x7f6da0073740 2024-05-09T10:50:24.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.674+0000 7fce98c6b640 1 -- 172.21.15.98:0/2779381409 --> [v2:172.21.15.98:6808/2633494785,v1:172.21.15.98:6809/2633494785] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fce78005c90 con 0x7fce78001650 2024-05-09T10:50:24.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.674+0000 7fce82ffd640 1 -- 172.21.15.98:0/2779381409 <== osd.3 v2:172.21.15.98:6808/2633494785 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fce78005c90 con 0x7fce78001650 2024-05-09T10:50:24.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.675+0000 7fce80ff9640 1 -- 172.21.15.98:0/2779381409 >> [v2:172.21.15.98:6808/2633494785,v1:172.21.15.98:6809/2633494785] conn(0x7fce78001650 msgr2=0x7fce78003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.675+0000 7fce80ff9640 1 --2- 172.21.15.98:0/2779381409 >> [v2:172.21.15.98:6808/2633494785,v1:172.21.15.98:6809/2633494785] conn(0x7fce78001650 0x7fce78003b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.675+0000 7fce80ff9640 1 -- 172.21.15.98:0/2779381409 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fce7406b5a0 msgr2=0x7fce7406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.675+0000 7fce80ff9640 1 --2- 172.21.15.98:0/2779381409 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fce7406b5a0 0x7fce7406da60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fce88035040 tx=0x7fce8802e000 comp rx=0 tx=0).stop 2024-05-09T10:50:24.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.675+0000 7fce80ff9640 1 -- 172.21.15.98:0/2779381409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9415a190 msgr2=0x7fce9415bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.675+0000 7fce80ff9640 1 --2- 172.21.15.98:0/2779381409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce9415a190 0x7fce9415bef0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fce940b1d90 tx=0x7fce8c00ef90 comp rx=0 tx=0).stop 2024-05-09T10:50:24.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.675+0000 7fce92d9d640 1 -- 172.21.15.98:0/2779381409 reap_dead start 2024-05-09T10:50:24.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.676+0000 7fce80ff9640 1 -- 172.21.15.98:0/2779381409 shutdown_connections 2024-05-09T10:50:24.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.676+0000 7f29b9c93640 1 -- 172.21.15.98:0/1655145821 --> [v2:172.21.15.98:6816/1902516290,v1:172.21.15.98:6817/1902516290] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f29b4070fe0 con 0x7f29a00737c0 2024-05-09T10:50:24.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.676+0000 7fce80ff9640 1 -- 172.21.15.98:0/2779381409 >> 172.21.15.98:0/2779381409 conn(0x7fce9406cae0 msgr2=0x7fce940cde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.676+0000 7fce80ff9640 1 -- 172.21.15.98:0/2779381409 shutdown_connections 2024-05-09T10:50:24.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.676+0000 7fce80ff9640 1 -- 172.21.15.98:0/2779381409 wait complete. 2024-05-09T10:50:24.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.676+0000 7f299bfff640 1 -- 172.21.15.98:0/1655145821 <== osd.1 v2:172.21.15.98:6816/1902516290 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f29b4070fe0 con 0x7f29a00737c0 2024-05-09T10:50:24.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.677+0000 7f92aa7fc640 1 -- 172.21.15.98:0/3896757311 --> [v2:172.21.15.105:6808/3698267721,v1:172.21.15.105:6809/3698267721] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9298002dd0 con 0x7f92a40737c0 2024-05-09T10:50:24.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.676+0000 7ffa4ad8b640 1 -- 172.21.15.98:0/1360316151 --> [v2:172.21.15.98:6800/3255566819,v1:172.21.15.98:6801/3255566819] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffa24002dd0 con 0x7ffa300737f0 2024-05-09T10:50:24.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.677+0000 7fb2da84a640 1 -- 172.21.15.98:0/2783980682 --> [v2:172.21.15.98:6824/4137049320,v1:172.21.15.98:6825/4137049320] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb2cc00c530 con 0x7fb2b80746d0 2024-05-09T10:50:24.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.677+0000 7f92bcff9640 1 -- 172.21.15.98:0/3896757311 <== osd.4 v2:172.21.15.105:6808/3698267721 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9298002dd0 con 0x7f92a40737c0 2024-05-09T10:50:24.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.677+0000 7f29b9c93640 1 -- 172.21.15.98:0/1655145821 >> [v2:172.21.15.98:6816/1902516290,v1:172.21.15.98:6817/1902516290] conn(0x7f29a00737c0 msgr2=0x7f29a0075be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.677+0000 7f29b9c93640 1 --2- 172.21.15.98:0/1655145821 >> [v2:172.21.15.98:6816/1902516290,v1:172.21.15.98:6817/1902516290] conn(0x7f29a00737c0 0x7f29a0075be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.677+0000 7ffa417fa640 1 -- 172.21.15.98:0/1360316151 <== osd.0 v2:172.21.15.98:6800/3255566819 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffa24002dd0 con 0x7ffa300737f0 2024-05-09T10:50:24.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.677+0000 7fb2ca7fc640 1 -- 172.21.15.98:0/2783980682 <== osd.2 v2:172.21.15.98:6824/4137049320 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb2cc00c530 con 0x7fb2b80746d0 2024-05-09T10:50:24.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7f29b9c93640 1 -- 172.21.15.98:0/1655145821 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f29a006b5a0 msgr2=0x7f29a006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7f29b9c93640 1 --2- 172.21.15.98:0/1655145821 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f29a006b5a0 0x7f29a006da60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f29a4034000 tx=0x7f29a402e000 comp rx=0 tx=0).stop 2024-05-09T10:50:24.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7f29b9c93640 1 -- 172.21.15.98:0/1655145821 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f29b40cf050 msgr2=0x7f29b41ed690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7f29b9c93640 1 --2- 172.21.15.98:0/1655145821 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f29b40cf050 0x7f29b41ed690 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f29a800d570 tx=0x7f29a800da30 comp rx=0 tx=0).stop 2024-05-09T10:50:24.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7f29b3fff640 1 -- 172.21.15.98:0/1655145821 reap_dead start 2024-05-09T10:50:24.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7f29b9c93640 1 -- 172.21.15.98:0/1655145821 shutdown_connections 2024-05-09T10:50:24.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7f29b9c93640 1 -- 172.21.15.98:0/1655145821 >> 172.21.15.98:0/1655145821 conn(0x7f29b406abe0 msgr2=0x7f29b40cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7ffa4ad8b640 1 -- 172.21.15.98:0/1360316151 >> [v2:172.21.15.98:6800/3255566819,v1:172.21.15.98:6801/3255566819] conn(0x7ffa300737f0 msgr2=0x7ffa30075c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7ffa4ad8b640 1 --2- 172.21.15.98:0/1360316151 >> [v2:172.21.15.98:6800/3255566819,v1:172.21.15.98:6801/3255566819] conn(0x7ffa300737f0 0x7ffa30075c10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7ffa4ad8b640 1 -- 172.21.15.98:0/1360316151 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ffa3006b5d0 msgr2=0x7ffa3006da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7ffa4ad8b640 1 --2- 172.21.15.98:0/1360316151 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ffa3006b5d0 0x7ffa3006da90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffa440704a0 tx=0x7ffa34008040 comp rx=0 tx=0).stop 2024-05-09T10:50:24.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7ffa4ad8b640 1 -- 172.21.15.98:0/1360316151 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa440b7520 msgr2=0x7ffa440b3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7ffa4ad8b640 1 --2- 172.21.15.98:0/1360316151 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa440b7520 0x7ffa440b3450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009f70 tx=0x7ffa3c004a40 comp rx=0 tx=0).stop 2024-05-09T10:50:24.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7f29b9c93640 1 -- 172.21.15.98:0/1655145821 shutdown_connections 2024-05-09T10:50:24.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7f29b9c93640 1 -- 172.21.15.98:0/1655145821 wait complete. 2024-05-09T10:50:24.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7fb2da84a640 1 -- 172.21.15.98:0/2783980682 >> [v2:172.21.15.98:6824/4137049320,v1:172.21.15.98:6825/4137049320] conn(0x7fb2b80746d0 msgr2=0x7fb2b8076b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7fb2da84a640 1 --2- 172.21.15.98:0/2783980682 >> [v2:172.21.15.98:6824/4137049320,v1:172.21.15.98:6825/4137049320] conn(0x7fb2b80746d0 0x7fb2b8076b70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7fb2da84a640 1 -- 172.21.15.98:0/2783980682 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb2b806b8d0 msgr2=0x7fb2b806dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7fb2da84a640 1 --2- 172.21.15.98:0/2783980682 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb2b806b8d0 0x7fb2b806dd90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0037000 tx=0x7fb2d0008030 comp rx=0 tx=0).stop 2024-05-09T10:50:24.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7f92aa7fc640 1 -- 172.21.15.98:0/3896757311 >> [v2:172.21.15.105:6808/3698267721,v1:172.21.15.105:6809/3698267721] conn(0x7f92a40737c0 msgr2=0x7f92a4075be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7f92aa7fc640 1 --2- 172.21.15.98:0/3896757311 >> [v2:172.21.15.105:6808/3698267721,v1:172.21.15.105:6809/3698267721] conn(0x7f92a40737c0 0x7f92a4075be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7f08877fe640 1 -- 172.21.15.98:0/294168447 --> [v2:172.21.15.105:6800/2110551679,v1:172.21.15.105:6801/2110551679] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f08a40cbde0 con 0x7f0888073870 2024-05-09T10:50:24.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7fb2da84a640 1 -- 172.21.15.98:0/2783980682 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2cc04de40 msgr2=0x7fb2cc17d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7fb2da84a640 1 --2- 172.21.15.98:0/2783980682 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2cc04de40 0x7fb2cc17d3c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400c3f0 tx=0x7fb2c400c8b0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7f6dacff9640 1 -- 172.21.15.98:0/1170321421 --> [v2:172.21.15.105:6824/3129650451,v1:172.21.15.105:6825/3129650451] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6dc006de80 con 0x7f6da0073740 2024-05-09T10:50:24.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.678+0000 7fb2da049640 1 -- 172.21.15.98:0/2783980682 reap_dead start 2024-05-09T10:50:24.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.679+0000 7ffa49328640 1 -- 172.21.15.98:0/1360316151 reap_dead start 2024-05-09T10:50:24.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.679+0000 7ffa4ad8b640 1 -- 172.21.15.98:0/1360316151 shutdown_connections 2024-05-09T10:50:24.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.679+0000 7ffa4ad8b640 1 -- 172.21.15.98:0/1360316151 >> 172.21.15.98:0/1360316151 conn(0x7ffa44066f20 msgr2=0x7ffa440b9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.679+0000 7f0899ffb640 1 -- 172.21.15.98:0/294168447 <== osd.5 v2:172.21.15.105:6800/2110551679 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f08a40cbde0 con 0x7f0888073870 2024-05-09T10:50:24.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.679+0000 7fb2da84a640 1 -- 172.21.15.98:0/2783980682 shutdown_connections 2024-05-09T10:50:24.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.679+0000 7fb2da84a640 1 -- 172.21.15.98:0/2783980682 >> 172.21.15.98:0/2783980682 conn(0x7fb2cc020040 msgr2=0x7fb2cc00c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.679+0000 7fb2da84a640 1 -- 172.21.15.98:0/2783980682 shutdown_connections 2024-05-09T10:50:24.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.679+0000 7ffa4ad8b640 1 -- 172.21.15.98:0/1360316151 shutdown_connections 2024-05-09T10:50:24.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.679+0000 7f92aa7fc640 1 -- 172.21.15.98:0/3896757311 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f92a406b5a0 msgr2=0x7f92a406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.679+0000 7ffa4ad8b640 1 -- 172.21.15.98:0/1360316151 wait complete. 2024-05-09T10:50:24.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.679+0000 7f92aa7fc640 1 --2- 172.21.15.98:0/3896757311 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f92a406b5a0 0x7f92a406da60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f92b80e3980 tx=0x7f92ac0090e0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.679+0000 7f92aa7fc640 1 -- 172.21.15.98:0/3896757311 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92b8039310 msgr2=0x7f92b80e2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.679+0000 7f92aa7fc640 1 --2- 172.21.15.98:0/3896757311 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92b8039310 0x7f92b80e2000 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f92b4000c00 tx=0x7f92b40046c0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.679+0000 7f6daeffd640 1 -- 172.21.15.98:0/1170321421 <== osd.7 v2:172.21.15.105:6824/3129650451 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6dc006de80 con 0x7f6da0073740 2024-05-09T10:50:24.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.679+0000 7f92c4bd7640 1 -- 172.21.15.98:0/3896757311 reap_dead start 2024-05-09T10:50:24.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.679+0000 7fb2da84a640 1 -- 172.21.15.98:0/2783980682 wait complete. 2024-05-09T10:50:24.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.679+0000 7f92aa7fc640 1 -- 172.21.15.98:0/3896757311 shutdown_connections 2024-05-09T10:50:24.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.679+0000 7f92aa7fc640 1 -- 172.21.15.98:0/3896757311 >> 172.21.15.98:0/3896757311 conn(0x7f92b8055bc0 msgr2=0x7f92b80df280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.679+0000 7f08877fe640 1 -- 172.21.15.98:0/294168447 >> [v2:172.21.15.105:6800/2110551679,v1:172.21.15.105:6801/2110551679] conn(0x7f0888073870 msgr2=0x7f0888075c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.679+0000 7f08877fe640 1 --2- 172.21.15.98:0/294168447 >> [v2:172.21.15.105:6800/2110551679,v1:172.21.15.105:6801/2110551679] conn(0x7f0888073870 0x7f0888075c90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.679+0000 7f62453b7640 1 -- 172.21.15.98:0/3183400844 --> [v2:172.21.15.105:6816/721380547,v1:172.21.15.105:6817/721380547] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6240069cb0 con 0x7f622c0737c0 2024-05-09T10:50:24.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.680+0000 7f92aa7fc640 1 -- 172.21.15.98:0/3896757311 shutdown_connections 2024-05-09T10:50:24.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.680+0000 7f92aa7fc640 1 -- 172.21.15.98:0/3896757311 wait complete. 2024-05-09T10:50:24.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.680+0000 7f6dacff9640 1 -- 172.21.15.98:0/1170321421 >> [v2:172.21.15.105:6824/3129650451,v1:172.21.15.105:6825/3129650451] conn(0x7f6da0073740 msgr2=0x7f6da0075b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.680+0000 7f6dacff9640 1 --2- 172.21.15.98:0/1170321421 >> [v2:172.21.15.105:6824/3129650451,v1:172.21.15.105:6825/3129650451] conn(0x7f6da0073740 0x7f6da0075b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.680+0000 7f08877fe640 1 -- 172.21.15.98:0/294168447 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f088806b6d0 msgr2=0x7f088806db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.680+0000 7f08877fe640 1 --2- 172.21.15.98:0/294168447 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f088806b6d0 0x7f088806db90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f08a0034000 tx=0x7f08a002e000 comp rx=0 tx=0).stop 2024-05-09T10:50:24.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.680+0000 7f08877fe640 1 -- 172.21.15.98:0/294168447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08a40cfbc0 msgr2=0x7f08a414f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.680+0000 7f08877fe640 1 --2- 172.21.15.98:0/294168447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08a40cfbc0 0x7f08a414f4f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f089c00cf60 tx=0x7f089c00cf90 comp rx=0 tx=0).stop 2024-05-09T10:50:24.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.680+0000 7f08a9f91640 1 -- 172.21.15.98:0/294168447 reap_dead start 2024-05-09T10:50:24.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.680+0000 7f6dacff9640 1 -- 172.21.15.98:0/1170321421 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6da006b5a0 msgr2=0x7f6da006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.680+0000 7f6dacff9640 1 --2- 172.21.15.98:0/1170321421 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6da006b5a0 0x7f6da006da60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6db0006fd0 tx=0x7f6db0008040 comp rx=0 tx=0).stop 2024-05-09T10:50:24.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.680+0000 7f6dacff9640 1 -- 172.21.15.98:0/1170321421 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6dc0165550 msgr2=0x7f6dc00c4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.680+0000 7f6dacff9640 1 --2- 172.21.15.98:0/1170321421 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6dc0165550 0x7f6dc00c4b60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6db4009960 tx=0x7f6db40046c0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.680+0000 7f6dbed9d640 1 -- 172.21.15.98:0/1170321421 reap_dead start 2024-05-09T10:50:24.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.680+0000 7f62277fe640 1 -- 172.21.15.98:0/3183400844 <== osd.6 v2:172.21.15.105:6816/721380547 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6240069cb0 con 0x7f622c0737c0 2024-05-09T10:50:24.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.680+0000 7f08877fe640 1 -- 172.21.15.98:0/294168447 shutdown_connections 2024-05-09T10:50:24.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.680+0000 7f08877fe640 1 -- 172.21.15.98:0/294168447 >> 172.21.15.98:0/294168447 conn(0x7f08a406abe0 msgr2=0x7f08a40b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.680+0000 7f6dacff9640 1 -- 172.21.15.98:0/1170321421 shutdown_connections 2024-05-09T10:50:24.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.680+0000 7f6dacff9640 1 -- 172.21.15.98:0/1170321421 >> 172.21.15.98:0/1170321421 conn(0x7f6dc006adc0 msgr2=0x7f6dc0071f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.680+0000 7f6dacff9640 1 -- 172.21.15.98:0/1170321421 shutdown_connections 2024-05-09T10:50:24.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.680+0000 7f6dacff9640 1 -- 172.21.15.98:0/1170321421 wait complete. 2024-05-09T10:50:24.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.680+0000 7f62453b7640 1 -- 172.21.15.98:0/3183400844 >> [v2:172.21.15.105:6816/721380547,v1:172.21.15.105:6817/721380547] conn(0x7f622c0737c0 msgr2=0x7f622c075be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.680+0000 7f62453b7640 1 --2- 172.21.15.98:0/3183400844 >> [v2:172.21.15.105:6816/721380547,v1:172.21.15.105:6817/721380547] conn(0x7f622c0737c0 0x7f622c075be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.681+0000 7f62453b7640 1 -- 172.21.15.98:0/3183400844 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f622c06b5a0 msgr2=0x7f622c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.681+0000 7f62453b7640 1 --2- 172.21.15.98:0/3183400844 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f622c06b5a0 0x7f622c06da60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6240068150 tx=0x7f623402e000 comp rx=0 tx=0).stop 2024-05-09T10:50:24.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.681+0000 7f08877fe640 1 -- 172.21.15.98:0/294168447 shutdown_connections 2024-05-09T10:50:24.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.681+0000 7f08877fe640 1 -- 172.21.15.98:0/294168447 wait complete. 2024-05-09T10:50:24.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.681+0000 7f62453b7640 1 -- 172.21.15.98:0/3183400844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624006e8a0 msgr2=0x7f62401f8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.682+0000 7f62453b7640 1 --2- 172.21.15.98:0/3183400844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624006e8a0 0x7f62401f8180 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f623000d5f0 tx=0x7f623000dab0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.682+0000 7f623ffff640 1 -- 172.21.15.98:0/3183400844 reap_dead start 2024-05-09T10:50:24.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.682+0000 7f62453b7640 1 -- 172.21.15.98:0/3183400844 shutdown_connections 2024-05-09T10:50:24.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.682+0000 7f62453b7640 1 -- 172.21.15.98:0/3183400844 >> 172.21.15.98:0/3183400844 conn(0x7f62400b2530 msgr2=0x7f62400ccbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.682+0000 7f62453b7640 1 -- 172.21.15.98:0/3183400844 shutdown_connections 2024-05-09T10:50:24.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.682+0000 7f62453b7640 1 -- 172.21.15.98:0/3183400844 wait complete. 2024-05-09T10:50:24.736 INFO:teuthology.orchestra.run.smithi098.stdout:55834574853 2024-05-09T10:50:24.737 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-09T10:50:24.739 INFO:teuthology.orchestra.run.smithi098.stdout:55834574856 2024-05-09T10:50:24.739 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-09T10:50:24.739 INFO:teuthology.orchestra.run.smithi098.stdout:55834574856 2024-05-09T10:50:24.740 INFO:teuthology.orchestra.run.smithi098.stdout:55834574855 2024-05-09T10:50:24.741 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-09T10:50:24.741 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-09T10:50:24.742 INFO:teuthology.orchestra.run.smithi098.stdout:55834574856 2024-05-09T10:50:24.742 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-09T10:50:24.743 INFO:teuthology.orchestra.run.smithi098.stdout:55834574855 2024-05-09T10:50:24.743 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-09T10:50:24.744 INFO:teuthology.orchestra.run.smithi098.stdout:55834574855 2024-05-09T10:50:24.744 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-09T10:50:24.745 INFO:teuthology.orchestra.run.smithi098.stdout:55834574854 2024-05-09T10:50:24.745 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-09T10:50:24.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.954+0000 7fc9d4d89640 1 Processor -- start 2024-05-09T10:50:24.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.955+0000 7fc9d4d89640 1 -- start start 2024-05-09T10:50:24.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.955+0000 7f7011dbd640 1 Processor -- start 2024-05-09T10:50:24.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.955+0000 7f7011dbd640 1 -- start start 2024-05-09T10:50:24.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.955+0000 7f6069963640 1 Processor -- start 2024-05-09T10:50:24.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.956+0000 7fc9d4d89640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc9d0157970 0x7fc9d0070f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.956+0000 7fe35da1f640 1 Processor -- start 2024-05-09T10:50:24.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.956+0000 7fe35da1f640 1 -- start start 2024-05-09T10:50:24.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.956+0000 7f7011dbd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f700c0b4b00 0x7f700c0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.956+0000 7f6069963640 1 -- start start 2024-05-09T10:50:24.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.956+0000 7f7011dbd640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700c0b3830 con 0x7f700c159d20 2024-05-09T10:50:24.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.956+0000 7f7011dbd640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700c0b3b10 con 0x7f700c159940 2024-05-09T10:50:24.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.956+0000 7f7011dbd640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0af650 con 0x7f700c0b4b00 2024-05-09T10:50:24.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.957+0000 7f55fa8bd640 1 Processor -- start 2024-05-09T10:50:24.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.957+0000 7f70115bc640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f700c159d20 0x7f700c0afae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55976/0 (socket says 172.21.15.98:55976) 2024-05-09T10:50:24.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.957+0000 7f70115bc640 1 -- 172.21.15.98:0/642196007 learned_addr learned my addr 172.21.15.98:0/642196007 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:24.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.957+0000 7f700bfff640 1 --2- 172.21.15.98:0/642196007 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f700c0b4b00 0x7f700c0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.957+0000 7f6069963640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f605c00aad0 0x7f605c04fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.957+0000 7f6069963640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f605c053310 con 0x7f605c0e3400 2024-05-09T10:50:24.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.957+0000 7f700b7fe640 1 -- 172.21.15.98:0/642196007 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889116876 0 0) 0x7f700c0b3830 con 0x7f700c159d20 2024-05-09T10:50:24.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.957+0000 7f700b7fe640 1 -- 172.21.15.98:0/642196007 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ff4003520 con 0x7f700c159d20 2024-05-09T10:50:24.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.957+0000 7fe35da1f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe3500eacf0 0x7fe35004fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.957+0000 7fe35da1f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe350006e90 0x7fe350007320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.957+0000 7fe35da1f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe350053710 con 0x7fe35004ffc0 2024-05-09T10:50:24.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.957+0000 7fe35da1f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe350054320 con 0x7fe3500eacf0 2024-05-09T10:50:24.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.957+0000 7fe35da1f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3500539f0 con 0x7fe350006e90 2024-05-09T10:50:24.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.957+0000 7fe3577fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe350006e90 0x7fe350007320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.957+0000 7fe3577fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe350006e90 0x7fe350007320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:53218/0 (socket says 172.21.15.98:53218) 2024-05-09T10:50:24.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.957+0000 7fe3577fe640 1 -- 172.21.15.98:0/1884959890 learned_addr learned my addr 172.21.15.98:0/1884959890 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.957+0000 7fe356ffd640 1 --2- 172.21.15.98:0/1884959890 >> v2:172.21.15.105:3300/0 conn(0x7fe3500eacf0 0x7fe35004fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7fc9d4d89640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc9d0070750 0x7fc9d00cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.957+0000 7f60637fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f605c0e3400 0x7f605c0df130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55980/0 (socket says 172.21.15.98:55980) 2024-05-09T10:50:24.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.957+0000 7f60637fe640 1 -- 172.21.15.98:0/423308877 learned_addr learned my addr 172.21.15.98:0/423308877 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:24.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.957+0000 7f6062ffd640 1 --2- 172.21.15.98:0/423308877 >> v2:172.21.15.98:3300/0 conn(0x7f605c00aad0 0x7f605c04fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7fc9d4d89640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9d00718e0 con 0x7fc9d0071460 2024-05-09T10:50:24.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7fc9d4d89640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d015ac40 con 0x7fc9d0070750 2024-05-09T10:50:24.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7fc9d4d89640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d015a390 con 0x7fc9d0157970 2024-05-09T10:50:24.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7fc9ced9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc9d0070750 0x7fc9d00cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7fc9cdd9b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc9d0071460 0x7fc9d006c290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55988/0 (socket says 172.21.15.98:55988) 2024-05-09T10:50:24.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7fc9ce59c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc9d0157970 0x7fc9d0070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7fc9ced9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc9d0070750 0x7fc9d00cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42036/0 (socket says 172.21.15.98:42036) 2024-05-09T10:50:24.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7fc9ce59c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc9d0157970 0x7fc9d0070f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.98:35206/0 (socket says 172.21.15.98:35206) 2024-05-09T10:50:24.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7fc9ce59c640 1 -- 172.21.15.98:0/3250641195 learned_addr learned my addr 172.21.15.98:0/3250641195 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7f6069963640 1 -- 172.21.15.98:0/423308877 --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f605c0535f0 con 0x7f605c054690 2024-05-09T10:50:24.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7f5a5313c640 1 Processor -- start 2024-05-09T10:50:24.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.957+0000 7f55fa8bd640 1 -- start start 2024-05-09T10:50:24.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7f55fa8bd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55f415c770 0x7f55f4163330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7f55fa8bd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f55f415cb50 0x7f55f40cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7f55fa8bd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f55f40cfbc0 0x7f55f40ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7f55fa8bd640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f40d0030 con 0x7f55f415cb50 2024-05-09T10:50:24.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7f55fa8bd640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f4163870 con 0x7f55f40cfbc0 2024-05-09T10:50:24.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7f55fa8bd640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f4166240 con 0x7f55f415c770 2024-05-09T10:50:24.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7f700bfff640 1 -- 172.21.15.98:0/642196007 >> v1:172.21.15.105:6789/0 conn(0x7f700c159940 legacy=0x7f700c0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7f700bfff640 1 -- 172.21.15.98:0/642196007 >> v1:172.21.15.98:6789/0 conn(0x7f700c159d20 legacy=0x7f700c0afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7f700bfff640 1 -- 172.21.15.98:0/642196007 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c0af000 con 0x7f700c0b4b00 2024-05-09T10:50:24.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7fc9cd59a640 1 -- 172.21.15.98:0/3250641195 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3012250660 0 0) 0x7fc9d00718e0 con 0x7fc9d0071460 2024-05-09T10:50:24.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7fe3577fe640 1 -- 172.21.15.98:0/1884959890 >> v2:172.21.15.105:3300/0 conn(0x7fe3500eacf0 msgr2=0x7fe35004fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:24.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7fe3577fe640 1 --2- 172.21.15.98:0/1884959890 >> v2:172.21.15.105:3300/0 conn(0x7fe3500eacf0 0x7fe35004fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7fe3577fe640 1 -- 172.21.15.98:0/1884959890 >> v1:172.21.15.105:6789/0 conn(0x7fe35004ffc0 legacy=0x7fe350037bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:24.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7fe3577fe640 1 -- 172.21.15.98:0/1884959890 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe350053cd0 con 0x7fe350006e90 2024-05-09T10:50:24.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7f55f8e5a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f55f40cfbc0 0x7f55f40ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7f55f8e5a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f55f40cfbc0 0x7f55f40ae4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.98:35218/0 (socket says 172.21.15.98:35218) 2024-05-09T10:50:24.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7f55f8e5a640 1 -- 172.21.15.98:0/1985177958 learned_addr learned my addr 172.21.15.98:0/1985177958 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7f55f3fff640 1 --2- 172.21.15.98:0/1985177958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55f415c770 0x7f55f4163330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.959+0000 7fc9cd59a640 1 -- 172.21.15.98:0/3250641195 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9b8003520 con 0x7fc9d0071460 2024-05-09T10:50:24.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7f6069963640 1 -- 172.21.15.98:0/423308877 --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f605c0538d0 con 0x7f605c00aad0 2024-05-09T10:50:24.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.959+0000 7f6061ffb640 1 -- 172.21.15.98:0/423308877 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1542929418 0 0) 0x7f605c053310 con 0x7f605c0e3400 2024-05-09T10:50:24.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.959+0000 7f55f37fe640 1 --2- 172.21.15.98:0/1985177958 >> v2:172.21.15.98:3300/0 conn(0x7f55f415cb50 0x7f55f40cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.959+0000 7f55f8e5a640 1 -- 172.21.15.98:0/1985177958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55f415c770 msgr2=0x7f55f4163330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:24.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.959+0000 7f55f8e5a640 1 --2- 172.21.15.98:0/1985177958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55f415c770 0x7f55f4163330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.959+0000 7f55f8e5a640 1 -- 172.21.15.98:0/1985177958 >> v2:172.21.15.98:3300/0 conn(0x7f55f415cb50 msgr2=0x7f55f40cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:24.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.959+0000 7f55f8e5a640 1 --2- 172.21.15.98:0/1985177958 >> v2:172.21.15.98:3300/0 conn(0x7f55f415cb50 0x7f55f40cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.959+0000 7f55f8e5a640 1 -- 172.21.15.98:0/1985177958 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f4166520 con 0x7f55f40cfbc0 2024-05-09T10:50:24.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.959+0000 7fe3577fe640 1 --2- 172.21.15.98:0/1884959890 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe350006e90 0x7fe350007320 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe3580646b0 tx=0x7fe3580a5440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79b387348c16b7c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.959+0000 7f700bfff640 1 --2- 172.21.15.98:0/642196007 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f700c0b4b00 0x7f700c0b31a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f70000097d0 tx=0x7f700002a120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88b8be818ee5adb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.959+0000 7fe355ffb640 1 -- 172.21.15.98:0/1884959890 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3580b1070 con 0x7fe350006e90 2024-05-09T10:50:24.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.959+0000 7fe355ffb640 1 -- 172.21.15.98:0/1884959890 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3580aa7d0 con 0x7fe350006e90 2024-05-09T10:50:24.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.959+0000 7fc9ce59c640 1 -- 172.21.15.98:0/3250641195 >> v2:172.21.15.98:3300/0 conn(0x7fc9d0070750 msgr2=0x7fc9d00cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:24.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.959+0000 7fc9ce59c640 1 --2- 172.21.15.98:0/3250641195 >> v2:172.21.15.98:3300/0 conn(0x7fc9d0070750 0x7fc9d00cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.959+0000 7fc9ce59c640 1 -- 172.21.15.98:0/3250641195 >> v1:172.21.15.98:6789/0 conn(0x7fc9d0071460 legacy=0x7fc9d006c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.959+0000 7fc9ce59c640 1 -- 172.21.15.98:0/3250641195 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d015a5f0 con 0x7fc9d0157970 2024-05-09T10:50:24.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f6061ffb640 1 -- 172.21.15.98:0/423308877 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f604c003520 con 0x7f605c0e3400 2024-05-09T10:50:24.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f6061ffb640 1 -- 172.21.15.98:0/423308877 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 66301119 0 0) 0x7f605c0535f0 con 0x7f605c054690 2024-05-09T10:50:24.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f700b7fe640 1 -- 172.21.15.98:0/642196007 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7000037070 con 0x7f700c0b4b00 2024-05-09T10:50:24.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.958+0000 7f5a5313c640 1 -- start start 2024-05-09T10:50:24.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f6061ffb640 1 -- 172.21.15.98:0/423308877 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f605c053310 con 0x7f605c054690 2024-05-09T10:50:24.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f5a5313c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5a4c0b20d0 0x7f5a4c0b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f5a5313c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5a4c0b4180 0x7f5a4c0afd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f5a5313c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a4c0cc8e0 con 0x7f5a4c161df0 2024-05-09T10:50:24.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f5a5313c640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a4c0b4c80 con 0x7f5a4c0b4180 2024-05-09T10:50:24.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f5a5313c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a4c0b49a0 con 0x7f5a4c0b20d0 2024-05-09T10:50:24.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f700b7fe640 1 -- 172.21.15.98:0/642196007 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700002ae30 con 0x7f700c0b4b00 2024-05-09T10:50:24.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f700b7fe640 1 -- 172.21.15.98:0/642196007 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7000032640 con 0x7f700c0b4b00 2024-05-09T10:50:24.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7fc9ced9d640 1 --2- 172.21.15.98:0/3250641195 >> v2:172.21.15.98:3300/0 conn(0x7fc9d0070750 0x7fc9d00cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:24.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f6062ffd640 1 -- 172.21.15.98:0/423308877 >> v1:172.21.15.105:6789/0 conn(0x7f605c054690 legacy=0x7f605c0e2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f6062ffd640 1 -- 172.21.15.98:0/423308877 >> v1:172.21.15.98:6789/0 conn(0x7f605c0e3400 legacy=0x7f605c0df130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f6062ffd640 1 -- 172.21.15.98:0/423308877 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605c053bb0 con 0x7f605c00aad0 2024-05-09T10:50:24.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f5a51939640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5a4c0b20d0 0x7f5a4c0b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f5a51939640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5a4c0b20d0 0x7f5a4c0b3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.98:35230/0 (socket says 172.21.15.98:35230) 2024-05-09T10:50:24.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f5a51939640 1 -- 172.21.15.98:0/1272946851 learned_addr learned my addr 172.21.15.98:0/1272946851 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7fe35da1f640 1 -- 172.21.15.98:0/1884959890 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe350006e90 msgr2=0x7fe350007320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7fe35da1f640 1 --2- 172.21.15.98:0/1884959890 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe350006e90 0x7fe350007320 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe3580646b0 tx=0x7fe3580a5440 comp rx=0 tx=0).stop 2024-05-09T10:50:24.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f55f8e5a640 1 --2- 172.21.15.98:0/1985177958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f40cfbc0 0x7f55f40ae4c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f55f40cdee0 tx=0x7f55ec02a690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b5a7800d38fe30f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f5a5293b640 1 --2- 172.21.15.98:0/1272946851 >> v2:172.21.15.98:3300/0 conn(0x7f5a4c0b4180 0x7f5a4c0afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7fc9ce59c640 1 --2- 172.21.15.98:0/3250641195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9d0157970 0x7fc9d0070f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc9d00b2660 tx=0x7fc9c002c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a10c8ad5f8ae1499 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f55f2ffd640 1 -- 172.21.15.98:0/1985177958 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec036070 con 0x7f55f40cfbc0 2024-05-09T10:50:24.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f55f2ffd640 1 -- 172.21.15.98:0/1985177958 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec031600 con 0x7f55f40cfbc0 2024-05-09T10:50:24.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7fe35da1f640 1 -- 172.21.15.98:0/1884959890 shutdown_connections 2024-05-09T10:50:24.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7fe35da1f640 1 --2- 172.21.15.98:0/1884959890 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe350006e90 0x7fe350007320 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7fe35da1f640 1 --2- 172.21.15.98:0/1884959890 >> v2:172.21.15.105:3300/0 conn(0x7fe3500eacf0 0x7fe35004fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7fe35da1f640 1 -- 172.21.15.98:0/1884959890 >> 172.21.15.98:0/1884959890 conn(0x7fe350009fa0 msgr2=0x7fe35000a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.961+0000 7f63a946f640 1 Processor -- start 2024-05-09T10:50:24.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.961+0000 7f55f2ffd640 1 -- 172.21.15.98:0/1985177958 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec0319d0 con 0x7f55f40cfbc0 2024-05-09T10:50:24.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.961+0000 7f7011dbd640 1 -- 172.21.15.98:0/642196007 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f700c0b4b00 msgr2=0x7f700c0b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7fc9cd59a640 1 -- 172.21.15.98:0/3250641195 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0008be0 con 0x7fc9d0157970 2024-05-09T10:50:24.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.961+0000 7fc9cd59a640 1 -- 172.21.15.98:0/3250641195 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c0008d80 con 0x7fc9d0157970 2024-05-09T10:50:24.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.961+0000 7fc9cd59a640 1 -- 172.21.15.98:0/3250641195 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c000a2f0 con 0x7fc9d0157970 2024-05-09T10:50:24.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f5a51939640 1 -- 172.21.15.98:0/1272946851 >> v2:172.21.15.98:3300/0 conn(0x7f5a4c0b4180 msgr2=0x7f5a4c0afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:24.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f5a51939640 1 --2- 172.21.15.98:0/1272946851 >> v2:172.21.15.98:3300/0 conn(0x7f5a4c0b4180 0x7f5a4c0afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f5a51939640 1 -- 172.21.15.98:0/1272946851 >> v1:172.21.15.98:6789/0 conn(0x7f5a4c161df0 legacy=0x7f5a4c0b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:24.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.960+0000 7f5a51939640 1 -- 172.21.15.98:0/1272946851 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4c0af270 con 0x7f5a4c0b20d0 2024-05-09T10:50:24.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.961+0000 7f7011dbd640 1 --2- 172.21.15.98:0/642196007 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f700c0b4b00 0x7f700c0b31a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f70000097d0 tx=0x7f700002a120 comp rx=0 tx=0).stop 2024-05-09T10:50:24.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.961+0000 7fe35da1f640 1 -- 172.21.15.98:0/1884959890 shutdown_connections 2024-05-09T10:50:24.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.961+0000 7f63a946f640 1 -- start start 2024-05-09T10:50:24.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.961+0000 7f55fa8bd640 1 -- 172.21.15.98:0/1985177958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f40cfbc0 msgr2=0x7f55f40ae4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.961+0000 7f55fa8bd640 1 --2- 172.21.15.98:0/1985177958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f40cfbc0 0x7f55f40ae4c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f55f40cdee0 tx=0x7f55ec02a690 comp rx=0 tx=0).stop 2024-05-09T10:50:24.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.961+0000 7f7011dbd640 1 -- 172.21.15.98:0/642196007 shutdown_connections 2024-05-09T10:50:24.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.961+0000 7f7011dbd640 1 --2- 172.21.15.98:0/642196007 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f700c0b4b00 0x7f700c0b31a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.961+0000 7f6062ffd640 1 --2- 172.21.15.98:0/423308877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f605c00aad0 0x7f605c04fff0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6064069a50 tx=0x7f60640a5510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852507f5efc9833a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.961+0000 7f7011dbd640 1 -- 172.21.15.98:0/642196007 >> 172.21.15.98:0/642196007 conn(0x7f700c06abe0 msgr2=0x7f700c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.961+0000 7f55fa8bd640 1 -- 172.21.15.98:0/1985177958 shutdown_connections 2024-05-09T10:50:24.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.961+0000 7f55fa8bd640 1 --2- 172.21.15.98:0/1985177958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f40cfbc0 0x7f55f40ae4c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.961+0000 7f55fa8bd640 1 --2- 172.21.15.98:0/1985177958 >> v2:172.21.15.98:3300/0 conn(0x7f55f415cb50 0x7f55f40cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.961+0000 7f55fa8bd640 1 --2- 172.21.15.98:0/1985177958 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55f415c770 0x7f55f4163330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.961+0000 7f55fa8bd640 1 -- 172.21.15.98:0/1985177958 >> 172.21.15.98:0/1985177958 conn(0x7f55f406abe0 msgr2=0x7f55f406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.962+0000 7f6061ffb640 1 -- 172.21.15.98:0/423308877 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60640b1070 con 0x7f605c00aad0 2024-05-09T10:50:24.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.962+0000 7f5a51939640 1 --2- 172.21.15.98:0/1272946851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a4c0b20d0 0x7f5a4c0b3c40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c06e1f0 tx=0x7f5a4802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2fcd38b7d5943cb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.962+0000 7f7011dbd640 1 -- 172.21.15.98:0/642196007 shutdown_connections 2024-05-09T10:50:24.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.962+0000 7f6061ffb640 1 -- 172.21.15.98:0/423308877 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60640aa7d0 con 0x7f605c00aad0 2024-05-09T10:50:24.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.962+0000 7f6061ffb640 1 -- 172.21.15.98:0/423308877 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60640aabc0 con 0x7f605c00aad0 2024-05-09T10:50:24.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.962+0000 7f5a51138640 1 -- 172.21.15.98:0/1272946851 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a48037070 con 0x7f5a4c0b20d0 2024-05-09T10:50:24.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.961+0000 7fe35da1f640 1 -- 172.21.15.98:0/1884959890 wait complete. 2024-05-09T10:50:24.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.962+0000 7f5a51138640 1 -- 172.21.15.98:0/1272946851 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a48032440 con 0x7f5a4c0b20d0 2024-05-09T10:50:24.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.962+0000 7f63a946f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f639c0df6e0 0x7f639c054cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.962+0000 7f63a946f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f639c053c20 con 0x7f639c050080 2024-05-09T10:50:24.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.962+0000 7f63a946f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f639c053f00 con 0x7f639c03ad40 2024-05-09T10:50:24.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.962+0000 7f63a946f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f639c0541e0 con 0x7f639c0df6e0 2024-05-09T10:50:24.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.962+0000 7f63a2ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f639c0df6e0 0x7f639c054cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.962+0000 7f63a37fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f639c050080 0x7f639c050490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56018/0 (socket says 172.21.15.98:56018) 2024-05-09T10:50:24.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.962+0000 7f63a37fe640 1 -- 172.21.15.98:0/2403122990 learned_addr learned my addr 172.21.15.98:0/2403122990 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:24.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.962+0000 7f7011dbd640 1 -- 172.21.15.98:0/642196007 wait complete. 2024-05-09T10:50:24.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.962+0000 7fc9d4d89640 1 -- 172.21.15.98:0/3250641195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9d0157970 msgr2=0x7fc9d0070f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.963+0000 7fc9d4d89640 1 --2- 172.21.15.98:0/3250641195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9d0157970 0x7fc9d0070f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc9d00b2660 tx=0x7fc9c002c390 comp rx=0 tx=0).stop 2024-05-09T10:50:24.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.963+0000 7f6069963640 1 -- 172.21.15.98:0/423308877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f605c00aad0 msgr2=0x7f605c04fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.963+0000 7f6069963640 1 --2- 172.21.15.98:0/423308877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f605c00aad0 0x7f605c04fff0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6064069a50 tx=0x7f60640a5510 comp rx=0 tx=0).stop 2024-05-09T10:50:24.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.963+0000 7f63a1ffb640 1 -- 172.21.15.98:0/2403122990 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129107604 0 0) 0x7f639c053c20 con 0x7f639c050080 2024-05-09T10:50:24.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.963+0000 7f63a2ffd640 1 -- 172.21.15.98:0/2403122990 >> v1:172.21.15.105:6789/0 conn(0x7f639c03ad40 legacy=0x7f639c0533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.963+0000 7f6069963640 1 -- 172.21.15.98:0/423308877 shutdown_connections 2024-05-09T10:50:24.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.963+0000 7f6069963640 1 --2- 172.21.15.98:0/423308877 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f605c00aad0 0x7f605c04fff0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.963+0000 7f6069963640 1 -- 172.21.15.98:0/423308877 >> 172.21.15.98:0/423308877 conn(0x7f605c009fe0 msgr2=0x7f605c00a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.963+0000 7f63a2ffd640 1 -- 172.21.15.98:0/2403122990 >> v1:172.21.15.98:6789/0 conn(0x7f639c050080 legacy=0x7f639c050490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.963+0000 7fc9d4d89640 1 -- 172.21.15.98:0/3250641195 shutdown_connections 2024-05-09T10:50:24.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.963+0000 7f63a2ffd640 1 -- 172.21.15.98:0/2403122990 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639c0544c0 con 0x7f639c0df6e0 2024-05-09T10:50:24.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.963+0000 7f55fa8bd640 1 -- 172.21.15.98:0/1985177958 shutdown_connections 2024-05-09T10:50:24.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.963+0000 7fc9d4d89640 1 --2- 172.21.15.98:0/3250641195 >> v2:172.21.15.98:3300/0 conn(0x7fc9d0070750 0x7fc9d00cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.963+0000 7fc9d4d89640 1 --2- 172.21.15.98:0/3250641195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9d0157970 0x7fc9d0070f20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.963+0000 7fc9d4d89640 1 -- 172.21.15.98:0/3250641195 >> 172.21.15.98:0/3250641195 conn(0x7fc9d00b15e0 msgr2=0x7fc9d00b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.963+0000 7fe35da1f640 1 Processor -- start 2024-05-09T10:50:24.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.963+0000 7fc9d4d89640 1 -- 172.21.15.98:0/3250641195 shutdown_connections 2024-05-09T10:50:24.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.963+0000 7f6069963640 1 -- 172.21.15.98:0/423308877 shutdown_connections 2024-05-09T10:50:24.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.963+0000 7f7011dbd640 1 Processor -- start 2024-05-09T10:50:24.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.963+0000 7f5a5313c640 1 -- 172.21.15.98:0/1272946851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a4c0b20d0 msgr2=0x7f5a4c0b3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.963+0000 7f5a5313c640 1 --2- 172.21.15.98:0/1272946851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a4c0b20d0 0x7f5a4c0b3c40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c06e1f0 tx=0x7f5a4802a1a0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.964+0000 7f7011dbd640 1 -- start start 2024-05-09T10:50:24.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.964+0000 7fe35da1f640 1 -- start start 2024-05-09T10:50:24.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.964+0000 7f63a2ffd640 1 --2- 172.21.15.98:0/2403122990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f639c0df6e0 0x7f639c054cc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f63940097a0 tx=0x7f639402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c5b8e8f8c6c0950 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.963+0000 7f55fa8bd640 1 -- 172.21.15.98:0/1985177958 wait complete. 2024-05-09T10:50:24.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.964+0000 7fc9d4d89640 1 -- 172.21.15.98:0/3250641195 wait complete. 2024-05-09T10:50:24.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.963+0000 7f6069963640 1 -- 172.21.15.98:0/423308877 wait complete. 2024-05-09T10:50:24.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.964+0000 7f63a1ffb640 1 -- 172.21.15.98:0/2403122990 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394037070 con 0x7f639c0df6e0 2024-05-09T10:50:24.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.964+0000 7f63a1ffb640 1 -- 172.21.15.98:0/2403122990 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6394032440 con 0x7f639c0df6e0 2024-05-09T10:50:24.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.964+0000 7f63a1ffb640 1 -- 172.21.15.98:0/2403122990 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394032810 con 0x7f639c0df6e0 2024-05-09T10:50:24.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.964+0000 7f5a5313c640 1 -- 172.21.15.98:0/1272946851 shutdown_connections 2024-05-09T10:50:24.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.964+0000 7f5a5313c640 1 --2- 172.21.15.98:0/1272946851 >> v2:172.21.15.98:3300/0 conn(0x7f5a4c0b4180 0x7f5a4c0afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.964+0000 7f5a5313c640 1 --2- 172.21.15.98:0/1272946851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a4c0b20d0 0x7f5a4c0b3c40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.964+0000 7f5a5313c640 1 -- 172.21.15.98:0/1272946851 >> 172.21.15.98:0/1272946851 conn(0x7f5a4c06c140 msgr2=0x7f5a4c06c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.965+0000 7f7011dbd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f700c0b4b00 0x7f700c15a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.965+0000 7f6069963640 1 Processor -- start 2024-05-09T10:50:24.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.965+0000 7f5a5313c640 1 -- 172.21.15.98:0/1272946851 shutdown_connections 2024-05-09T10:50:24.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.965+0000 7f55fa8bd640 1 Processor -- start 2024-05-09T10:50:24.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.965+0000 7f7010dbb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f700c0b4b00 0x7f700c15a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.965+0000 7fe35da1f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe350006e90 0x7fe3500d9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.965+0000 7fe35da1f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe35004ffc0 0x7fe35004d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.965+0000 7fe35da1f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3500eacf0 0x7fe35004daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.965+0000 7fe3567fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe35004ffc0 0x7fe35004d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.965+0000 7fe356ffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe350006e90 0x7fe3500d9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.965+0000 7fe356ffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe350006e90 0x7fe3500d9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:53246/0 (socket says 172.21.15.98:53246) 2024-05-09T10:50:24.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.965+0000 7fe3567fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe35004ffc0 0x7fe35004d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42070/0 (socket says 172.21.15.98:42070) 2024-05-09T10:50:24.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.965+0000 7f7010dbb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f700c0b4b00 0x7f700c15a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42072/0 (socket says 172.21.15.98:42072) 2024-05-09T10:50:24.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.965+0000 7fe35da1f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe350054930 con 0x7fe35004ffc0 2024-05-09T10:50:24.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.965+0000 7fe35da1f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3500547b0 con 0x7fe3500eacf0 2024-05-09T10:50:24.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.965+0000 7fe3577fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3500eacf0 0x7fe35004daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.965+0000 7fe356ffd640 1 -- 172.21.15.98:0/1019050705 learned_addr learned my addr 172.21.15.98:0/1019050705 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.965+0000 7fe35da1f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35004f540 con 0x7fe350006e90 2024-05-09T10:50:24.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.965+0000 7f7010dbb640 1 -- 172.21.15.98:0/4000074640 learned_addr learned my addr 172.21.15.98:0/4000074640 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.965+0000 7f7011dbd640 1 --2- 172.21.15.98:0/4000074640 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f700c159940 0x7f700c1f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.965+0000 7f6069963640 1 -- start start 2024-05-09T10:50:24.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7f63a946f640 1 -- 172.21.15.98:0/2403122990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f639c0df6e0 msgr2=0x7f639c054cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.965+0000 7f55fa8bd640 1 -- start start 2024-05-09T10:50:24.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7f63a946f640 1 --2- 172.21.15.98:0/2403122990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f639c0df6e0 0x7f639c054cc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f63940097a0 tx=0x7f639402a1c0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7f63a946f640 1 -- 172.21.15.98:0/2403122990 shutdown_connections 2024-05-09T10:50:24.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7f63a946f640 1 --2- 172.21.15.98:0/2403122990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f639c0df6e0 0x7f639c054cc0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7f63a946f640 1 -- 172.21.15.98:0/2403122990 >> 172.21.15.98:0/2403122990 conn(0x7f639c055340 msgr2=0x7f639c055750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:24.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7fe356ffd640 1 -- 172.21.15.98:0/1019050705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3500eacf0 msgr2=0x7fe35004daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7fe356ffd640 1 --2- 172.21.15.98:0/1019050705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3500eacf0 0x7fe35004daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7fe356ffd640 1 -- 172.21.15.98:0/1019050705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe35004ffc0 msgr2=0x7fe35004d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7fe356ffd640 1 --2- 172.21.15.98:0/1019050705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe35004ffc0 0x7fe35004d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7fe356ffd640 1 -- 172.21.15.98:0/1019050705 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3500e9080 con 0x7fe350006e90 2024-05-09T10:50:24.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7f7011dbd640 1 --2- 172.21.15.98:0/4000074640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f700c159d20 0x7f700c14a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7f63a946f640 1 -- 172.21.15.98:0/2403122990 shutdown_connections 2024-05-09T10:50:24.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7fe356ffd640 1 --2- 172.21.15.98:0/1019050705 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe350006e90 0x7fe3500d9470 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe34800e870 tx=0x7fe34800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:24.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7f700bfff640 1 --2- 172.21.15.98:0/4000074640 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f700c159940 0x7f700c1f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7f63a946f640 1 -- 172.21.15.98:0/2403122990 wait complete. 2024-05-09T10:50:24.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7f7011dbd640 1 -- 172.21.15.98:0/4000074640 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0720e0 con 0x7f700c0b4b00 2024-05-09T10:50:24.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7f70115bc640 1 --2- 172.21.15.98:0/4000074640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f700c159d20 0x7f700c14a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7f7011dbd640 1 -- 172.21.15.98:0/4000074640 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0aeab0 con 0x7f700c159d20 2024-05-09T10:50:24.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7fc9d4d89640 1 Processor -- start 2024-05-09T10:50:24.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7f7011dbd640 1 -- 172.21.15.98:0/4000074640 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700c072260 con 0x7f700c159940 2024-05-09T10:50:24.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7fc9d4d89640 1 -- start start 2024-05-09T10:50:24.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.967+0000 7f55fa8bd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f40cfbc0 0x7f55f414a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.967+0000 7f55fa8bd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55f415c770 0x7f55f4146f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.967+0000 7f6069963640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f605c00aad0 0x7f605c0cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.967+0000 7f55f3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f40cfbc0 0x7f55f414a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.967+0000 7f55f3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f40cfbc0 0x7f55f414a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.98:35284/0 (socket says 172.21.15.98:35284) 2024-05-09T10:50:24.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.967+0000 7f6069963640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f605c054690 0x7f605c0caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.967+0000 7f55f3fff640 1 -- 172.21.15.98:0/2421328453 learned_addr learned my addr 172.21.15.98:0/2421328453 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.967+0000 7f55fa8bd640 1 --2- 172.21.15.98:0/2421328453 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55f415cb50 0x7f55f4148130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.967+0000 7f6069963640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f605c0e3400 0x7f605c0e08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.967+0000 7f6062ffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f605c00aad0 0x7f605c0cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.967+0000 7f6062ffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f605c00aad0 0x7f605c0cc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:53276/0 (socket says 172.21.15.98:53276) 2024-05-09T10:50:24.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.967+0000 7f55fa8bd640 1 -- 172.21.15.98:0/2421328453 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40ce2f0 con 0x7f55f415c770 2024-05-09T10:50:24.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.967+0000 7f55fa8bd640 1 -- 172.21.15.98:0/2421328453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f4166ae0 con 0x7f55f40cfbc0 2024-05-09T10:50:24.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.967+0000 7f60627fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f605c054690 0x7f605c0caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.967+0000 7f60637fe640 1 --2- 172.21.15.98:0/2275308017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f605c0e3400 0x7f605c0e08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:24.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.967+0000 7f6062ffd640 1 -- 172.21.15.98:0/2275308017 learned_addr learned my addr 172.21.15.98:0/2275308017 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:24.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.967+0000 7f55fa8bd640 1 -- 172.21.15.98:0/2421328453 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40ce470 con 0x7f55f415cb50 2024-05-09T10:50:24.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.967+0000 7f6069963640 1 -- 172.21.15.98:0/2275308017 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605c050530 con 0x7f605c0e3400 2024-05-09T10:50:24.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.967+0000 7f6069963640 1 -- 172.21.15.98:0/2275308017 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605c054170 con 0x7f605c054690 2024-05-09T10:50:24.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f6069963640 1 -- 172.21.15.98:0/2275308017 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f605c0e04e0 con 0x7f605c00aad0 2024-05-09T10:50:24.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7f5a5313c640 1 -- 172.21.15.98:0/1272946851 wait complete. 2024-05-09T10:50:24.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f5a5313c640 1 Processor -- start 2024-05-09T10:50:24.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f5a5313c640 1 -- start start 2024-05-09T10:50:24.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f7010dbb640 1 -- 172.21.15.98:0/4000074640 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f700c159940 msgr2=0x7f700c1f43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f7010dbb640 1 --2- 172.21.15.98:0/4000074640 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f700c159940 0x7f700c1f43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f7010dbb640 1 -- 172.21.15.98:0/4000074640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f700c159d20 msgr2=0x7f700c14a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:24.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f7010dbb640 1 --2- 172.21.15.98:0/4000074640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f700c159d20 0x7f700c14a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:24.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f7010dbb640 1 -- 172.21.15.98:0/4000074640 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c1f48e0 con 0x7f700c0b4b00 2024-05-09T10:50:24.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7f034669e640 1 Processor -- start 2024-05-09T10:50:24.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7f034669e640 1 -- start start 2024-05-09T10:50:24.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f034669e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0338050190 0x7f0338038070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:24.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f034669e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0338053c60 0x7f0338006a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:25.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f033ffff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f03380df210 0x7f033804fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.98:41280/0 (socket says 172.21.15.98:41280) 2024-05-09T10:50:25.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f034669e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03380544b0 con 0x7f03380df210 2024-05-09T10:50:25.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f034669e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03380541d0 con 0x7f0338050190 2024-05-09T10:50:25.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f034669e640 1 -- 172.21.15.98:0/658339199 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0338005f20 con 0x7f0338053c60 2024-05-09T10:50:25.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f033ffff640 1 -- 172.21.15.98:0/658339199 learned_addr learned my addr 172.21.15.98:0/658339199 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:25.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f6062ffd640 1 -- 172.21.15.98:0/2275308017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f605c054690 msgr2=0x7f605c0caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f6062ffd640 1 --2- 172.21.15.98:0/2275308017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f605c054690 0x7f605c0caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f6062ffd640 1 -- 172.21.15.98:0/2275308017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f605c0e3400 msgr2=0x7f605c0e08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f6062ffd640 1 --2- 172.21.15.98:0/2275308017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f605c0e3400 0x7f605c0e08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f63a946f640 1 Processor -- start 2024-05-09T10:50:25.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f6062ffd640 1 -- 172.21.15.98:0/2275308017 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605c0cb380 con 0x7f605c00aad0 2024-05-09T10:50:25.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f60637fe640 1 --2- 172.21.15.98:0/2275308017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f605c0e3400 0x7f605c0e08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:50:25.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f63a946f640 1 -- start start 2024-05-09T10:50:25.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f033f7fe640 1 --2- 172.21.15.98:0/658339199 >> v2:172.21.15.98:3300/0 conn(0x7f0338050190 0x7f0338038070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:25.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7fe33ffff640 1 -- 172.21.15.98:0/1019050705 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe348017070 con 0x7fe350006e90 2024-05-09T10:50:25.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7fe33ffff640 1 -- 172.21.15.98:0/1019050705 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34800ee60 con 0x7fe350006e90 2024-05-09T10:50:25.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.966+0000 7fe33ffff640 1 -- 172.21.15.98:0/1019050705 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34800cb80 con 0x7fe350006e90 2024-05-09T10:50:25.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.967+0000 7fe35da1f640 1 -- 172.21.15.98:0/1019050705 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35004e150 con 0x7fe350006e90 2024-05-09T10:50:25.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.967+0000 7fe35da1f640 1 -- 172.21.15.98:0/1019050705 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3500d9d20 con 0x7fe350006e90 2024-05-09T10:50:25.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f55f37fe640 1 --2- 172.21.15.98:0/2421328453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55f415c770 0x7f55f4146f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:25.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f55f3fff640 1 -- 172.21.15.98:0/2421328453 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55f415cb50 msgr2=0x7f55f4148130 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T10:50:25.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f55f3fff640 1 --2- 172.21.15.98:0/2421328453 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55f415cb50 0x7f55f4148130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f55f3fff640 1 -- 172.21.15.98:0/2421328453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55f415c770 msgr2=0x7f55f4146f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f55f3fff640 1 --2- 172.21.15.98:0/2421328453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55f415c770 0x7f55f4146f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f55f3fff640 1 -- 172.21.15.98:0/2421328453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f41479e0 con 0x7f55f40cfbc0 2024-05-09T10:50:25.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7fc9d4d89640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9d0070750 0x7fc9d014b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:25.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7fc9d4d89640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9d0071460 0x7fc9d01f3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:25.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7fc9ce59c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9d0070750 0x7fc9d014b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:25.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7fc9ce59c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9d0070750 0x7fc9d014b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:53314/0 (socket says 172.21.15.98:53314) 2024-05-09T10:50:25.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7fc9ce59c640 1 -- 172.21.15.98:0/1178907971 learned_addr learned my addr 172.21.15.98:0/1178907971 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:25.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7fc9cdd9b640 1 --2- 172.21.15.98:0/1178907971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9d0071460 0x7fc9d01f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:25.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7fc9d4d89640 1 --2- 172.21.15.98:0/1178907971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9d0157970 0x7fc9d0156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:25.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7fc9d4d89640 1 -- 172.21.15.98:0/1178907971 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d006c9a0 con 0x7fc9d0071460 2024-05-09T10:50:25.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7fc9d4d89640 1 -- 172.21.15.98:0/1178907971 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d0157d50 con 0x7fc9d0157970 2024-05-09T10:50:25.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7fc9d4d89640 1 -- 172.21.15.98:0/1178907971 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d006cb20 con 0x7fc9d0070750 2024-05-09T10:50:25.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f5a5313c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5a4c0b20d0 0x7f5a4c1467c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:25.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f5a5313c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a4c0b4180 0x7f5a4c147d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:25.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f5a5213a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5a4c0b20d0 0x7f5a4c1467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:25.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f5a5213a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5a4c0b20d0 0x7f5a4c1467c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:53322/0 (socket says 172.21.15.98:53322) 2024-05-09T10:50:25.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.968+0000 7f7010dbb640 1 --2- 172.21.15.98:0/4000074640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f700c0b4b00 0x7f700c15a100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f700400cb20 tx=0x7f7004002160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:25.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f70097fa640 1 -- 172.21.15.98:0/4000074640 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7004016020 con 0x7f700c0b4b00 2024-05-09T10:50:25.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f70097fa640 1 -- 172.21.15.98:0/4000074640 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7004006d50 con 0x7f700c0b4b00 2024-05-09T10:50:25.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f70097fa640 1 -- 172.21.15.98:0/4000074640 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7004003bb0 con 0x7f700c0b4b00 2024-05-09T10:50:25.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f70115bc640 1 --2- 172.21.15.98:0/4000074640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f700c159d20 0x7f700c14a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:25.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f700bfff640 1 --2- 172.21.15.98:0/4000074640 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f700c159940 0x7f700c1f43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:25.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f7011dbd640 1 -- 172.21.15.98:0/4000074640 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700c156300 con 0x7f700c0b4b00 2024-05-09T10:50:25.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f7011dbd640 1 -- 172.21.15.98:0/4000074640 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700c1568c0 con 0x7f700c0b4b00 2024-05-09T10:50:25.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f5a51939640 1 --2- 172.21.15.98:0/2047359425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a4c0b4180 0x7f5a4c147d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:25.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f5a5213a640 1 -- 172.21.15.98:0/2047359425 learned_addr learned my addr 172.21.15.98:0/2047359425 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:25.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f5a5313c640 1 --2- 172.21.15.98:0/2047359425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a4c161df0 0x7f5a4c148260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:25.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f5a5313c640 1 -- 172.21.15.98:0/2047359425 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c0b45b0 con 0x7f5a4c161df0 2024-05-09T10:50:25.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f5a5313c640 1 -- 172.21.15.98:0/2047359425 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c0af830 con 0x7f5a4c0b4180 2024-05-09T10:50:25.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f5a5313c640 1 -- 172.21.15.98:0/2047359425 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c0b4730 con 0x7f5a4c0b20d0 2024-05-09T10:50:25.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7fe33dffb640 1 -- 172.21.15.98:0/1019050705 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe350005380 con 0x7fe350006e90 2024-05-09T10:50:25.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f6062ffd640 1 --2- 172.21.15.98:0/2275308017 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f605c00aad0 0x7f605c0cc420 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6064073630 tx=0x7f60640736d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:25.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f604bfff640 1 -- 172.21.15.98:0/2275308017 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60640ba070 con 0x7f605c00aad0 2024-05-09T10:50:25.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f604bfff640 1 -- 172.21.15.98:0/2275308017 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60640ac9b0 con 0x7f605c00aad0 2024-05-09T10:50:25.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f55f3fff640 1 --2- 172.21.15.98:0/2421328453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f40cfbc0 0x7f55f414a6f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f55e400e870 tx=0x7f55e400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:25.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f55f0ff9640 1 -- 172.21.15.98:0/2421328453 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e4017070 con 0x7f55f40cfbc0 2024-05-09T10:50:25.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f55f0ff9640 1 -- 172.21.15.98:0/2421328453 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e400ee60 con 0x7f55f40cfbc0 2024-05-09T10:50:25.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f55f0ff9640 1 -- 172.21.15.98:0/2421328453 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e400cb00 con 0x7f55f40cfbc0 2024-05-09T10:50:25.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f63a946f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f639c03ad40 0x7f639c0e23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:25.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f63a946f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f639c050080 0x7f639c0cd630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:25.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f033f7fe640 1 -- 172.21.15.98:0/658339199 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0338053c60 msgr2=0x7f0338006a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:25.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f033f7fe640 1 --2- 172.21.15.98:0/658339199 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0338053c60 0x7f0338006a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f033f7fe640 1 -- 172.21.15.98:0/658339199 >> v1:172.21.15.105:6789/0 conn(0x7f03380df210 legacy=0x7f033804fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:25.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7f033f7fe640 1 -- 172.21.15.98:0/658339199 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0338006200 con 0x7f0338050190 2024-05-09T10:50:25.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.969+0000 7fc9ced9d640 1 --2- 172.21.15.98:0/1178907971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9d0157970 0x7fc9d0156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:25.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7fc9ced9d640 1 -- 172.21.15.98:0/1178907971 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9d0070750 msgr2=0x7fc9d014b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7fc9ced9d640 1 --2- 172.21.15.98:0/1178907971 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9d0070750 0x7fc9d014b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7fc9ced9d640 1 -- 172.21.15.98:0/1178907971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9d0071460 msgr2=0x7fc9d01f3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7fc9ced9d640 1 --2- 172.21.15.98:0/1178907971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9d0071460 0x7fc9d01f3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7fc9ced9d640 1 -- 172.21.15.98:0/1178907971 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d014cb00 con 0x7fc9d0157970 2024-05-09T10:50:25.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f604bfff640 1 -- 172.21.15.98:0/2275308017 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60640b1040 con 0x7f605c00aad0 2024-05-09T10:50:25.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f6069963640 1 -- 172.21.15.98:0/2275308017 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f605c0cb660 con 0x7f605c00aad0 2024-05-09T10:50:25.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f6069963640 1 -- 172.21.15.98:0/2275308017 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f605c0cbc20 con 0x7f605c00aad0 2024-05-09T10:50:25.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f5a51939640 1 -- 172.21.15.98:0/2047359425 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5a4c0b20d0 msgr2=0x7f5a4c1467c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f5a51939640 1 --2- 172.21.15.98:0/2047359425 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5a4c0b20d0 0x7f5a4c1467c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f5a51939640 1 -- 172.21.15.98:0/2047359425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a4c161df0 msgr2=0x7f5a4c148260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f5a51939640 1 --2- 172.21.15.98:0/2047359425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a4c161df0 0x7f5a4c148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f5a51939640 1 -- 172.21.15.98:0/2047359425 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4c147260 con 0x7f5a4c0b4180 2024-05-09T10:50:25.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f5a5213a640 1 --2- 172.21.15.98:0/2047359425 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5a4c0b20d0 0x7f5a4c1467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:50:25.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f63a2ffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f639c03ad40 0x7f639c0e23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:25.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f63a2ffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f639c03ad40 0x7f639c0e23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:53326/0 (socket says 172.21.15.98:53326) 2024-05-09T10:50:25.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f63a2ffd640 1 -- 172.21.15.98:0/3863670514 learned_addr learned my addr 172.21.15.98:0/3863670514 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:25.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f63a946f640 1 --2- 172.21.15.98:0/3863670514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f639c0df6e0 0x7f639c036f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:25.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f033f7fe640 1 --2- 172.21.15.98:0/658339199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0338050190 0x7f0338038070 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f032c0097a0 tx=0x7f032c02c2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f6efe98e2730ab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:25.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f63a946f640 1 -- 172.21.15.98:0/3863670514 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c0063b0 con 0x7f639c0df6e0 2024-05-09T10:50:25.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f63a946f640 1 -- 172.21.15.98:0/3863670514 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c006230 con 0x7f639c050080 2024-05-09T10:50:25.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f63a946f640 1 -- 172.21.15.98:0/3863670514 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639c006530 con 0x7f639c03ad40 2024-05-09T10:50:25.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f63a27fc640 1 --2- 172.21.15.98:0/3863670514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f639c050080 0x7f639c0cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:25.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7f63a37fe640 1 --2- 172.21.15.98:0/3863670514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f639c0df6e0 0x7f639c036f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:25.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.971+0000 7f55fa8bd640 1 -- 172.21.15.98:0/2421328453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f4147cc0 con 0x7f55f40cfbc0 2024-05-09T10:50:25.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.971+0000 7f55fa8bd640 1 -- 172.21.15.98:0/2421328453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f41412f0 con 0x7f55f40cfbc0 2024-05-09T10:50:25.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.971+0000 7f5a51939640 1 --2- 172.21.15.98:0/2047359425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a4c0b4180 0x7f5a4c147d20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5a48031000 tx=0x7f5a480046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:25.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.970+0000 7fe33ffff640 1 -- 172.21.15.98:0/1019050705 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe34801a070 con 0x7fe350006e90 2024-05-09T10:50:25.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.971+0000 7f55f37fe640 1 --2- 172.21.15.98:0/2421328453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55f415c770 0x7f55f4146f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:25.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.971+0000 7f033effd640 1 -- 172.21.15.98:0/658339199 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c039070 con 0x7f0338050190 2024-05-09T10:50:25.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.971+0000 7f033effd640 1 -- 172.21.15.98:0/658339199 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032c034440 con 0x7f0338050190 2024-05-09T10:50:25.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.971+0000 7f7011dbd640 1 -- 172.21.15.98:0/4000074640 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f700c04ef20 con 0x7f700c0b4b00 2024-05-09T10:50:25.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.971+0000 7fc9ced9d640 1 --2- 172.21.15.98:0/1178907971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9d0157970 0x7fc9d0156140 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800cdf0 tx=0x7fc9c800a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:25.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.971+0000 7f5a42ffd640 1 -- 172.21.15.98:0/2047359425 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a48037040 con 0x7f5a4c0b4180 2024-05-09T10:50:25.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.971+0000 7f63a27fc640 1 -- 172.21.15.98:0/3863670514 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f639c03ad40 msgr2=0x7f639c0e23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.971+0000 7f63a27fc640 1 --2- 172.21.15.98:0/3863670514 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f639c03ad40 0x7f639c0e23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.971+0000 7f63a27fc640 1 -- 172.21.15.98:0/3863670514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f639c0df6e0 msgr2=0x7f639c036f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.971+0000 7f63a27fc640 1 --2- 172.21.15.98:0/3863670514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f639c0df6e0 0x7f639c036f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.971+0000 7f63a27fc640 1 -- 172.21.15.98:0/3863670514 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639c0e28f0 con 0x7f639c050080 2024-05-09T10:50:25.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.971+0000 7fe33ffff640 1 --2- 172.21.15.98:0/1019050705 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe34406b5a0 0x7fe34406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:25.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.971+0000 7fe33ffff640 1 -- 172.21.15.98:0/1019050705 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe34808a410 con 0x7fe350006e90 2024-05-09T10:50:25.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.972+0000 7fc9beffd640 1 -- 172.21.15.98:0/1178907971 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c801a070 con 0x7fc9d0157970 2024-05-09T10:50:25.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.972+0000 7fc9beffd640 1 -- 172.21.15.98:0/1178907971 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c800d840 con 0x7fc9d0157970 2024-05-09T10:50:25.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.972+0000 7f034669e640 1 -- 172.21.15.98:0/658339199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0338050190 msgr2=0x7f0338038070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.972+0000 7f034669e640 1 --2- 172.21.15.98:0/658339199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0338050190 0x7f0338038070 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f032c0097a0 tx=0x7f032c02c2c0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.972+0000 7fc9d4d89640 1 -- 172.21.15.98:0/1178907971 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d014ba60 con 0x7fc9d0157970 2024-05-09T10:50:25.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.972+0000 7fc9d4d89640 1 -- 172.21.15.98:0/1178907971 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d014c020 con 0x7fc9d0157970 2024-05-09T10:50:25.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.972+0000 7fc9beffd640 1 -- 172.21.15.98:0/1178907971 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c800dc70 con 0x7fc9d0157970 2024-05-09T10:50:25.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.972+0000 7f63a2ffd640 1 --2- 172.21.15.98:0/3863670514 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f639c03ad40 0x7f639c0e23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:25.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.972+0000 7f034669e640 1 -- 172.21.15.98:0/658339199 shutdown_connections 2024-05-09T10:50:25.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.972+0000 7f034669e640 1 --2- 172.21.15.98:0/658339199 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0338053c60 0x7f0338006a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.972+0000 7f034669e640 1 --2- 172.21.15.98:0/658339199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0338050190 0x7f0338038070 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.972+0000 7f034669e640 1 -- 172.21.15.98:0/658339199 >> 172.21.15.98:0/658339199 conn(0x7f0338009fa0 msgr2=0x7f033800a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:25.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.972+0000 7f63a27fc640 1 --2- 172.21.15.98:0/3863670514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f639c050080 0x7f639c0cd630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f639c03b1b0 tx=0x7f639800c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:25.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.971+0000 7f5a42ffd640 1 -- 172.21.15.98:0/2047359425 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a48030a70 con 0x7f5a4c0b4180 2024-05-09T10:50:25.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.972+0000 7f5a5313c640 1 -- 172.21.15.98:0/2047359425 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a4c147540 con 0x7f5a4c0b4180 2024-05-09T10:50:25.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.972+0000 7f5a5313c640 1 -- 172.21.15.98:0/2047359425 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a4c14ddb0 con 0x7f5a4c0b4180 2024-05-09T10:50:25.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.972+0000 7f5a42ffd640 1 -- 172.21.15.98:0/2047359425 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a48037040 con 0x7f5a4c0b4180 2024-05-09T10:50:25.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.972+0000 7f604bfff640 1 -- 172.21.15.98:0/2275308017 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f60640aa690 con 0x7f605c00aad0 2024-05-09T10:50:25.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.971+0000 7f6069963640 1 -- 172.21.15.98:0/2275308017 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f605c008a00 con 0x7f605c00aad0 2024-05-09T10:50:25.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.973+0000 7fe3567fc640 1 --2- 172.21.15.98:0/1019050705 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe34406b5a0 0x7fe34406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:25.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.973+0000 7f604bfff640 1 --2- 172.21.15.98:0/2275308017 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f605006b5a0 0x7f605006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:25.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.973+0000 7f638b7fe640 1 -- 172.21.15.98:0/3863670514 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6398005220 con 0x7f639c050080 2024-05-09T10:50:25.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.973+0000 7f638b7fe640 1 -- 172.21.15.98:0/3863670514 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63980053c0 con 0x7f639c050080 2024-05-09T10:50:25.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.973+0000 7f638b7fe640 1 -- 172.21.15.98:0/3863670514 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6398007250 con 0x7f639c050080 2024-05-09T10:50:25.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.973+0000 7f604bfff640 1 -- 172.21.15.98:0/2275308017 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6064125330 con 0x7f605c00aad0 2024-05-09T10:50:25.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.973+0000 7f034669e640 1 -- 172.21.15.98:0/658339199 shutdown_connections 2024-05-09T10:50:25.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.973+0000 7f034669e640 1 -- 172.21.15.98:0/658339199 wait complete. 2024-05-09T10:50:25.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.974+0000 7f60627fc640 1 --2- 172.21.15.98:0/2275308017 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f605006b5a0 0x7f605006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:25.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.973+0000 7fe3567fc640 1 --2- 172.21.15.98:0/1019050705 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe34406b5a0 0x7fe34406da60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe350005320 tx=0x7fe34c007e30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:25.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.973+0000 7f55f0ff9640 1 -- 172.21.15.98:0/2421328453 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f55e401a070 con 0x7f55f40cfbc0 2024-05-09T10:50:25.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.974+0000 7f5a5313c640 1 -- 172.21.15.98:0/2047359425 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a4c04ef20 con 0x7f5a4c0b4180 2024-05-09T10:50:25.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.974+0000 7f63a946f640 1 -- 172.21.15.98:0/3863670514 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639c0e06e0 con 0x7f639c050080 2024-05-09T10:50:25.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.974+0000 7f63a946f640 1 -- 172.21.15.98:0/3863670514 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639c0e0ca0 con 0x7f639c050080 2024-05-09T10:50:25.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.974+0000 7f55fa8bd640 1 -- 172.21.15.98:0/2421328453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f40cf380 con 0x7f55f40cfbc0 2024-05-09T10:50:25.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.974+0000 7f55f0ff9640 1 --2- 172.21.15.98:0/2421328453 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f55e006b5d0 0x7f55e006da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:25.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.974+0000 7f55f0ff9640 1 -- 172.21.15.98:0/2421328453 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55e408a260 con 0x7f55f40cfbc0 2024-05-09T10:50:25.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.973+0000 7f70097fa640 1 -- 172.21.15.98:0/4000074640 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f700400a7c0 con 0x7f700c0b4b00 2024-05-09T10:50:25.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.973+0000 7f70097fa640 1 --2- 172.21.15.98:0/4000074640 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6ff806b5a0 0x7f6ff806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:25.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.973+0000 7f70097fa640 1 -- 172.21.15.98:0/4000074640 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7004089e80 con 0x7f700c0b4b00 2024-05-09T10:50:25.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.974+0000 7f034669e640 1 Processor -- start 2024-05-09T10:50:25.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.975+0000 7f55f37fe640 1 --2- 172.21.15.98:0/2421328453 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f55e006b5d0 0x7f55e006da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:25.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.975+0000 7f700bfff640 1 --2- 172.21.15.98:0/4000074640 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6ff806b5a0 0x7f6ff806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:25.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.973+0000 7fc9d4d89640 1 -- 172.21.15.98:0/1178907971 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d004ef20 con 0x7fc9d0157970 2024-05-09T10:50:25.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.975+0000 7f034669e640 1 -- start start 2024-05-09T10:50:25.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.975+0000 7fc9beffd640 1 -- 172.21.15.98:0/1178907971 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc9c801d070 con 0x7fc9d0157970 2024-05-09T10:50:25.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.975+0000 7f63a946f640 1 -- 172.21.15.98:0/3863670514 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639c04bc00 con 0x7f639c050080 2024-05-09T10:50:25.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.975+0000 7f5a42ffd640 1 -- 172.21.15.98:0/2047359425 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a48044020 con 0x7f5a4c0b4180 2024-05-09T10:50:25.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.975+0000 7f5a42ffd640 1 --2- 172.21.15.98:0/2047359425 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5a3006b8d0 0x7f5a3006dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:25.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.976+0000 7f700bfff640 1 --2- 172.21.15.98:0/4000074640 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6ff806b5a0 0x7f6ff806da60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7000034000 tx=0x7f700002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:25.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.976+0000 7f034669e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0338050190 0x7f0338040840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:25.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.976+0000 7f034669e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0338053c60 0x7f0338049630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:25.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.976+0000 7f034669e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03380df210 0x7f0338049b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:25.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.976+0000 7f5a5213a640 1 --2- 172.21.15.98:0/2047359425 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5a3006b8d0 0x7f5a3006dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:25.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.976+0000 7f5a42ffd640 1 -- 172.21.15.98:0/2047359425 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a48042020 con 0x7f5a4c0b4180 2024-05-09T10:50:25.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.976+0000 7fc9beffd640 1 --2- 172.21.15.98:0/1178907971 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc9b006b520 0x7fc9b006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:25.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.976+0000 7f034669e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0338037a80 con 0x7f0338050190 2024-05-09T10:50:25.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.976+0000 7f034669e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0338037900 con 0x7f0338053c60 2024-05-09T10:50:25.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.976+0000 7f034669e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03380df5f0 con 0x7f03380df210 2024-05-09T10:50:25.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.976+0000 7f638b7fe640 1 -- 172.21.15.98:0/3863670514 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6398019020 con 0x7f639c050080 2024-05-09T10:50:25.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.976+0000 7f033ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0338050190 0x7f0338040840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:25.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.976+0000 7fc9beffd640 1 -- 172.21.15.98:0/1178907971 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9c808a990 con 0x7fc9d0157970 2024-05-09T10:50:25.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.976+0000 7f033ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0338050190 0x7f0338040840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42126/0 (socket says 172.21.15.98:42126) 2024-05-09T10:50:25.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.976+0000 7fe33ffff640 1 -- 172.21.15.98:0/1019050705 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe348058b90 con 0x7fe350006e90 2024-05-09T10:50:25.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.976+0000 7f033ffff640 1 -- 172.21.15.98:0/3905005262 learned_addr learned my addr 172.21.15.98:0/3905005262 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:25.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.976+0000 7f60627fc640 1 --2- 172.21.15.98:0/2275308017 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f605006b5a0 0x7f605006da60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f605c038ae0 tx=0x7f6058009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:25.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.976+0000 7f0344c3b640 1 --2- 172.21.15.98:0/3905005262 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03380df210 0x7f0338049b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:25.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.976+0000 7f033f7fe640 1 --2- 172.21.15.98:0/3905005262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0338053c60 0x7f0338049630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:25.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.976+0000 7f55f37fe640 1 --2- 172.21.15.98:0/2421328453 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f55e006b5d0 0x7f55e006da90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f55f40726b0 tx=0x7f55e8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:25.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.976+0000 7fc9ce59c640 1 --2- 172.21.15.98:0/1178907971 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc9b006b520 0x7fc9b006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:25.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.976+0000 7f5a5213a640 1 --2- 172.21.15.98:0/2047359425 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5a3006b8d0 0x7f5a3006dd90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c15e0c0 tx=0x7f5a4400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:25.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.977+0000 7f638b7fe640 1 --2- 172.21.15.98:0/3863670514 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f639006b520 0x7f639006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:25.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.977+0000 7f0344c3b640 1 -- 172.21.15.98:0/3905005262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0338053c60 msgr2=0x7f0338049630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.977+0000 7f0344c3b640 1 --2- 172.21.15.98:0/3905005262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0338053c60 0x7f0338049630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.977+0000 7f0344c3b640 1 -- 172.21.15.98:0/3905005262 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0338050190 msgr2=0x7f0338040840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.977+0000 7f0344c3b640 1 --2- 172.21.15.98:0/3905005262 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0338050190 0x7f0338040840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.977+0000 7f0344c3b640 1 -- 172.21.15.98:0/3905005262 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033804c3e0 con 0x7f03380df210 2024-05-09T10:50:25.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.977+0000 7f033f7fe640 1 --2- 172.21.15.98:0/3905005262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0338053c60 0x7f0338049630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:50:25.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.977+0000 7f033ffff640 1 --2- 172.21.15.98:0/3905005262 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0338050190 0x7f0338040840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:25.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.977+0000 7f638b7fe640 1 -- 172.21.15.98:0/3863670514 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6398055d30 con 0x7f639c050080 2024-05-09T10:50:25.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.977+0000 7f0344c3b640 1 --2- 172.21.15.98:0/3905005262 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03380df210 0x7f0338049b70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0340069340 tx=0x7f0340068000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:25.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.977+0000 7f63a2ffd640 1 --2- 172.21.15.98:0/3863670514 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f639006b520 0x7f639006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:25.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.977+0000 7fc9ce59c640 1 --2- 172.21.15.98:0/1178907971 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc9b006b520 0x7fc9b006d9e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0000c00 tx=0x7fc9c000e040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:25.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.977+0000 7f63a2ffd640 1 --2- 172.21.15.98:0/3863670514 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f639006b520 0x7f639006d9e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6394009560 tx=0x7f639402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:25.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.980+0000 7f55f0ff9640 1 -- 172.21.15.98:0/2421328453 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f55f40cf380 con 0x7f55f40cfbc0 2024-05-09T10:50:25.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.980+0000 7f70097fa640 1 -- 172.21.15.98:0/4000074640 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7004020020 con 0x7f700c0b4b00 2024-05-09T10:50:25.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.980+0000 7f033cff9640 1 -- 172.21.15.98:0/3905005262 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03400723d0 con 0x7f03380df210 2024-05-09T10:50:25.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.980+0000 7f034669e640 1 -- 172.21.15.98:0/3905005262 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033804be60 con 0x7f03380df210 2024-05-09T10:50:25.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.980+0000 7f034669e640 1 -- 172.21.15.98:0/3905005262 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033804c7d0 con 0x7f03380df210 2024-05-09T10:50:25.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.981+0000 7f033cff9640 1 -- 172.21.15.98:0/3905005262 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0340072590 con 0x7f03380df210 2024-05-09T10:50:25.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.981+0000 7f033cff9640 1 -- 172.21.15.98:0/3905005262 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03400729c0 con 0x7f03380df210 2024-05-09T10:50:25.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.980+0000 7f604bfff640 1 -- 172.21.15.98:0/2275308017 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f60640c4020 con 0x7f605c00aad0 2024-05-09T10:50:25.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.982+0000 7f033cff9640 1 -- 172.21.15.98:0/3905005262 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0340093030 con 0x7f03380df210 2024-05-09T10:50:25.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.982+0000 7f034669e640 1 -- 172.21.15.98:0/3905005262 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0338005380 con 0x7f03380df210 2024-05-09T10:50:25.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.982+0000 7f033cff9640 1 --2- 172.21.15.98:0/3905005262 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f032406b520 0x7f032406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:25.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.983+0000 7f033cff9640 1 -- 172.21.15.98:0/3905005262 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0340104280 con 0x7f03380df210 2024-05-09T10:50:25.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.983+0000 7f033ffff640 1 --2- 172.21.15.98:0/3905005262 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f032406b520 0x7f032406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:25.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.983+0000 7f033ffff640 1 --2- 172.21.15.98:0/3905005262 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f032406b520 0x7f032406d9e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0334005220 tx=0x7f0334009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:25.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.984+0000 7fc9beffd640 1 -- 172.21.15.98:0/1178907971 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc9c80598f0 con 0x7fc9d0157970 2024-05-09T10:50:25.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.987+0000 7f033cff9640 1 -- 172.21.15.98:0/3905005262 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f034009a020 con 0x7f03380df210 2024-05-09T10:50:25.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.989+0000 7f5a42ffd640 1 -- 172.21.15.98:0/2047359425 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a48095e20 con 0x7f5a4c0b4180 2024-05-09T10:50:25.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:24.994+0000 7f638b7fe640 1 -- 172.21.15.98:0/3863670514 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f639c04bc00 con 0x7f639c050080 2024-05-09T10:50:25.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.379+0000 7f6069963640 1 -- 172.21.15.98:0/2275308017 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f605c0e1f40 con 0x7f605c00aad0 2024-05-09T10:50:25.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.379+0000 7f604bfff640 1 -- 172.21.15.98:0/2275308017 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f60640f3bd0 con 0x7f605c00aad0 2024-05-09T10:50:25.373 INFO:teuthology.orchestra.run.smithi098.stdout:55834574854 2024-05-09T10:50:25.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.382+0000 7f6049ffb640 1 -- 172.21.15.98:0/2275308017 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f605006b5a0 msgr2=0x7f605006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.383+0000 7f6049ffb640 1 --2- 172.21.15.98:0/2275308017 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f605006b5a0 0x7f605006da60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f605c038ae0 tx=0x7f6058009040 comp rx=0 tx=0).stop 2024-05-09T10:50:25.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.383+0000 7f6049ffb640 1 -- 172.21.15.98:0/2275308017 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f605c00aad0 msgr2=0x7f605c0cc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.383+0000 7f6049ffb640 1 --2- 172.21.15.98:0/2275308017 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f605c00aad0 0x7f605c0cc420 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6064073630 tx=0x7f60640736d0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.383+0000 7f6049ffb640 1 -- 172.21.15.98:0/2275308017 shutdown_connections 2024-05-09T10:50:25.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.383+0000 7f6049ffb640 1 --2- 172.21.15.98:0/2275308017 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f605006b5a0 0x7f605006da60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.383+0000 7f6049ffb640 1 --2- 172.21.15.98:0/2275308017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f605c0e3400 0x7f605c0e08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.383+0000 7f6049ffb640 1 --2- 172.21.15.98:0/2275308017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f605c054690 0x7f605c0caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.383+0000 7f6049ffb640 1 --2- 172.21.15.98:0/2275308017 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f605c00aad0 0x7f605c0cc420 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.383+0000 7f6049ffb640 1 -- 172.21.15.98:0/2275308017 >> 172.21.15.98:0/2275308017 conn(0x7f605c009fe0 msgr2=0x7f605c038000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:25.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.384+0000 7f6049ffb640 1 -- 172.21.15.98:0/2275308017 shutdown_connections 2024-05-09T10:50:25.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.384+0000 7f6049ffb640 1 -- 172.21.15.98:0/2275308017 wait complete. 2024-05-09T10:50:25.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.387+0000 7fe33dffb640 1 -- 172.21.15.98:0/1019050705 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe35000a700 con 0x7fe350006e90 2024-05-09T10:50:25.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.388+0000 7fe33ffff640 1 -- 172.21.15.98:0/1019050705 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe34805cc30 con 0x7fe350006e90 2024-05-09T10:50:25.381 INFO:teuthology.orchestra.run.smithi098.stdout:55834574854 2024-05-09T10:50:25.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.391+0000 7f55fa8bd640 1 -- 172.21.15.98:0/2421328453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f55f4069cb0 con 0x7f55f40cfbc0 2024-05-09T10:50:25.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.391+0000 7fe33dffb640 1 -- 172.21.15.98:0/1019050705 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe34406b5a0 msgr2=0x7fe34406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.391+0000 7fe33dffb640 1 --2- 172.21.15.98:0/1019050705 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe34406b5a0 0x7fe34406da60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe350005320 tx=0x7fe34c007e30 comp rx=0 tx=0).stop 2024-05-09T10:50:25.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.391+0000 7f55f0ff9640 1 -- 172.21.15.98:0/2421328453 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f55e4059150 con 0x7f55f40cfbc0 2024-05-09T10:50:25.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.391+0000 7f5a5313c640 1 -- 172.21.15.98:0/2047359425 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5a4c071e60 con 0x7f5a4c0b4180 2024-05-09T10:50:25.385 INFO:teuthology.orchestra.run.smithi098.stdout:55834574855 2024-05-09T10:50:25.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.391+0000 7fe33dffb640 1 -- 172.21.15.98:0/1019050705 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe350006e90 msgr2=0x7fe3500d9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.391+0000 7fe33dffb640 1 --2- 172.21.15.98:0/1019050705 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe350006e90 0x7fe3500d9470 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe34800e870 tx=0x7fe34800ed30 comp rx=0 tx=0).stop 2024-05-09T10:50:25.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.391+0000 7fe33dffb640 1 -- 172.21.15.98:0/1019050705 shutdown_connections 2024-05-09T10:50:25.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.391+0000 7fe33dffb640 1 --2- 172.21.15.98:0/1019050705 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe34406b5a0 0x7fe34406da60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.392+0000 7fe33dffb640 1 --2- 172.21.15.98:0/1019050705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3500eacf0 0x7fe35004daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.392+0000 7fe33dffb640 1 --2- 172.21.15.98:0/1019050705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe35004ffc0 0x7fe35004d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.392+0000 7f5a42ffd640 1 -- 172.21.15.98:0/2047359425 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5a4c0cc8e0 con 0x7f5a4c0b4180 2024-05-09T10:50:25.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.392+0000 7fe33dffb640 1 --2- 172.21.15.98:0/1019050705 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe350006e90 0x7fe3500d9470 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.392+0000 7fe33dffb640 1 -- 172.21.15.98:0/1019050705 >> 172.21.15.98:0/1019050705 conn(0x7fe350009fa0 msgr2=0x7fe35004a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:25.387 INFO:teuthology.orchestra.run.smithi098.stdout:55834574855 2024-05-09T10:50:25.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.392+0000 7fe33dffb640 1 -- 172.21.15.98:0/1019050705 shutdown_connections 2024-05-09T10:50:25.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.392+0000 7fe33dffb640 1 -- 172.21.15.98:0/1019050705 wait complete. 2024-05-09T10:50:25.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.395+0000 7f55fa8bd640 1 -- 172.21.15.98:0/2421328453 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f55e006b5d0 msgr2=0x7f55e006da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.395+0000 7f55fa8bd640 1 --2- 172.21.15.98:0/2421328453 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f55e006b5d0 0x7f55e006da90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f55f40726b0 tx=0x7f55e8008040 comp rx=0 tx=0).stop 2024-05-09T10:50:25.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.395+0000 7fc9d4d89640 1 -- 172.21.15.98:0/1178907971 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc9d006d280 con 0x7fc9d0157970 2024-05-09T10:50:25.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.395+0000 7f55fa8bd640 1 -- 172.21.15.98:0/2421328453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f40cfbc0 msgr2=0x7f55f414a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.395+0000 7f55fa8bd640 1 --2- 172.21.15.98:0/2421328453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f40cfbc0 0x7f55f414a6f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f55e400e870 tx=0x7f55e400ed30 comp rx=0 tx=0).stop 2024-05-09T10:50:25.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.395+0000 7f7011dbd640 1 -- 172.21.15.98:0/4000074640 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f700c0cb980 con 0x7f700c0b4b00 2024-05-09T10:50:25.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.395+0000 7f55fa8bd640 1 -- 172.21.15.98:0/2421328453 shutdown_connections 2024-05-09T10:50:25.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.395+0000 7f55fa8bd640 1 --2- 172.21.15.98:0/2421328453 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f55e006b5d0 0x7f55e006da90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.395+0000 7f55fa8bd640 1 --2- 172.21.15.98:0/2421328453 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55f415cb50 0x7f55f4148130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.395+0000 7f55fa8bd640 1 --2- 172.21.15.98:0/2421328453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55f415c770 0x7f55f4146f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.395+0000 7f55fa8bd640 1 --2- 172.21.15.98:0/2421328453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55f40cfbc0 0x7f55f414a6f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.395+0000 7f55fa8bd640 1 -- 172.21.15.98:0/2421328453 >> 172.21.15.98:0/2421328453 conn(0x7f55f406abe0 msgr2=0x7f55f40b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:25.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.395+0000 7f5a5313c640 1 -- 172.21.15.98:0/2047359425 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5a3006b8d0 msgr2=0x7f5a3006dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.395+0000 7f5a5313c640 1 --2- 172.21.15.98:0/2047359425 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5a3006b8d0 0x7f5a3006dd90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c15e0c0 tx=0x7f5a4400b040 comp rx=0 tx=0).stop 2024-05-09T10:50:25.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.395+0000 7f5a5313c640 1 -- 172.21.15.98:0/2047359425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a4c0b4180 msgr2=0x7f5a4c147d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.395+0000 7f5a5313c640 1 --2- 172.21.15.98:0/2047359425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a4c0b4180 0x7f5a4c147d20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5a48031000 tx=0x7f5a480046c0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.396+0000 7f5a5313c640 1 -- 172.21.15.98:0/2047359425 shutdown_connections 2024-05-09T10:50:25.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.396+0000 7f5a5313c640 1 --2- 172.21.15.98:0/2047359425 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5a3006b8d0 0x7f5a3006dd90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.396+0000 7f5a5313c640 1 --2- 172.21.15.98:0/2047359425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a4c161df0 0x7f5a4c148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.396+0000 7f5a5313c640 1 --2- 172.21.15.98:0/2047359425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a4c0b4180 0x7f5a4c147d20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.396+0000 7f5a5313c640 1 --2- 172.21.15.98:0/2047359425 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5a4c0b20d0 0x7f5a4c1467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.396+0000 7f5a5313c640 1 -- 172.21.15.98:0/2047359425 >> 172.21.15.98:0/2047359425 conn(0x7f5a4c06c140 msgr2=0x7f5a4c0cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:25.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.396+0000 7f5a5313c640 1 -- 172.21.15.98:0/2047359425 shutdown_connections 2024-05-09T10:50:25.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.396+0000 7f5a5313c640 1 -- 172.21.15.98:0/2047359425 wait complete. 2024-05-09T10:50:25.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.396+0000 7fc9beffd640 1 -- 172.21.15.98:0/1178907971 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc9c80591f0 con 0x7fc9d0157970 2024-05-09T10:50:25.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.396+0000 7f70097fa640 1 -- 172.21.15.98:0/4000074640 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f700c1f48e0 con 0x7f700c0b4b00 2024-05-09T10:50:25.395 INFO:teuthology.orchestra.run.smithi098.stdout:55834574852 2024-05-09T10:50:25.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.396+0000 7f55fa8bd640 1 -- 172.21.15.98:0/2421328453 shutdown_connections 2024-05-09T10:50:25.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.396+0000 7f55fa8bd640 1 -- 172.21.15.98:0/2421328453 wait complete. 2024-05-09T10:50:25.396 INFO:teuthology.orchestra.run.smithi098.stdout:55834574855 2024-05-09T10:50:25.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.399+0000 7f6ff2ffd640 1 -- 172.21.15.98:0/4000074640 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6ff806b5a0 msgr2=0x7f6ff806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.399+0000 7f6ff2ffd640 1 --2- 172.21.15.98:0/4000074640 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6ff806b5a0 0x7f6ff806da60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7000034000 tx=0x7f700002e000 comp rx=0 tx=0).stop 2024-05-09T10:50:25.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.399+0000 7f6ff2ffd640 1 -- 172.21.15.98:0/4000074640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f700c0b4b00 msgr2=0x7f700c15a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.399+0000 7f6ff2ffd640 1 --2- 172.21.15.98:0/4000074640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f700c0b4b00 0x7f700c15a100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f700400cb20 tx=0x7f7004002160 comp rx=0 tx=0).stop 2024-05-09T10:50:25.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.400+0000 7fc9bcff9640 1 -- 172.21.15.98:0/1178907971 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc9b006b520 msgr2=0x7fc9b006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.400+0000 7fc9bcff9640 1 --2- 172.21.15.98:0/1178907971 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc9b006b520 0x7fc9b006d9e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0000c00 tx=0x7fc9c000e040 comp rx=0 tx=0).stop 2024-05-09T10:50:25.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.400+0000 7fc9bcff9640 1 -- 172.21.15.98:0/1178907971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9d0157970 msgr2=0x7fc9d0156140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.400+0000 7f6ff2ffd640 1 -- 172.21.15.98:0/4000074640 shutdown_connections 2024-05-09T10:50:25.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.400+0000 7f6ff2ffd640 1 --2- 172.21.15.98:0/4000074640 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6ff806b5a0 0x7f6ff806da60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.400+0000 7f6ff2ffd640 1 --2- 172.21.15.98:0/4000074640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f700c159d20 0x7f700c14a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.400+0000 7f6ff2ffd640 1 --2- 172.21.15.98:0/4000074640 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f700c159940 0x7f700c1f43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.400+0000 7f6ff2ffd640 1 --2- 172.21.15.98:0/4000074640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f700c0b4b00 0x7f700c15a100 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.400+0000 7fc9bcff9640 1 --2- 172.21.15.98:0/1178907971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9d0157970 0x7fc9d0156140 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800cdf0 tx=0x7fc9c800a710 comp rx=0 tx=0).stop 2024-05-09T10:50:25.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.400+0000 7f6ff2ffd640 1 -- 172.21.15.98:0/4000074640 >> 172.21.15.98:0/4000074640 conn(0x7f700c06abe0 msgr2=0x7f700c0b2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:25.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.400+0000 7fc9bcff9640 1 -- 172.21.15.98:0/1178907971 shutdown_connections 2024-05-09T10:50:25.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.400+0000 7fc9bcff9640 1 --2- 172.21.15.98:0/1178907971 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc9b006b520 0x7fc9b006d9e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.400+0000 7fc9bcff9640 1 --2- 172.21.15.98:0/1178907971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9d0157970 0x7fc9d0156140 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.400+0000 7fc9bcff9640 1 --2- 172.21.15.98:0/1178907971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9d0071460 0x7fc9d01f3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.400+0000 7fc9bcff9640 1 --2- 172.21.15.98:0/1178907971 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9d0070750 0x7fc9d014b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.400+0000 7f6ff2ffd640 1 -- 172.21.15.98:0/4000074640 shutdown_connections 2024-05-09T10:50:25.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.400+0000 7fc9bcff9640 1 -- 172.21.15.98:0/1178907971 >> 172.21.15.98:0/1178907971 conn(0x7fc9d00b15e0 msgr2=0x7fc9d00b4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:25.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.400+0000 7f6ff2ffd640 1 -- 172.21.15.98:0/4000074640 wait complete. 2024-05-09T10:50:25.400 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-05-09T10:50:25.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.402+0000 7fc9bcff9640 1 -- 172.21.15.98:0/1178907971 shutdown_connections 2024-05-09T10:50:25.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.402+0000 7f034669e640 1 -- 172.21.15.98:0/3905005262 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0338052bd0 con 0x7f03380df210 2024-05-09T10:50:25.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.402+0000 7fc9bcff9640 1 -- 172.21.15.98:0/1178907971 wait complete. 2024-05-09T10:50:25.401 INFO:teuthology.orchestra.run.smithi098.stdout:55834574853 2024-05-09T10:50:25.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.403+0000 7f033cff9640 1 -- 172.21.15.98:0/3905005262 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f03400d31f0 con 0x7f03380df210 2024-05-09T10:50:25.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.406+0000 7f034669e640 1 -- 172.21.15.98:0/3905005262 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f032406b520 msgr2=0x7f032406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.406+0000 7f034669e640 1 --2- 172.21.15.98:0/3905005262 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f032406b520 0x7f032406d9e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0334005220 tx=0x7f0334009040 comp rx=0 tx=0).stop 2024-05-09T10:50:25.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.406+0000 7f034669e640 1 -- 172.21.15.98:0/3905005262 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03380df210 msgr2=0x7f0338049b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.406+0000 7f034669e640 1 --2- 172.21.15.98:0/3905005262 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03380df210 0x7f0338049b70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0340069340 tx=0x7f0340068000 comp rx=0 tx=0).stop 2024-05-09T10:50:25.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.406+0000 7f63a946f640 1 -- 172.21.15.98:0/3863670514 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f639c054cc0 con 0x7f639c050080 2024-05-09T10:50:25.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.407+0000 7f034669e640 1 -- 172.21.15.98:0/3905005262 shutdown_connections 2024-05-09T10:50:25.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.407+0000 7f034669e640 1 --2- 172.21.15.98:0/3905005262 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f032406b520 0x7f032406d9e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.407+0000 7f034669e640 1 --2- 172.21.15.98:0/3905005262 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03380df210 0x7f0338049b70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.407+0000 7f034669e640 1 --2- 172.21.15.98:0/3905005262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0338053c60 0x7f0338049630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.407+0000 7f034669e640 1 --2- 172.21.15.98:0/3905005262 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0338050190 0x7f0338040840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.407+0000 7f034669e640 1 -- 172.21.15.98:0/3905005262 >> 172.21.15.98:0/3905005262 conn(0x7f0338009fa0 msgr2=0x7f0338036e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:25.404 INFO:teuthology.orchestra.run.smithi098.stdout:55834574854 2024-05-09T10:50:25.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.407+0000 7f638b7fe640 1 -- 172.21.15.98:0/3863670514 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f639c054cc0 con 0x7f639c050080 2024-05-09T10:50:25.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.407+0000 7f034669e640 1 -- 172.21.15.98:0/3905005262 shutdown_connections 2024-05-09T10:50:25.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.407+0000 7f034669e640 1 -- 172.21.15.98:0/3905005262 wait complete. 2024-05-09T10:50:25.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.411+0000 7f63a946f640 1 -- 172.21.15.98:0/3863670514 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f639006b520 msgr2=0x7f639006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.411+0000 7f63a946f640 1 --2- 172.21.15.98:0/3863670514 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f639006b520 0x7f639006d9e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6394009560 tx=0x7f639402e000 comp rx=0 tx=0).stop 2024-05-09T10:50:25.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.411+0000 7f63a946f640 1 -- 172.21.15.98:0/3863670514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f639c050080 msgr2=0x7f639c0cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:25.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.411+0000 7f63a946f640 1 --2- 172.21.15.98:0/3863670514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f639c050080 0x7f639c0cd630 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f639c03b1b0 tx=0x7f639800c820 comp rx=0 tx=0).stop 2024-05-09T10:50:25.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.411+0000 7f63a946f640 1 -- 172.21.15.98:0/3863670514 shutdown_connections 2024-05-09T10:50:25.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.411+0000 7f63a946f640 1 --2- 172.21.15.98:0/3863670514 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f639006b520 0x7f639006d9e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.411+0000 7f63a946f640 1 --2- 172.21.15.98:0/3863670514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f639c0df6e0 0x7f639c036f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.411+0000 7f63a946f640 1 --2- 172.21.15.98:0/3863670514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f639c050080 0x7f639c0cd630 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f639c03b1b0 tx=0x7f639800c820 comp rx=0 tx=0).stop 2024-05-09T10:50:25.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.411+0000 7f63a946f640 1 --2- 172.21.15.98:0/3863670514 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f639c03ad40 0x7f639c0e23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:25.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.411+0000 7f63a946f640 1 -- 172.21.15.98:0/3863670514 >> 172.21.15.98:0/3863670514 conn(0x7f639c055340 msgr2=0x7f639c04e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:25.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.411+0000 7f63a946f640 1 -- 172.21.15.98:0/3863670514 shutdown_connections 2024-05-09T10:50:25.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:25.411+0000 7f63a946f640 1 -- 172.21.15.98:0/3863670514 wait complete. 2024-05-09T10:50:25.407 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2024-05-09T10:50:25.414 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.0 2024-05-09T10:50:25.415 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.7 2024-05-09T10:50:25.416 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.1 2024-05-09T10:50:25.416 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2024-05-09T10:50:25.419 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2024-05-09T10:50:25.420 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2024-05-09T10:50:26.401 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-09T10:50:26.409 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-09T10:50:26.415 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-09T10:50:26.415 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-09T10:50:26.417 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-09T10:50:26.417 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-09T10:50:26.420 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-09T10:50:26.420 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-09T10:50:26.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.572+0000 7f7675e38640 1 Processor -- start 2024-05-09T10:50:26.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.572+0000 7f7675e38640 1 -- start start 2024-05-09T10:50:26.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.574+0000 7f7675e38640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7670071b00 0x7f767006e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.574+0000 7f7675e38640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f767006eb70 0x7f76700af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.574+0000 7f7675e38640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76701553e0 con 0x7f7670155000 2024-05-09T10:50:26.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.574+0000 7f7675e38640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f767006f000 con 0x7f7670071b00 2024-05-09T10:50:26.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.574+0000 7f7675e38640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76700b4430 con 0x7f767006eb70 2024-05-09T10:50:26.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.574+0000 7f766ffff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f767006eb70 0x7f76700af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.574+0000 7f766ffff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f767006eb70 0x7f76700af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:53344/0 (socket says 172.21.15.98:53344) 2024-05-09T10:50:26.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.574+0000 7f766ffff640 1 -- 172.21.15.98:0/4024585675 learned_addr learned my addr 172.21.15.98:0/4024585675 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:26.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.575+0000 7f766effd640 1 --2- 172.21.15.98:0/4024585675 >> v2:172.21.15.98:3300/0 conn(0x7f7670071b00 0x7f767006e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.575+0000 7f766ffff640 1 -- 172.21.15.98:0/4024585675 >> v2:172.21.15.98:3300/0 conn(0x7f7670071b00 msgr2=0x7f767006e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:26.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.575+0000 7f766ffff640 1 --2- 172.21.15.98:0/4024585675 >> v2:172.21.15.98:3300/0 conn(0x7f7670071b00 0x7f767006e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.575+0000 7f766ffff640 1 -- 172.21.15.98:0/4024585675 >> v1:172.21.15.98:6789/0 conn(0x7f7670155000 legacy=0x7f76700713f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T10:50:26.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.575+0000 7f766ffff640 1 -- 172.21.15.98:0/4024585675 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76700b3860 con 0x7f767006eb70 2024-05-09T10:50:26.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.576+0000 7f766effd640 1 --2- 172.21.15.98:0/4024585675 >> v2:172.21.15.98:3300/0 conn(0x7f7670071b00 0x7f767006e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:50:26.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.577+0000 7f766ffff640 1 --2- 172.21.15.98:0/4024585675 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f767006eb70 0x7f76700af4d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f76700b45b0 tx=0x7f766802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36c9091135c14f0e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.578+0000 7f766e7fc640 1 -- 172.21.15.98:0/4024585675 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7668036070 con 0x7f767006eb70 2024-05-09T10:50:26.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.578+0000 7f766e7fc640 1 -- 172.21.15.98:0/4024585675 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7668031600 con 0x7f767006eb70 2024-05-09T10:50:26.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.579+0000 7f7675e38640 1 -- 172.21.15.98:0/4024585675 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f767006eb70 msgr2=0x7f76700af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.579+0000 7f7675e38640 1 --2- 172.21.15.98:0/4024585675 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f767006eb70 0x7f76700af4d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f76700b45b0 tx=0x7f766802a6d0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.579+0000 7f7675e38640 1 -- 172.21.15.98:0/4024585675 shutdown_connections 2024-05-09T10:50:26.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.579+0000 7f7675e38640 1 --2- 172.21.15.98:0/4024585675 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f767006eb70 0x7f76700af4d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.579+0000 7f7675e38640 1 --2- 172.21.15.98:0/4024585675 >> v2:172.21.15.98:3300/0 conn(0x7f7670071b00 0x7f767006e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.580+0000 7f7675e38640 1 -- 172.21.15.98:0/4024585675 >> 172.21.15.98:0/4024585675 conn(0x7f76700b24c0 msgr2=0x7f76700b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:26.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.580+0000 7f7675e38640 1 -- 172.21.15.98:0/4024585675 shutdown_connections 2024-05-09T10:50:26.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.580+0000 7f7675e38640 1 -- 172.21.15.98:0/4024585675 wait complete. 2024-05-09T10:50:26.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.582+0000 7f7675e38640 1 Processor -- start 2024-05-09T10:50:26.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.582+0000 7f7675e38640 1 -- start start 2024-05-09T10:50:26.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.584+0000 7f7675e38640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7670071b00 0x7f7670154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.584+0000 7f7675e38640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7670155000 0x7f76701fa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.584+0000 7f766effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7670155000 0x7f76701fa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.584+0000 7f766effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7670155000 0x7f76701fa6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42140/0 (socket says 172.21.15.98:42140) 2024-05-09T10:50:26.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.584+0000 7f766f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7670071b00 0x7f7670154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.584+0000 7f766f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7670071b00 0x7f7670154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.98:35330/0 (socket says 172.21.15.98:35330) 2024-05-09T10:50:26.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.585+0000 7f766effd640 1 -- 172.21.15.98:0/2847156261 learned_addr learned my addr 172.21.15.98:0/2847156261 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:26.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.585+0000 7f7675e38640 1 --2- 172.21.15.98:0/2847156261 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76701f8c00 0x7f76700cec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.585+0000 7f7675e38640 1 -- 172.21.15.98:0/2847156261 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76700b40c0 con 0x7f7670155000 2024-05-09T10:50:26.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.585+0000 7f7675e38640 1 -- 172.21.15.98:0/2847156261 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76700b3de0 con 0x7f7670071b00 2024-05-09T10:50:26.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.585+0000 7f766ffff640 1 --2- 172.21.15.98:0/2847156261 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76701f8c00 0x7f76700cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.585+0000 7f7675e38640 1 -- 172.21.15.98:0/2847156261 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767015bdd0 con 0x7f76701f8c00 2024-05-09T10:50:26.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.586+0000 7f766effd640 1 -- 172.21.15.98:0/2847156261 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76701f8c00 msgr2=0x7f76700cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.586+0000 7f766effd640 1 --2- 172.21.15.98:0/2847156261 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76701f8c00 0x7f76700cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.586+0000 7f766effd640 1 -- 172.21.15.98:0/2847156261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7670071b00 msgr2=0x7f7670154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.586+0000 7f766effd640 1 --2- 172.21.15.98:0/2847156261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7670071b00 0x7f7670154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.586+0000 7f766effd640 1 -- 172.21.15.98:0/2847156261 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76701546c0 con 0x7f7670155000 2024-05-09T10:50:26.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.586+0000 7f766effd640 1 --2- 172.21.15.98:0/2847156261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7670155000 0x7f76701fa6f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f766000eca0 tx=0x7f766000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.587+0000 7f7657fff640 1 -- 172.21.15.98:0/2847156261 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7660017070 con 0x7f7670155000 2024-05-09T10:50:26.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.587+0000 7f7675e38640 1 -- 172.21.15.98:0/2847156261 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76701f9650 con 0x7f7670155000 2024-05-09T10:50:26.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.587+0000 7f7675e38640 1 -- 172.21.15.98:0/2847156261 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76701f9c10 con 0x7f7670155000 2024-05-09T10:50:26.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.590+0000 7f7657fff640 1 -- 172.21.15.98:0/2847156261 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766000cc30 con 0x7f7670155000 2024-05-09T10:50:26.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.590+0000 7f7657fff640 1 -- 172.21.15.98:0/2847156261 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7660013470 con 0x7f7670155000 2024-05-09T10:50:26.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.591+0000 7f7657fff640 1 -- 172.21.15.98:0/2847156261 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f766001a070 con 0x7f7670155000 2024-05-09T10:50:26.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.592+0000 7f7657fff640 1 --2- 172.21.15.98:0/2847156261 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f765c06b5d0 0x7f765c06da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.593+0000 7f766f7fe640 1 --2- 172.21.15.98:0/2847156261 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f765c06b5d0 0x7f765c06da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.593+0000 7f766f7fe640 1 --2- 172.21.15.98:0/2847156261 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f765c06b5d0 0x7f765c06da90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7664006fd0 tx=0x7f7664008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.594+0000 7f7657fff640 1 -- 172.21.15.98:0/2847156261 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f766008a1c0 con 0x7f7670155000 2024-05-09T10:50:26.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.595+0000 7f7675e38640 1 -- 172.21.15.98:0/2847156261 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7650005290 con 0x7f7670155000 2024-05-09T10:50:26.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.604+0000 7f7657fff640 1 -- 172.21.15.98:0/2847156261 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f76701546c0 con 0x7f7670155000 2024-05-09T10:50:26.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.630+0000 7f3e1096f640 1 Processor -- start 2024-05-09T10:50:26.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.630+0000 7f3e1096f640 1 -- start start 2024-05-09T10:50:26.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.630+0000 7f1c5d09a640 1 Processor -- start 2024-05-09T10:50:26.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.631+0000 7f914c84d640 1 Processor -- start 2024-05-09T10:50:26.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.631+0000 7f51125c8640 1 Processor -- start 2024-05-09T10:50:26.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.631+0000 7f0a3c13b640 1 Processor -- start 2024-05-09T10:50:26.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.631+0000 7f51125c8640 1 -- start start 2024-05-09T10:50:26.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.631+0000 7f0a3c13b640 1 -- start start 2024-05-09T10:50:26.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.631+0000 7f3153cf7640 1 Processor -- start 2024-05-09T10:50:26.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.632+0000 7ff299add640 1 Processor -- start 2024-05-09T10:50:26.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.632+0000 7ff299add640 1 -- start start 2024-05-09T10:50:26.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.632+0000 7f3e1096f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3e040e12a0 0x7f3e040534e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.632+0000 7f3e1096f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3e040e4d20 0x7f3e04053a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.631+0000 7f1c5d09a640 1 -- start start 2024-05-09T10:50:26.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.632+0000 7f51125c8640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f510c155000 0x7f510c0713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.632+0000 7f51125c8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f510c06ed40 0x7f510c0af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.632+0000 7f51125c8640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f510c0b3860 con 0x7f510c071930 2024-05-09T10:50:26.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.631+0000 7f914c84d640 1 -- start start 2024-05-09T10:50:26.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.632+0000 7f914c84d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f914806c0a0 0x7f914815e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.632+0000 7f51125c8640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f510c071d90 con 0x7f510c155000 2024-05-09T10:50:26.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.632+0000 7f51125c8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f510c0b4430 con 0x7f510c06ed40 2024-05-09T10:50:26.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.632+0000 7f3e0a59c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3e04054b40 0x7f3e040e1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.98:41286/0 (socket says 172.21.15.98:41286) 2024-05-09T10:50:26.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.632+0000 7f3e0ad9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3e040e12a0 0x7f3e040534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.632+0000 7f3e0ad9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3e040e12a0 0x7f3e040534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42148/0 (socket says 172.21.15.98:42148) 2024-05-09T10:50:26.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.632+0000 7f1c5d09a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1c580d05d0 0x7f1c580b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f914c84d640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f914806c510 con 0x7f914806bbf0 2024-05-09T10:50:26.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f1c56d9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f1c58165550 0x7f1c580cfec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.98:41300/0 (socket says 172.21.15.98:41300) 2024-05-09T10:50:26.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f0a3c13b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a3415dea0 0x7f0a340ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f914c84d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91480b5140 con 0x7f914815a190 2024-05-09T10:50:26.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f914c84d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914815a990 con 0x7f914806c0a0 2024-05-09T10:50:26.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f510bfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f510c155000 0x7f510c0713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f1c5d09a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1c58165930 0x7f1c580b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f1c56d9d640 1 -- 172.21.15.98:0/3055867303 learned_addr learned my addr 172.21.15.98:0/3055867303 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:26.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f913ffff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f914806bbf0 0x7f914815dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56022/0 (socket says 172.21.15.98:56022) 2024-05-09T10:50:26.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f913ffff640 1 -- 172.21.15.98:0/1210436219 learned_addr learned my addr 172.21.15.98:0/1210436219 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:26.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f1c5d09a640 1 -- 172.21.15.98:0/3055867303 --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c580acd40 con 0x7f1c58165550 2024-05-09T10:50:26.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f0a3c13b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0a340d1350 0x7f0a340b72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f510bfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f510c155000 0x7f510c0713f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42164/0 (socket says 172.21.15.98:42164) 2024-05-09T10:50:26.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f1c5d09a640 1 -- 172.21.15.98:0/3055867303 --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c580b2f40 con 0x7f1c58165930 2024-05-09T10:50:26.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f1c5d09a640 1 -- 172.21.15.98:0/3055867303 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c58166b60 con 0x7f1c580d05d0 2024-05-09T10:50:26.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f5110b65640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f510c06ed40 0x7f510c0af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f0a396d6640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0a340cece0 0x7f0a340cf120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.98:41310/0 (socket says 172.21.15.98:41310) 2024-05-09T10:50:26.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f0a39ed7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a3415dea0 0x7f0a340ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f0a39ed7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a3415dea0 0x7f0a340ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:53364/0 (socket says 172.21.15.98:53364) 2024-05-09T10:50:26.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f0a39ed7640 1 -- 172.21.15.98:0/2701303169 learned_addr learned my addr 172.21.15.98:0/2701303169 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:26.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7ff299add640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff2940cf050 0x7ff29415a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f5110b65640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f510c06ed40 0x7f510c0af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.98:35338/0 (socket says 172.21.15.98:35338) 2024-05-09T10:50:26.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f5110b65640 1 -- 172.21.15.98:0/1183523683 learned_addr learned my addr 172.21.15.98:0/1183523683 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:26.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.632+0000 7f3153cf7640 1 -- start start 2024-05-09T10:50:26.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f3153cf7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f314403b0e0 0x7f31440526b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f9146d9d640 1 --2- 172.21.15.98:0/1210436219 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f914806c0a0 0x7f914815e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f0a3c13b640 1 -- 172.21.15.98:0/2701303169 --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a340d25c0 con 0x7f0a340cece0 2024-05-09T10:50:26.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7ff299add640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2940cf430 0x7ff2940cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f3151a93640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f31440eaca0 0x7f31440483f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.98:41326/0 (socket says 172.21.15.98:41326) 2024-05-09T10:50:26.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f0a3c13b640 1 -- 172.21.15.98:0/2701303169 --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a340d2b80 con 0x7f0a340d1350 2024-05-09T10:50:26.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f0a3c13b640 1 -- 172.21.15.98:0/2701303169 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a340d28a0 con 0x7f0a3415dea0 2024-05-09T10:50:26.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7ff292ffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff29406dac0 0x7ff294071400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.98:41334/0 (socket says 172.21.15.98:41334) 2024-05-09T10:50:26.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7ff299add640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff29415a950 con 0x7ff29406dac0 2024-05-09T10:50:26.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7ff292ffd640 1 -- 172.21.15.98:0/3887697759 learned_addr learned my addr 172.21.15.98:0/3887697759 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:26.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7ff299add640 1 -- 172.21.15.98:0/3887697759 --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff29415ac30 con 0x7ff2940cf050 2024-05-09T10:50:26.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7ff299add640 1 -- 172.21.15.98:0/3887697759 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29415af10 con 0x7ff2940cf430 2024-05-09T10:50:26.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f3153cf7640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f314403a320 con 0x7f3144048b00 2024-05-09T10:50:26.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f3151292640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f3144048b00 0x7f31440537d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56038/0 (socket says 172.21.15.98:56038) 2024-05-09T10:50:26.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f3151292640 1 -- 172.21.15.98:0/2966189059 learned_addr learned my addr 172.21.15.98:0/2966189059 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:26.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f3153cf7640 1 -- 172.21.15.98:0/2966189059 --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f314403a600 con 0x7f31440eaca0 2024-05-09T10:50:26.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f3153cf7640 1 -- 172.21.15.98:0/2966189059 --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3144052bf0 con 0x7f314403b0e0 2024-05-09T10:50:26.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f1c5759e640 1 --2- 172.21.15.98:0/3055867303 >> v2:172.21.15.98:3300/0 conn(0x7f1c58165930 0x7f1c580b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.633+0000 7f3e0a59c640 1 -- 172.21.15.98:0/3259113503 learned_addr learned my addr 172.21.15.98:0/3259113503 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:26.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f3e1096f640 1 -- 172.21.15.98:0/3259113503 --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e04054fd0 con 0x7f3e04054b40 2024-05-09T10:50:26.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7ff293fff640 1 --2- 172.21.15.98:0/3887697759 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2940cf430 0x7ff2940cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f5110b65640 1 -- 172.21.15.98:0/1183523683 >> v2:172.21.15.98:3300/0 conn(0x7f510c155000 msgr2=0x7f510c0713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:26.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f5110b65640 1 --2- 172.21.15.98:0/1183523683 >> v2:172.21.15.98:3300/0 conn(0x7f510c155000 0x7f510c0713f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f5110b65640 1 -- 172.21.15.98:0/1183523683 >> v1:172.21.15.98:6789/0 conn(0x7f510c071930 legacy=0x7f510c06e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:26.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f1c55d9b640 1 -- 172.21.15.98:0/3055867303 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3136693039 0 0) 0x7f1c580acd40 con 0x7f1c58165550 2024-05-09T10:50:26.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f3e1096f640 1 -- 172.21.15.98:0/3259113503 --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e040552b0 con 0x7f3e040e12a0 2024-05-09T10:50:26.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f1c5659c640 1 --2- 172.21.15.98:0/3055867303 >> v2:172.21.15.105:3300/0 conn(0x7f1c580d05d0 0x7f1c580b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f5110b65640 1 -- 172.21.15.98:0/1183523683 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510c1553e0 con 0x7f510c06ed40 2024-05-09T10:50:26.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f3e1096f640 1 -- 172.21.15.98:0/3259113503 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e04055590 con 0x7f3e040e4d20 2024-05-09T10:50:26.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f1c55d9b640 1 -- 172.21.15.98:0/3055867303 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c40003520 con 0x7f1c58165550 2024-05-09T10:50:26.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f0a38ed5640 1 -- 172.21.15.98:0/2701303169 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3013356660 0 0) 0x7f0a340d25c0 con 0x7f0a340cece0 2024-05-09T10:50:26.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f0a38ed5640 1 -- 172.21.15.98:0/2701303169 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a20003520 con 0x7f0a340cece0 2024-05-09T10:50:26.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f0a39ed7640 1 -- 172.21.15.98:0/2701303169 >> v2:172.21.15.98:3300/0 conn(0x7f0a340d1350 msgr2=0x7f0a340b72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:26.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f0a39ed7640 1 --2- 172.21.15.98:0/2701303169 >> v2:172.21.15.98:3300/0 conn(0x7f0a340d1350 0x7f0a340b72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f3e0b59e640 1 --2- 172.21.15.98:0/3259113503 >> v2:172.21.15.105:3300/0 conn(0x7f3e040e4d20 0x7f3e04053a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f0a39ed7640 1 -- 172.21.15.98:0/2701303169 >> v1:172.21.15.105:6789/0 conn(0x7f0a340cece0 legacy=0x7f0a340cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f1c55d9b640 1 -- 172.21.15.98:0/3055867303 <== mon.1 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 614964339 0 0) 0x7f1c40003520 con 0x7f1c58165550 2024-05-09T10:50:26.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f0a39ed7640 1 -- 172.21.15.98:0/2701303169 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a340b6af0 con 0x7f0a3415dea0 2024-05-09T10:50:26.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f1c55d9b640 1 -- 172.21.15.98:0/3055867303 >> v2:172.21.15.105:3300/0 conn(0x7f1c580d05d0 msgr2=0x7f1c580b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:26.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f1c55d9b640 1 --2- 172.21.15.98:0/3055867303 >> v2:172.21.15.105:3300/0 conn(0x7f1c580d05d0 0x7f1c580b46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f9145d9b640 1 -- 172.21.15.98:0/1210436219 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3374860141 0 0) 0x7f91480b5140 con 0x7f914815a190 2024-05-09T10:50:26.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f9145d9b640 1 -- 172.21.15.98:0/1210436219 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9128005730 con 0x7f914815a190 2024-05-09T10:50:26.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f9145d9b640 1 -- 172.21.15.98:0/1210436219 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2464849010 0 0) 0x7f914806c510 con 0x7f914806bbf0 2024-05-09T10:50:26.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f3e09d9b640 1 -- 172.21.15.98:0/3259113503 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1439767005 0 0) 0x7f3e04054fd0 con 0x7f3e04054b40 2024-05-09T10:50:26.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f9145d9b640 1 -- 172.21.15.98:0/1210436219 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91480b5140 con 0x7f914806bbf0 2024-05-09T10:50:26.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f3e09d9b640 1 -- 172.21.15.98:0/3259113503 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3df4003520 con 0x7f3e04054b40 2024-05-09T10:50:26.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f1c55d9b640 1 -- 172.21.15.98:0/3055867303 >> v2:172.21.15.98:3300/0 conn(0x7f1c58165930 msgr2=0x7f1c580b4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:26.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f1c55d9b640 1 --2- 172.21.15.98:0/3055867303 >> v2:172.21.15.98:3300/0 conn(0x7f1c58165930 0x7f1c580b4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f1c55d9b640 1 -- 172.21.15.98:0/3055867303 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c58165f50 con 0x7f1c58165550 2024-05-09T10:50:26.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f3152294640 1 --2- 172.21.15.98:0/2966189059 >> v2:172.21.15.98:3300/0 conn(0x7f314403b0e0 0x7f31440526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f3150a91640 1 -- 172.21.15.98:0/2966189059 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 948509776 0 0) 0x7f314403a320 con 0x7f3144048b00 2024-05-09T10:50:26.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f3150a91640 1 -- 172.21.15.98:0/2966189059 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3138003520 con 0x7f3144048b00 2024-05-09T10:50:26.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f1c55d9b640 1 -- 172.21.15.98:0/3055867303 <== mon.1 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f1c48001670 con 0x7f1c58165550 2024-05-09T10:50:26.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f1c55d9b640 1 -- 172.21.15.98:0/3055867303 >> v1:172.21.15.105:6789/0 conn(0x7f1c58165550 legacy=0x7f1c580cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.635+0000 7f3e0b59e640 1 -- 172.21.15.98:0/3259113503 >> v2:172.21.15.98:3300/0 conn(0x7f3e040e12a0 msgr2=0x7f3e040534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:26.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.635+0000 7f3e0b59e640 1 --2- 172.21.15.98:0/3259113503 >> v2:172.21.15.98:3300/0 conn(0x7f3e040e12a0 0x7f3e040534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.635+0000 7f3e0b59e640 1 -- 172.21.15.98:0/3259113503 >> v1:172.21.15.105:6789/0 conn(0x7f3e04054b40 legacy=0x7f3e040e1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f1c55d9b640 1 --2- 172.21.15.98:0/3055867303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c400039b0 0x7f1c40003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.635+0000 7f3e0b59e640 1 -- 172.21.15.98:0/3259113503 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e04055870 con 0x7f3e040e4d20 2024-05-09T10:50:26.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f1c55d9b640 1 --2- 172.21.15.98:0/3055867303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c40004800 0x7f1c40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f1c55d9b640 1 --2- 172.21.15.98:0/3055867303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1c400051c0 0x7f1c4000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f1c55d9b640 1 -- 172.21.15.98:0/3055867303 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c58166b60 con 0x7f1c40004800 2024-05-09T10:50:26.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f1c55d9b640 1 -- 172.21.15.98:0/3055867303 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4000bf80 con 0x7f1c400039b0 2024-05-09T10:50:26.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.635+0000 7f3e0ad9d640 1 --2- 172.21.15.98:0/3259113503 >> v2:172.21.15.98:3300/0 conn(0x7f3e040e12a0 0x7f3e040534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:26.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f1c55d9b640 1 -- 172.21.15.98:0/3055867303 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4000c180 con 0x7f1c400051c0 2024-05-09T10:50:26.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7ff2927fc640 1 -- 172.21.15.98:0/3887697759 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1694852240 0 0) 0x7ff29415a950 con 0x7ff29406dac0 2024-05-09T10:50:26.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7ff2927fc640 1 -- 172.21.15.98:0/3887697759 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff27c003520 con 0x7ff29406dac0 2024-05-09T10:50:26.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7ff2937fe640 1 --2- 172.21.15.98:0/3887697759 >> v2:172.21.15.98:3300/0 conn(0x7ff2940cf050 0x7ff29415a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7ff2937fe640 1 -- 172.21.15.98:0/3887697759 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2940cf430 msgr2=0x7ff2940cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:26.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7ff2937fe640 1 --2- 172.21.15.98:0/3887697759 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2940cf430 0x7ff2940cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7ff2937fe640 1 -- 172.21.15.98:0/3887697759 >> v1:172.21.15.105:6789/0 conn(0x7ff29406dac0 legacy=0x7ff294071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7ff2937fe640 1 -- 172.21.15.98:0/3887697759 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29415b1f0 con 0x7ff2940cf050 2024-05-09T10:50:26.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.635+0000 7f1c5759e640 1 --2- 172.21.15.98:0/3055867303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1c400051c0 0x7f1c4000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.635+0000 7f3150a91640 1 -- 172.21.15.98:0/2966189059 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191500199 0 0) 0x7f314403a600 con 0x7f31440eaca0 2024-05-09T10:50:26.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.635+0000 7f3150a91640 1 -- 172.21.15.98:0/2966189059 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f314403a320 con 0x7f31440eaca0 2024-05-09T10:50:26.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.635+0000 7f5110b65640 1 --2- 172.21.15.98:0/1183523683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f510c06ed40 0x7f510c0af4d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f510c0b3440 tx=0x7f510402a6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba73d5662bb1ef93 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.635+0000 7f1c56d9d640 1 --2- 172.21.15.98:0/3055867303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c400039b0 0x7f1c40003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.635+0000 7f0a39ed7640 1 --2- 172.21.15.98:0/2701303169 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a3415dea0 0x7f0a340ce7a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0a30009870 tx=0x7f0a3002a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c0402dcc6a16047 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.635+0000 7f1c5759e640 1 -- 172.21.15.98:0/3055867303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c400039b0 msgr2=0x7f1c40003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:26.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.635+0000 7f1c5759e640 1 --2- 172.21.15.98:0/3055867303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c400039b0 0x7f1c40003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7ff2937fe640 1 --2- 172.21.15.98:0/3887697759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2940cf050 0x7ff29415a370 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff2840097a0 tx=0x7ff28402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110a66ca3f9ba92b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f510b7fe640 1 -- 172.21.15.98:0/1183523683 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5104036070 con 0x7f510c06ed40 2024-05-09T10:50:26.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f510b7fe640 1 -- 172.21.15.98:0/1183523683 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5104031600 con 0x7f510c06ed40 2024-05-09T10:50:26.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.635+0000 7f3150a91640 1 -- 172.21.15.98:0/2966189059 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 217151040 0 0) 0x7f3138003520 con 0x7f3144048b00 2024-05-09T10:50:26.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.635+0000 7f3150a91640 1 -- 172.21.15.98:0/2966189059 >> v2:172.21.15.98:3300/0 conn(0x7f314403b0e0 msgr2=0x7f31440526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:26.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.635+0000 7f3150a91640 1 --2- 172.21.15.98:0/2966189059 >> v2:172.21.15.98:3300/0 conn(0x7f314403b0e0 0x7f31440526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f3150a91640 1 -- 172.21.15.98:0/2966189059 >> v1:172.21.15.105:6789/0 conn(0x7f31440eaca0 legacy=0x7f31440483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f3150a91640 1 -- 172.21.15.98:0/2966189059 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314403a8e0 con 0x7f3144048b00 2024-05-09T10:50:26.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f3150a91640 1 -- 172.21.15.98:0/2966189059 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f314c04f440 con 0x7f3144048b00 2024-05-09T10:50:26.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f3150a91640 1 -- 172.21.15.98:0/2966189059 >> v1:172.21.15.98:6789/0 conn(0x7f3144048b00 legacy=0x7f31440537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f9146d9d640 1 -- 172.21.15.98:0/1210436219 >> v1:172.21.15.105:6789/0 conn(0x7f914815a190 legacy=0x7f91480b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.634+0000 7f9146d9d640 1 -- 172.21.15.98:0/1210436219 >> v1:172.21.15.98:6789/0 conn(0x7f914806bbf0 legacy=0x7f914815dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.635+0000 7f9146d9d640 1 -- 172.21.15.98:0/1210436219 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914815ac70 con 0x7f914806c0a0 2024-05-09T10:50:26.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f9146d9d640 1 --2- 172.21.15.98:0/1210436219 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f914806c0a0 0x7f914815e3f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9140009cb0 tx=0x7f914002a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4da17bd81311be8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f3150a91640 1 --2- 172.21.15.98:0/2966189059 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3138003b40 0x7f3138003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f3150a91640 1 --2- 172.21.15.98:0/2966189059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3138004800 0x7f3138004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f3150a91640 1 --2- 172.21.15.98:0/2966189059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31380051c0 0x7f313800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f3150a91640 1 -- 172.21.15.98:0/2966189059 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313800c030 con 0x7f3138004800 2024-05-09T10:50:26.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f3150a91640 1 -- 172.21.15.98:0/2966189059 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313800c1e0 con 0x7f31380051c0 2024-05-09T10:50:26.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f3150a91640 1 -- 172.21.15.98:0/2966189059 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f313800c3e0 con 0x7f3138003b40 2024-05-09T10:50:26.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f0a38ed5640 1 -- 172.21.15.98:0/2701303169 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a30037070 con 0x7f0a3415dea0 2024-05-09T10:50:26.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f0a38ed5640 1 -- 172.21.15.98:0/2701303169 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3002ae50 con 0x7f0a3415dea0 2024-05-09T10:50:26.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.635+0000 7f1c5759e640 1 -- 172.21.15.98:0/3055867303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c40004800 msgr2=0x7f1c40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:26.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f0a38ed5640 1 -- 172.21.15.98:0/2701303169 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a30037070 con 0x7f0a3415dea0 2024-05-09T10:50:26.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f1c5659c640 1 --2- 172.21.15.98:0/3055867303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c40004800 0x7f1c40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f1c5759e640 1 --2- 172.21.15.98:0/3055867303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c40004800 0x7f1c40004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f510b7fe640 1 -- 172.21.15.98:0/1183523683 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5104036070 con 0x7f510c06ed40 2024-05-09T10:50:26.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f1c5759e640 1 -- 172.21.15.98:0/3055867303 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c58165f50 con 0x7f1c400051c0 2024-05-09T10:50:26.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f1c5659c640 1 --2- 172.21.15.98:0/3055867303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c40004800 0x7f1c40004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:26.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7ff2927fc640 1 -- 172.21.15.98:0/3887697759 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284037070 con 0x7ff2940cf050 2024-05-09T10:50:26.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f9145d9b640 1 -- 172.21.15.98:0/1210436219 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9140035020 con 0x7f914806c0a0 2024-05-09T10:50:26.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f9145d9b640 1 -- 172.21.15.98:0/1210436219 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9140007540 con 0x7f914806c0a0 2024-05-09T10:50:26.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f9145d9b640 1 -- 172.21.15.98:0/1210436219 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9130001670 con 0x7f914806c0a0 2024-05-09T10:50:26.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7ff2927fc640 1 -- 172.21.15.98:0/3887697759 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff284032440 con 0x7ff2940cf050 2024-05-09T10:50:26.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7ff2927fc640 1 -- 172.21.15.98:0/3887697759 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284037070 con 0x7ff2940cf050 2024-05-09T10:50:26.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f3e0b59e640 1 --2- 172.21.15.98:0/3259113503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e040e4d20 0x7f3e04053a20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400bfa0 tx=0x7f3dfc02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=717e3bea67280df2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.637+0000 7f51125c8640 1 -- 172.21.15.98:0/1183523683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f510c06ed40 msgr2=0x7f510c0af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.637+0000 7f3e09d9b640 1 -- 172.21.15.98:0/3259113503 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc037070 con 0x7f3e040e4d20 2024-05-09T10:50:26.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.637+0000 7f51125c8640 1 --2- 172.21.15.98:0/1183523683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f510c06ed40 0x7f510c0af4d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f510c0b3440 tx=0x7f510402a6d0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.637+0000 7f3e09d9b640 1 -- 172.21.15.98:0/3259113503 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dfc032440 con 0x7f3e040e4d20 2024-05-09T10:50:26.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.637+0000 7f1c5759e640 1 -- 172.21.15.98:0/3055867303 reap_dead start 2024-05-09T10:50:26.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.637+0000 7f3e09d9b640 1 -- 172.21.15.98:0/3259113503 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc032810 con 0x7f3e040e4d20 2024-05-09T10:50:26.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.637+0000 7f0a3c13b640 1 -- 172.21.15.98:0/2701303169 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a3415dea0 msgr2=0x7f0a340ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.637+0000 7f0a3c13b640 1 --2- 172.21.15.98:0/2701303169 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a3415dea0 0x7f0a340ce7a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0a30009870 tx=0x7f0a3002a300 comp rx=0 tx=0).stop 2024-05-09T10:50:26.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f3151a93640 1 --2- 172.21.15.98:0/2966189059 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3138003b40 0x7f3138003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f3152294640 1 --2- 172.21.15.98:0/2966189059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31380051c0 0x7f313800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f3151292640 1 --2- 172.21.15.98:0/2966189059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3138004800 0x7f3138004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f3151a93640 1 -- 172.21.15.98:0/2966189059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31380051c0 msgr2=0x7f313800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:26.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f3151a93640 1 --2- 172.21.15.98:0/2966189059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31380051c0 0x7f313800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f3151a93640 1 -- 172.21.15.98:0/2966189059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3138004800 msgr2=0x7f3138004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:26.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f3151a93640 1 --2- 172.21.15.98:0/2966189059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3138004800 0x7f3138004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f3152294640 1 -- 172.21.15.98:0/2966189059 reap_dead start 2024-05-09T10:50:26.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.636+0000 7f3151a93640 1 -- 172.21.15.98:0/2966189059 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3138003520 con 0x7f3138003b40 2024-05-09T10:50:26.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.637+0000 7f51125c8640 1 -- 172.21.15.98:0/1183523683 shutdown_connections 2024-05-09T10:50:26.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.637+0000 7f51125c8640 1 --2- 172.21.15.98:0/1183523683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f510c06ed40 0x7f510c0af4d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.637+0000 7f51125c8640 1 --2- 172.21.15.98:0/1183523683 >> v2:172.21.15.98:3300/0 conn(0x7f510c155000 0x7f510c0713f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.637+0000 7f51125c8640 1 -- 172.21.15.98:0/1183523683 >> 172.21.15.98:0/1183523683 conn(0x7f510c0b24c0 msgr2=0x7f510c0b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:26.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.637+0000 7f0a3c13b640 1 -- 172.21.15.98:0/2701303169 shutdown_connections 2024-05-09T10:50:26.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.637+0000 7f0a3c13b640 1 --2- 172.21.15.98:0/2701303169 >> v2:172.21.15.98:3300/0 conn(0x7f0a340d1350 0x7f0a340b72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.637+0000 7f0a3c13b640 1 --2- 172.21.15.98:0/2701303169 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a3415dea0 0x7f0a340ce7a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.637+0000 7f0a3c13b640 1 -- 172.21.15.98:0/2701303169 >> 172.21.15.98:0/2701303169 conn(0x7f0a3409cee0 msgr2=0x7f0a340d3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:26.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.637+0000 7f51125c8640 1 -- 172.21.15.98:0/1183523683 shutdown_connections 2024-05-09T10:50:26.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.637+0000 7f914c84d640 1 -- 172.21.15.98:0/1210436219 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f914806c0a0 msgr2=0x7f914815e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.637+0000 7f914c84d640 1 --2- 172.21.15.98:0/1210436219 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f914806c0a0 0x7f914815e3f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9140009cb0 tx=0x7f914002a640 comp rx=0 tx=0).stop 2024-05-09T10:50:26.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.637+0000 7ff299add640 1 -- 172.21.15.98:0/3887697759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2940cf050 msgr2=0x7ff29415a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.637+0000 7f51125c8640 1 -- 172.21.15.98:0/1183523683 wait complete. 2024-05-09T10:50:26.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7ff299add640 1 --2- 172.21.15.98:0/3887697759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2940cf050 0x7ff29415a370 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff2840097a0 tx=0x7ff28402a180 comp rx=0 tx=0).stop 2024-05-09T10:50:26.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7f0a3c13b640 1 -- 172.21.15.98:0/2701303169 shutdown_connections 2024-05-09T10:50:26.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7f914c84d640 1 -- 172.21.15.98:0/1210436219 shutdown_connections 2024-05-09T10:50:26.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7f1c5759e640 1 --2- 172.21.15.98:0/3055867303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1c400051c0 0x7f1c4000b9c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009f90 tx=0x7f1c40005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b734a24ae462492 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7f0a3c13b640 1 -- 172.21.15.98:0/2701303169 wait complete. 2024-05-09T10:50:26.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7f914c84d640 1 --2- 172.21.15.98:0/1210436219 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f914806c0a0 0x7f914815e3f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7f914c84d640 1 -- 172.21.15.98:0/1210436219 >> 172.21.15.98:0/1210436219 conn(0x7f914806cae0 msgr2=0x7f914806cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:26.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7ff299add640 1 -- 172.21.15.98:0/3887697759 shutdown_connections 2024-05-09T10:50:26.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7ff299add640 1 --2- 172.21.15.98:0/3887697759 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2940cf430 0x7ff2940cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7f3e1096f640 1 -- 172.21.15.98:0/3259113503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e040e4d20 msgr2=0x7f3e04053a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7ff299add640 1 --2- 172.21.15.98:0/3887697759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2940cf050 0x7ff29415a370 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7f3151a93640 1 --2- 172.21.15.98:0/2966189059 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3138003b40 0x7f3138003f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3148001510 tx=0x7f3138005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53e7328e0fb4a3b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7f3e1096f640 1 --2- 172.21.15.98:0/3259113503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e040e4d20 0x7f3e04053a20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400bfa0 tx=0x7f3dfc02a1a0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7ff299add640 1 -- 172.21.15.98:0/3887697759 >> 172.21.15.98:0/3887697759 conn(0x7ff29406abe0 msgr2=0x7ff29406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:26.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7f1c55d9b640 1 -- 172.21.15.98:0/3055867303 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c50035020 con 0x7f1c400051c0 2024-05-09T10:50:26.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7f1c55d9b640 1 -- 172.21.15.98:0/3055867303 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c50003b90 con 0x7f1c400051c0 2024-05-09T10:50:26.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7f1c55d9b640 1 -- 172.21.15.98:0/3055867303 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c50009c10 con 0x7f1c400051c0 2024-05-09T10:50:26.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7f3e1096f640 1 -- 172.21.15.98:0/3259113503 shutdown_connections 2024-05-09T10:50:26.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7f3e1096f640 1 --2- 172.21.15.98:0/3259113503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e040e4d20 0x7f3e04053a20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7f3e1096f640 1 --2- 172.21.15.98:0/3259113503 >> v2:172.21.15.98:3300/0 conn(0x7f3e040e12a0 0x7f3e040534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7f3150a91640 1 -- 172.21.15.98:0/2966189059 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3148001550 con 0x7f3138003b40 2024-05-09T10:50:26.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7f3150a91640 1 -- 172.21.15.98:0/2966189059 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314803f070 con 0x7f3138003b40 2024-05-09T10:50:26.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7f3150a91640 1 -- 172.21.15.98:0/2966189059 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3148007070 con 0x7f3138003b40 2024-05-09T10:50:26.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7f3e1096f640 1 -- 172.21.15.98:0/3259113503 >> 172.21.15.98:0/3259113503 conn(0x7f3e04020030 msgr2=0x7f3e04056d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:26.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7ff299add640 1 -- 172.21.15.98:0/3887697759 shutdown_connections 2024-05-09T10:50:26.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7ff299add640 1 -- 172.21.15.98:0/3887697759 wait complete. 2024-05-09T10:50:26.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7f3e1096f640 1 -- 172.21.15.98:0/3259113503 shutdown_connections 2024-05-09T10:50:26.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7f3e1096f640 1 -- 172.21.15.98:0/3259113503 wait complete. 2024-05-09T10:50:26.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.639+0000 7f1c5d09a640 1 -- 172.21.15.98:0/3055867303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1c400051c0 msgr2=0x7f1c4000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.639+0000 7f1c5d09a640 1 --2- 172.21.15.98:0/3055867303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1c400051c0 0x7f1c4000b9c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009f90 tx=0x7f1c40005830 comp rx=0 tx=0).stop 2024-05-09T10:50:26.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.638+0000 7f914c84d640 1 -- 172.21.15.98:0/1210436219 shutdown_connections 2024-05-09T10:50:26.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.639+0000 7f1c5d09a640 1 -- 172.21.15.98:0/3055867303 shutdown_connections 2024-05-09T10:50:26.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.639+0000 7f1c5d09a640 1 --2- 172.21.15.98:0/3055867303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1c400051c0 0x7f1c4000b9c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.639+0000 7f1c5d09a640 1 -- 172.21.15.98:0/3055867303 >> 172.21.15.98:0/3055867303 conn(0x7f1c5806adc0 msgr2=0x7f1c5806b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:26.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.639+0000 7f914c84d640 1 -- 172.21.15.98:0/1210436219 wait complete. 2024-05-09T10:50:26.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.639+0000 7f3153cf7640 1 -- 172.21.15.98:0/2966189059 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3138003b40 msgr2=0x7f3138003f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.639+0000 7f3153cf7640 1 --2- 172.21.15.98:0/2966189059 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3138003b40 0x7f3138003f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3148001510 tx=0x7f3138005830 comp rx=0 tx=0).stop 2024-05-09T10:50:26.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.639+0000 7f51125c8640 1 Processor -- start 2024-05-09T10:50:26.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.639+0000 7f1c5d09a640 1 -- 172.21.15.98:0/3055867303 shutdown_connections 2024-05-09T10:50:26.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.640+0000 7f0a3c13b640 1 Processor -- start 2024-05-09T10:50:26.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.640+0000 7f0a3c13b640 1 -- start start 2024-05-09T10:50:26.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.639+0000 7f1c5d09a640 1 -- 172.21.15.98:0/3055867303 wait complete. 2024-05-09T10:50:26.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.640+0000 7ff299add640 1 Processor -- start 2024-05-09T10:50:26.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.639+0000 7f51125c8640 1 -- start start 2024-05-09T10:50:26.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.640+0000 7ff299add640 1 -- start start 2024-05-09T10:50:26.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.640+0000 7f51125c8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f510c06ed40 0x7f510c1566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.640+0000 7f51125c8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f510c071930 0x7f510c156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.640+0000 7f51125c8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f510c155000 0x7f510c142ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.640+0000 7f51125c8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510c0b40c0 con 0x7f510c06ed40 2024-05-09T10:50:26.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.640+0000 7f51125c8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510c0b3de0 con 0x7f510c071930 2024-05-09T10:50:26.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.640+0000 7f51125c8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510c15bdd0 con 0x7f510c155000 2024-05-09T10:50:26.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.641+0000 7f3e1096f640 1 Processor -- start 2024-05-09T10:50:26.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.641+0000 7f510bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f510c06ed40 0x7f510c1566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.641+0000 7f3e1096f640 1 -- start start 2024-05-09T10:50:26.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.641+0000 7f914c84d640 1 Processor -- start 2024-05-09T10:50:26.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.641+0000 7f510bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f510c06ed40 0x7f510c1566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42218/0 (socket says 172.21.15.98:42218) 2024-05-09T10:50:26.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.641+0000 7f510bfff640 1 -- 172.21.15.98:0/3599426409 learned_addr learned my addr 172.21.15.98:0/3599426409 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:26.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.641+0000 7f914c84d640 1 -- start start 2024-05-09T10:50:26.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.641+0000 7f0a3c13b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a340cece0 0x7f0a340cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.641+0000 7f0a3c13b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a340d1350 0x7f0a34159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.641+0000 7f0a3c13b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a3415dea0 0x7f0a340cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.641+0000 7f510bfff640 1 -- 172.21.15.98:0/3599426409 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f510c155000 msgr2=0x7f510c142ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.641+0000 7f1c5d09a640 1 Processor -- start 2024-05-09T10:50:26.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.641+0000 7ff299add640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff29406dac0 0x7ff2940cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.641+0000 7ff299add640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2940cf050 0x7ff2940acf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.641+0000 7f5110b65640 1 --2- 172.21.15.98:0/3599426409 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f510c155000 0x7f510c142ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.641+0000 7ff299add640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2940cf430 0x7ff2940aaec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.641+0000 7ff2937fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff29406dac0 0x7ff2940cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7ff299add640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29406df40 con 0x7ff29406dac0 2024-05-09T10:50:26.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7ff299add640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff294071b10 con 0x7ff2940cf430 2024-05-09T10:50:26.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7ff299add640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29406e0c0 con 0x7ff2940cf050 2024-05-09T10:50:26.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7ff2937fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff29406dac0 0x7ff2940cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42238/0 (socket says 172.21.15.98:42238) 2024-05-09T10:50:26.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f510bfff640 1 --2- 172.21.15.98:0/3599426409 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f510c155000 0x7f510c142ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7ff2937fe640 1 -- 172.21.15.98:0/1115161796 learned_addr learned my addr 172.21.15.98:0/1115161796 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:26.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7ff292ffd640 1 --2- 172.21.15.98:0/1115161796 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2940cf050 0x7ff2940acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7ff293fff640 1 --2- 172.21.15.98:0/1115161796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2940cf430 0x7ff2940aaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f510bfff640 1 -- 172.21.15.98:0/3599426409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f510c071930 msgr2=0x7f510c156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f5103fff640 1 --2- 172.21.15.98:0/3599426409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f510c071930 0x7f510c156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f510bfff640 1 --2- 172.21.15.98:0/3599426409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f510c071930 0x7f510c156c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f510bfff640 1 -- 172.21.15.98:0/3599426409 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510c148eb0 con 0x7f510c06ed40 2024-05-09T10:50:26.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.641+0000 7f1c5d09a640 1 -- start start 2024-05-09T10:50:26.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f3153cf7640 1 -- 172.21.15.98:0/2966189059 shutdown_connections 2024-05-09T10:50:26.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f3153cf7640 1 --2- 172.21.15.98:0/2966189059 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3138003b40 0x7f3138003f20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f3153cf7640 1 -- 172.21.15.98:0/2966189059 >> 172.21.15.98:0/2966189059 conn(0x7f314401e6f0 msgr2=0x7f3144055420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:26.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7ff292ffd640 1 -- 172.21.15.98:0/1115161796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2940cf430 msgr2=0x7ff2940aaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f0a3a6d8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a3415dea0 0x7f0a340cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7ff292ffd640 1 --2- 172.21.15.98:0/1115161796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2940cf430 0x7ff2940aaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f3e1096f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e040e12a0 0x7f3e040e3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7ff292ffd640 1 -- 172.21.15.98:0/1115161796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff29406dac0 msgr2=0x7ff2940cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f5110b65640 1 --2- 172.21.15.98:0/3599426409 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f510c155000 0x7f510c142ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:26.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7ff292ffd640 1 --2- 172.21.15.98:0/1115161796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff29406dac0 0x7ff2940cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7ff292ffd640 1 -- 172.21.15.98:0/1115161796 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2940ad9a0 con 0x7ff2940cf050 2024-05-09T10:50:26.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f5103fff640 1 --2- 172.21.15.98:0/3599426409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f510c071930 0x7f510c156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:26.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f3e1096f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e040e4d20 0x7f3e040d8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7ff2937fe640 1 --2- 172.21.15.98:0/1115161796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff29406dac0 0x7ff2940cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:50:26.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f914c84d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f914806bbf0 0x7f914815cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f0a3a6d8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a3415dea0 0x7f0a340cc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42230/0 (socket says 172.21.15.98:42230) 2024-05-09T10:50:26.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f0a396d6640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a340d1350 0x7f0a34159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f0a396d6640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a340d1350 0x7f0a34159a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:53392/0 (socket says 172.21.15.98:53392) 2024-05-09T10:50:26.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7ff292ffd640 1 --2- 172.21.15.98:0/1115161796 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2940cf050 0x7ff2940acf30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff28800c390 tx=0x7ff288001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f0a39ed7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a340cece0 0x7f0a340cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f3153cf7640 1 -- 172.21.15.98:0/2966189059 shutdown_connections 2024-05-09T10:50:26.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f914c84d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f914806c0a0 0x7f91481f32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f0a39ed7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a340cece0 0x7f0a340cacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.98:35398/0 (socket says 172.21.15.98:35398) 2024-05-09T10:50:26.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f914659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f914806bbf0 0x7f914815cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f0a3a6d8640 1 -- 172.21.15.98:0/1771840139 learned_addr learned my addr 172.21.15.98:0/1771840139 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:26.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f3e0ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e040e12a0 0x7f3e040e3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f3e0ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e040e12a0 0x7f3e040e3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.98:35420/0 (socket says 172.21.15.98:35420) 2024-05-09T10:50:26.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f3e0a59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e040e4d20 0x7f3e040d8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f0a3c13b640 1 -- 172.21.15.98:0/1771840139 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a340d2360 con 0x7f0a3415dea0 2024-05-09T10:50:26.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f3e0a59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e040e4d20 0x7f3e040d8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42240/0 (socket says 172.21.15.98:42240) 2024-05-09T10:50:26.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f914659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f914806bbf0 0x7f914815cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.98:35436/0 (socket says 172.21.15.98:35436) 2024-05-09T10:50:26.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f0a3c13b640 1 -- 172.21.15.98:0/1771840139 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a340d21e0 con 0x7f0a340cece0 2024-05-09T10:50:26.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f3e0ad9d640 1 -- 172.21.15.98:0/4280552849 learned_addr learned my addr 172.21.15.98:0/4280552849 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:26.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f914659c640 1 -- 172.21.15.98:0/2713684965 learned_addr learned my addr 172.21.15.98:0/2713684965 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:26.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f0a3c13b640 1 -- 172.21.15.98:0/1771840139 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a340b7830 con 0x7f0a340d1350 2024-05-09T10:50:26.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f3e1096f640 1 --2- 172.21.15.98:0/4280552849 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e040e38b0 0x7f3e040c5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f914c84d640 1 --2- 172.21.15.98:0/2713684965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f914815a190 0x7f91480cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7ff27bfff640 1 -- 172.21.15.98:0/1115161796 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff288017070 con 0x7ff2940cf050 2024-05-09T10:50:26.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f1c5d09a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c400051c0 0x7f1c5815e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7ff27bfff640 1 -- 172.21.15.98:0/1115161796 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff288005ce0 con 0x7ff2940cf050 2024-05-09T10:50:26.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f913ffff640 1 --2- 172.21.15.98:0/2713684965 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f914806c0a0 0x7f91481f32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f914c84d640 1 -- 172.21.15.98:0/2713684965 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91480b4d90 con 0x7f914815a190 2024-05-09T10:50:26.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7ff27bfff640 1 -- 172.21.15.98:0/1115161796 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2880072f0 con 0x7ff2940cf050 2024-05-09T10:50:26.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f914c84d640 1 -- 172.21.15.98:0/2713684965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91480af950 con 0x7f914806bbf0 2024-05-09T10:50:26.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7ff299add640 1 -- 172.21.15.98:0/1115161796 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2940adc80 con 0x7ff2940cf050 2024-05-09T10:50:26.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f914c84d640 1 -- 172.21.15.98:0/2713684965 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914814f500 con 0x7f914806c0a0 2024-05-09T10:50:26.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f1c5d09a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1c5809cd90 0x7f1c5809d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f9146d9d640 1 --2- 172.21.15.98:0/2713684965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f914815a190 0x7f91480cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f1c5d09a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c5809aa80 0x7f1c5809af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f1c56d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c400051c0 0x7f1c5815e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f1c56d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c400051c0 0x7f1c5815e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42258/0 (socket says 172.21.15.98:42258) 2024-05-09T10:50:26.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f0a3a6d8640 1 -- 172.21.15.98:0/1771840139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a340d1350 msgr2=0x7f0a34159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f0a3a6d8640 1 --2- 172.21.15.98:0/1771840139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a340d1350 0x7f0a34159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f0a3a6d8640 1 -- 172.21.15.98:0/1771840139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a340cece0 msgr2=0x7f0a340cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f0a3a6d8640 1 --2- 172.21.15.98:0/1771840139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a340cece0 0x7f0a340cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f1c5d09a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c581667f0 con 0x7f1c400051c0 2024-05-09T10:50:26.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f1c5d09a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c58166510 con 0x7f1c5809aa80 2024-05-09T10:50:26.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f1c5d09a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c580cfec0 con 0x7f1c5809cd90 2024-05-09T10:50:26.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f1c56d9d640 1 -- 172.21.15.98:0/1577331885 learned_addr learned my addr 172.21.15.98:0/1577331885 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:26.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f0a3a6d8640 1 -- 172.21.15.98:0/1771840139 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a340cb420 con 0x7f0a3415dea0 2024-05-09T10:50:26.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f0a39ed7640 1 --2- 172.21.15.98:0/1771840139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a340cece0 0x7f0a340cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:26.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7ff299add640 1 -- 172.21.15.98:0/1115161796 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2940ae240 con 0x7ff2940cf050 2024-05-09T10:50:26.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f1c5759e640 1 --2- 172.21.15.98:0/1577331885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c5809aa80 0x7f1c5809af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f1c5659c640 1 --2- 172.21.15.98:0/1577331885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1c5809cd90 0x7f1c5809d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f3e1096f640 1 -- 172.21.15.98:0/4280552849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e04057810 con 0x7f3e040e4d20 2024-05-09T10:50:26.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f3e1096f640 1 -- 172.21.15.98:0/4280552849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e04057690 con 0x7f3e040e12a0 2024-05-09T10:50:26.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f3e1096f640 1 -- 172.21.15.98:0/4280552849 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e040e1d90 con 0x7f3e040e38b0 2024-05-09T10:50:26.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f3e0b59e640 1 --2- 172.21.15.98:0/4280552849 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e040e38b0 0x7f3e040c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f914659c640 1 -- 172.21.15.98:0/2713684965 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f914806c0a0 msgr2=0x7f91481f32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f914659c640 1 --2- 172.21.15.98:0/2713684965 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f914806c0a0 0x7f91481f32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.643+0000 7f914659c640 1 -- 172.21.15.98:0/2713684965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f914815a190 msgr2=0x7f91480cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.644+0000 7f1c56d9d640 1 -- 172.21.15.98:0/1577331885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1c5809cd90 msgr2=0x7f1c5809d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.644+0000 7f914659c640 1 --2- 172.21.15.98:0/2713684965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f914815a190 0x7f91480cdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.644+0000 7f914659c640 1 -- 172.21.15.98:0/2713684965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914814f750 con 0x7f914806bbf0 2024-05-09T10:50:26.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.644+0000 7f9146d9d640 1 --2- 172.21.15.98:0/2713684965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f914815a190 0x7f91480cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:50:26.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.644+0000 7f1c56d9d640 1 --2- 172.21.15.98:0/1577331885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1c5809cd90 0x7f1c5809d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.644+0000 7f1c56d9d640 1 -- 172.21.15.98:0/1577331885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c5809aa80 msgr2=0x7f1c5809af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.644+0000 7f1c56d9d640 1 --2- 172.21.15.98:0/1577331885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c5809aa80 0x7f1c5809af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.644+0000 7f3e0a59c640 1 -- 172.21.15.98:0/4280552849 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e040e38b0 msgr2=0x7f3e040c5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.644+0000 7f3e0a59c640 1 --2- 172.21.15.98:0/4280552849 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e040e38b0 0x7f3e040c5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.644+0000 7f3e0a59c640 1 -- 172.21.15.98:0/4280552849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e040e12a0 msgr2=0x7f3e040e3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.644+0000 7f3e0a59c640 1 --2- 172.21.15.98:0/4280552849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e040e12a0 0x7f3e040e3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.644+0000 7f1c56d9d640 1 -- 172.21.15.98:0/1577331885 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5809b7d0 con 0x7f1c400051c0 2024-05-09T10:50:26.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.644+0000 7f0a3a6d8640 1 --2- 172.21.15.98:0/1771840139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a3415dea0 0x7f0a340cc4c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00b9e0 tx=0x7f0a2c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.644+0000 7f3e0a59c640 1 -- 172.21.15.98:0/4280552849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e040d7c80 con 0x7f3e040e4d20 2024-05-09T10:50:26.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.644+0000 7f914659c640 1 --2- 172.21.15.98:0/2713684965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f914806bbf0 0x7f914815cc50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f913800c500 tx=0x7f913800c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.642+0000 7f3153cf7640 1 -- 172.21.15.98:0/2966189059 wait complete. 2024-05-09T10:50:26.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.644+0000 7f3153cf7640 1 Processor -- start 2024-05-09T10:50:26.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.644+0000 7f3e0a59c640 1 --2- 172.21.15.98:0/4280552849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e040e4d20 0x7f3e040d8a00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c050190 tx=0x7f3e0c068000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.645+0000 7f913effd640 1 -- 172.21.15.98:0/2713684965 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91380052d0 con 0x7f914806bbf0 2024-05-09T10:50:26.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.645+0000 7f1c56d9d640 1 --2- 172.21.15.98:0/1577331885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c400051c0 0x7f1c5815e070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1c4800d810 tx=0x7f1c4800dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.645+0000 7f913effd640 1 -- 172.21.15.98:0/2713684965 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9138005490 con 0x7f914806bbf0 2024-05-09T10:50:26.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.645+0000 7f913effd640 1 -- 172.21.15.98:0/2713684965 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9138012870 con 0x7f914806bbf0 2024-05-09T10:50:26.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.645+0000 7f914c84d640 1 -- 172.21.15.98:0/2713684965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f914814fa30 con 0x7f914806bbf0 2024-05-09T10:50:26.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.645+0000 7f914c84d640 1 -- 172.21.15.98:0/2713684965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91481f2db0 con 0x7f914806bbf0 2024-05-09T10:50:26.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.645+0000 7ff299add640 1 -- 172.21.15.98:0/1115161796 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff294069cb0 con 0x7ff2940cf050 2024-05-09T10:50:26.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.646+0000 7ff27bfff640 1 -- 172.21.15.98:0/1115161796 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff28801a070 con 0x7ff2940cf050 2024-05-09T10:50:26.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.645+0000 7f3153cf7640 1 -- start start 2024-05-09T10:50:26.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.646+0000 7f3153cf7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3138003b40 0x7f31440cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.646+0000 7f0a2a7fc640 1 -- 172.21.15.98:0/1771840139 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c019070 con 0x7f0a3415dea0 2024-05-09T10:50:26.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.646+0000 7f3153cf7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f31440d80c0 0x7f31440cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.646+0000 7f0a3c13b640 1 -- 172.21.15.98:0/1771840139 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a340cb700 con 0x7f0a3415dea0 2024-05-09T10:50:26.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.646+0000 7f0a3c13b640 1 -- 172.21.15.98:0/1771840139 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a340cbcc0 con 0x7f0a3415dea0 2024-05-09T10:50:26.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.646+0000 7f0a2a7fc640 1 -- 172.21.15.98:0/1771840139 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2c00cb40 con 0x7f0a3415dea0 2024-05-09T10:50:26.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.646+0000 7f914c84d640 1 -- 172.21.15.98:0/2713684965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f914814fc00 con 0x7f914806bbf0 2024-05-09T10:50:26.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.646+0000 7f3153cf7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31440d4c60 0x7f314404e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.646+0000 7f3153cf7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314403a0d0 con 0x7f3138003b40 2024-05-09T10:50:26.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.646+0000 7f3151292640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f31440d80c0 0x7f31440cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.646+0000 7ff27bfff640 1 --2- 172.21.15.98:0/1115161796 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff28006b500 0x7ff28006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.646+0000 7f3151292640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f31440d80c0 0x7f31440cf2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:53426/0 (socket says 172.21.15.98:53426) 2024-05-09T10:50:26.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.646+0000 7f3151292640 1 -- 172.21.15.98:0/607732709 learned_addr learned my addr 172.21.15.98:0/607732709 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:26.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.646+0000 7f3152294640 1 --2- 172.21.15.98:0/607732709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31440d4c60 0x7f314404e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.646+0000 7f3153cf7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31440eb290 con 0x7f31440d4c60 2024-05-09T10:50:26.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.646+0000 7f3153cf7640 1 -- 172.21.15.98:0/607732709 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f314403ac50 con 0x7f31440d80c0 2024-05-09T10:50:26.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.646+0000 7ff27bfff640 1 -- 172.21.15.98:0/1115161796 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2880894b0 con 0x7ff2940cf050 2024-05-09T10:50:26.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.644+0000 7f510bfff640 1 --2- 172.21.15.98:0/3599426409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f510c06ed40 0x7f510c1566c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00d830 tx=0x7f50fc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.647+0000 7f0a2a7fc640 1 -- 172.21.15.98:0/1771840139 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c014440 con 0x7f0a3415dea0 2024-05-09T10:50:26.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.647+0000 7f3151a93640 1 --2- 172.21.15.98:0/607732709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3138003b40 0x7f31440cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.647+0000 7f3152294640 1 -- 172.21.15.98:0/607732709 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f31440d80c0 msgr2=0x7f31440cf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.647+0000 7f3152294640 1 --2- 172.21.15.98:0/607732709 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f31440d80c0 0x7f31440cf2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.647+0000 7f1c3f7fe640 1 -- 172.21.15.98:0/1577331885 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c48019070 con 0x7f1c400051c0 2024-05-09T10:50:26.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.647+0000 7f1c3f7fe640 1 -- 172.21.15.98:0/1577331885 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c480074e0 con 0x7f1c400051c0 2024-05-09T10:50:26.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.647+0000 7f1c3f7fe640 1 -- 172.21.15.98:0/1577331885 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c48004a00 con 0x7f1c400051c0 2024-05-09T10:50:26.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.647+0000 7f1c5d09a640 1 -- 172.21.15.98:0/1577331885 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5809bab0 con 0x7f1c400051c0 2024-05-09T10:50:26.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.647+0000 7f1c5d09a640 1 -- 172.21.15.98:0/1577331885 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5809c070 con 0x7f1c400051c0 2024-05-09T10:50:26.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.647+0000 7f3152294640 1 -- 172.21.15.98:0/607732709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3138003b40 msgr2=0x7f31440cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.647+0000 7f0a3c13b640 1 -- 172.21.15.98:0/1771840139 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1c005290 con 0x7f0a3415dea0 2024-05-09T10:50:26.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.647+0000 7f51097fa640 1 -- 172.21.15.98:0/3599426409 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc00b6c0 con 0x7f510c06ed40 2024-05-09T10:50:26.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.647+0000 7f51097fa640 1 -- 172.21.15.98:0/3599426409 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50fc019070 con 0x7f510c06ed40 2024-05-09T10:50:26.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.647+0000 7f51097fa640 1 -- 172.21.15.98:0/3599426409 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc00ba80 con 0x7f510c06ed40 2024-05-09T10:50:26.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.647+0000 7f51125c8640 1 -- 172.21.15.98:0/3599426409 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f510c149380 con 0x7f510c06ed40 2024-05-09T10:50:26.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.648+0000 7f51125c8640 1 -- 172.21.15.98:0/3599426409 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f510c148610 con 0x7f510c06ed40 2024-05-09T10:50:26.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.647+0000 7f3152294640 1 --2- 172.21.15.98:0/607732709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3138003b40 0x7f31440cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.647+0000 7f3152294640 1 -- 172.21.15.98:0/607732709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31440cfc00 con 0x7f31440d4c60 2024-05-09T10:50:26.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.648+0000 7f3152294640 1 --2- 172.21.15.98:0/607732709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31440d4c60 0x7f314404e8b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f314400dce0 tx=0x7f313c011cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.647+0000 7f3df37fe640 1 -- 172.21.15.98:0/4280552849 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c095020 con 0x7f3e040e4d20 2024-05-09T10:50:26.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.647+0000 7f3df37fe640 1 -- 172.21.15.98:0/4280552849 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c0918c0 con 0x7f3e040e4d20 2024-05-09T10:50:26.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.647+0000 7f3df37fe640 1 -- 172.21.15.98:0/4280552849 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c091cb0 con 0x7f3e040e4d20 2024-05-09T10:50:26.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.647+0000 7f3e1096f640 1 -- 172.21.15.98:0/4280552849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e0404cec0 con 0x7f3e040e4d20 2024-05-09T10:50:26.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.647+0000 7f3e1096f640 1 -- 172.21.15.98:0/4280552849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e040d8200 con 0x7f3e040e4d20 2024-05-09T10:50:26.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.647+0000 7f913effd640 1 -- 172.21.15.98:0/2713684965 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9138012a10 con 0x7f914806bbf0 2024-05-09T10:50:26.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.648+0000 7f913effd640 1 --2- 172.21.15.98:0/2713684965 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f912c06b4d0 0x7f912c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.648+0000 7f913effd640 1 -- 172.21.15.98:0/2713684965 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9138089e90 con 0x7f914806bbf0 2024-05-09T10:50:26.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.648+0000 7f913ffff640 1 --2- 172.21.15.98:0/2713684965 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f912c06b4d0 0x7f912c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.649+0000 7f1c5d09a640 1 -- 172.21.15.98:0/1577331885 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c38005290 con 0x7f1c400051c0 2024-05-09T10:50:26.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.649+0000 7ff2937fe640 1 --2- 172.21.15.98:0/1115161796 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff28006b500 0x7ff28006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.649+0000 7f3df17fa640 1 -- 172.21.15.98:0/4280552849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de4005290 con 0x7f3e040e4d20 2024-05-09T10:50:26.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.649+0000 7f913ffff640 1 --2- 172.21.15.98:0/2713684965 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f912c06b4d0 0x7f912c06d990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9130001600 tx=0x7f9130001570 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.649+0000 7f0a2a7fc640 1 -- 172.21.15.98:0/1771840139 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a2c01c070 con 0x7f0a3415dea0 2024-05-09T10:50:26.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.650+0000 7f0a2a7fc640 1 --2- 172.21.15.98:0/1771840139 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0a1806b5a0 0x7f0a1806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.650+0000 7f51125c8640 1 -- 172.21.15.98:0/3599426409 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f510c069cb0 con 0x7f510c06ed40 2024-05-09T10:50:26.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.650+0000 7f3df37fe640 1 -- 172.21.15.98:0/4280552849 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e0c098020 con 0x7f3e040e4d20 2024-05-09T10:50:26.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.650+0000 7f0a2a7fc640 1 -- 172.21.15.98:0/1771840139 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a2c089ec0 con 0x7f0a3415dea0 2024-05-09T10:50:26.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.650+0000 7ff2937fe640 1 --2- 172.21.15.98:0/1115161796 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff28006b500 0x7ff28006d9c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff2941531e0 tx=0x7ff28402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.651+0000 7f3df37fe640 1 --2- 172.21.15.98:0/4280552849 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3df806b520 0x7f3df806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.651+0000 7f3e0ad9d640 1 --2- 172.21.15.98:0/4280552849 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3df806b520 0x7f3df806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.651+0000 7f0a39ed7640 1 --2- 172.21.15.98:0/1771840139 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0a1806b5a0 0x7f0a1806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.651+0000 7ff27bfff640 1 -- 172.21.15.98:0/1115161796 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff288007e20 con 0x7ff2940cf050 2024-05-09T10:50:26.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.651+0000 7f1c3f7fe640 1 -- 172.21.15.98:0/1577331885 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c4801c070 con 0x7f1c400051c0 2024-05-09T10:50:26.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.651+0000 7f0a39ed7640 1 --2- 172.21.15.98:0/1771840139 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0a1806b5a0 0x7f0a1806da60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0a30034000 tx=0x7f0a3002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.651+0000 7f3e0ad9d640 1 --2- 172.21.15.98:0/4280552849 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3df806b520 0x7f3df806d9e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3e0000ac20 tx=0x7f3e00008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.651+0000 7f1c3f7fe640 1 --2- 172.21.15.98:0/1577331885 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1c4406b520 0x7f1c4406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.651+0000 7f51097fa640 1 -- 172.21.15.98:0/3599426409 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50fc01e460 con 0x7f510c06ed40 2024-05-09T10:50:26.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.652+0000 7f1c5659c640 1 --2- 172.21.15.98:0/1577331885 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1c4406b520 0x7f1c4406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.652+0000 7f51097fa640 1 --2- 172.21.15.98:0/3599426409 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f50ec06b520 0x7f50ec06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.652+0000 7f1c5659c640 1 --2- 172.21.15.98:0/1577331885 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1c4406b520 0x7f1c4406d9e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1c580ced60 tx=0x7f1c4c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.652+0000 7f5103fff640 1 --2- 172.21.15.98:0/3599426409 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f50ec06b520 0x7f50ec06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.652+0000 7f31427fc640 1 -- 172.21.15.98:0/607732709 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c019020 con 0x7f31440d4c60 2024-05-09T10:50:26.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.652+0000 7f31427fc640 1 -- 172.21.15.98:0/607732709 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c004240 con 0x7f31440d4c60 2024-05-09T10:50:26.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.652+0000 7f31427fc640 1 -- 172.21.15.98:0/607732709 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c00f770 con 0x7f31440d4c60 2024-05-09T10:50:26.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.652+0000 7f3153cf7640 1 -- 172.21.15.98:0/607732709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31440cfea0 con 0x7f31440d4c60 2024-05-09T10:50:26.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.653+0000 7f913effd640 1 -- 172.21.15.98:0/2713684965 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f914814fc00 con 0x7f914806bbf0 2024-05-09T10:50:26.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.653+0000 7f5103fff640 1 --2- 172.21.15.98:0/3599426409 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f50ec06b520 0x7f50ec06d9e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f50f4007fd0 tx=0x7f50f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.653+0000 7f3153cf7640 1 -- 172.21.15.98:0/607732709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31440cead0 con 0x7f31440d4c60 2024-05-09T10:50:26.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.654+0000 7f3153cf7640 1 -- 172.21.15.98:0/607732709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3144005810 con 0x7f31440d4c60 2024-05-09T10:50:26.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.655+0000 7f31427fc640 1 -- 172.21.15.98:0/607732709 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f313c01c020 con 0x7f31440d4c60 2024-05-09T10:50:26.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.655+0000 7f31427fc640 1 --2- 172.21.15.98:0/607732709 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f313006b5d0 0x7f313006da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:26.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.655+0000 7f31427fc640 1 -- 172.21.15.98:0/607732709 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f313c08a090 con 0x7f31440d4c60 2024-05-09T10:50:26.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.655+0000 7f3151a93640 1 --2- 172.21.15.98:0/607732709 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f313006b5d0 0x7f313006da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:26.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.656+0000 7f3151a93640 1 --2- 172.21.15.98:0/607732709 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f313006b5d0 0x7f313006da90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3148043000 tx=0x7f3148007f90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:26.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.657+0000 7f1c3f7fe640 1 -- 172.21.15.98:0/1577331885 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c480569b0 con 0x7f1c400051c0 2024-05-09T10:50:26.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.657+0000 7f51097fa640 1 -- 172.21.15.98:0/3599426409 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50fc057c40 con 0x7f510c06ed40 2024-05-09T10:50:26.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.657+0000 7f3df37fe640 1 -- 172.21.15.98:0/4280552849 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e0c0d5af0 con 0x7f3e040e4d20 2024-05-09T10:50:26.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.657+0000 7f0a2a7fc640 1 -- 172.21.15.98:0/1771840139 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0a2c0586d0 con 0x7f0a3415dea0 2024-05-09T10:50:26.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.659+0000 7f31427fc640 1 -- 172.21.15.98:0/607732709 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f313c023030 con 0x7f31440d4c60 2024-05-09T10:50:26.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.662+0000 7f3df37fe640 1 -- 172.21.15.98:0/4280552849 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3e0c09f030 con 0x7f3e040e4d20 2024-05-09T10:50:26.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.666+0000 7f1c3f7fe640 1 -- 172.21.15.98:0/1577331885 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1c48058910 con 0x7f1c400051c0 2024-05-09T10:50:26.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.671+0000 7f51097fa640 1 -- 172.21.15.98:0/3599426409 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f50fc05a2a0 con 0x7f510c06ed40 2024-05-09T10:50:26.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.999+0000 7f7675e38640 1 -- 172.21.15.98:0/2847156261 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7650005580 con 0x7f7670155000 2024-05-09T10:50:26.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:26.999+0000 7f7657fff640 1 -- 172.21.15.98:0/2847156261 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f76600590b0 con 0x7f7670155000 2024-05-09T10:50:26.993 INFO:teuthology.orchestra.run.smithi098.stdout:55834574855 2024-05-09T10:50:26.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.003+0000 7f7655ffb640 1 -- 172.21.15.98:0/2847156261 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f765c06b5d0 msgr2=0x7f765c06da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.003+0000 7f7655ffb640 1 --2- 172.21.15.98:0/2847156261 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f765c06b5d0 0x7f765c06da90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7664006fd0 tx=0x7f7664008040 comp rx=0 tx=0).stop 2024-05-09T10:50:26.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.003+0000 7f7655ffb640 1 -- 172.21.15.98:0/2847156261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7670155000 msgr2=0x7f76701fa6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:26.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.003+0000 7f7655ffb640 1 --2- 172.21.15.98:0/2847156261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7670155000 0x7f76701fa6f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f766000eca0 tx=0x7f766000c520 comp rx=0 tx=0).stop 2024-05-09T10:50:26.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.004+0000 7f7655ffb640 1 -- 172.21.15.98:0/2847156261 shutdown_connections 2024-05-09T10:50:26.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.004+0000 7f7655ffb640 1 --2- 172.21.15.98:0/2847156261 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f765c06b5d0 0x7f765c06da90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.004+0000 7f7655ffb640 1 --2- 172.21.15.98:0/2847156261 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f76701f8c00 0x7f76700cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.004+0000 7f7655ffb640 1 --2- 172.21.15.98:0/2847156261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7670155000 0x7f76701fa6f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.004+0000 7f7655ffb640 1 --2- 172.21.15.98:0/2847156261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7670071b00 0x7f7670154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:26.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.004+0000 7f7655ffb640 1 -- 172.21.15.98:0/2847156261 >> 172.21.15.98:0/2847156261 conn(0x7f76700b24c0 msgr2=0x7f767006d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:27.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.008+0000 7f7655ffb640 1 -- 172.21.15.98:0/2847156261 shutdown_connections 2024-05-09T10:50:27.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.008+0000 7f7655ffb640 1 -- 172.21.15.98:0/2847156261 wait complete. 2024-05-09T10:50:27.018 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-05-09T10:50:27.018 DEBUG:teuthology.parallel:result is None 2024-05-09T10:50:27.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.039+0000 7f3df17fa640 1 -- 172.21.15.98:0/4280552849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3de4005580 con 0x7f3e040e4d20 2024-05-09T10:50:27.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.040+0000 7f3df37fe640 1 -- 172.21.15.98:0/4280552849 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3e0c0d7a50 con 0x7f3e040e4d20 2024-05-09T10:50:27.033 INFO:teuthology.orchestra.run.smithi098.stdout:55834574853 2024-05-09T10:50:27.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.042+0000 7f3df17fa640 1 -- 172.21.15.98:0/4280552849 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3df806b520 msgr2=0x7f3df806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.042+0000 7f3df17fa640 1 --2- 172.21.15.98:0/4280552849 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3df806b520 0x7f3df806d9e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3e0000ac20 tx=0x7f3e00008040 comp rx=0 tx=0).stop 2024-05-09T10:50:27.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.042+0000 7f3df17fa640 1 -- 172.21.15.98:0/4280552849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e040e4d20 msgr2=0x7f3e040d8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.042+0000 7f3df17fa640 1 --2- 172.21.15.98:0/4280552849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e040e4d20 0x7f3e040d8a00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c050190 tx=0x7f3e0c068000 comp rx=0 tx=0).stop 2024-05-09T10:50:27.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.042+0000 7f3df17fa640 1 -- 172.21.15.98:0/4280552849 shutdown_connections 2024-05-09T10:50:27.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.042+0000 7f3df17fa640 1 --2- 172.21.15.98:0/4280552849 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3df806b520 0x7f3df806d9e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.042+0000 7f3df17fa640 1 --2- 172.21.15.98:0/4280552849 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e040e38b0 0x7f3e040c5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.042+0000 7f3df17fa640 1 --2- 172.21.15.98:0/4280552849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3e040e4d20 0x7f3e040d8a00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.042+0000 7f3df17fa640 1 --2- 172.21.15.98:0/4280552849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e040e12a0 0x7f3e040e3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.042+0000 7f3df17fa640 1 -- 172.21.15.98:0/4280552849 >> 172.21.15.98:0/4280552849 conn(0x7f3e04020030 msgr2=0x7f3e04056620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:27.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.042+0000 7f3df17fa640 1 -- 172.21.15.98:0/4280552849 shutdown_connections 2024-05-09T10:50:27.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.042+0000 7f3df17fa640 1 -- 172.21.15.98:0/4280552849 wait complete. 2024-05-09T10:50:27.047 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2024-05-09T10:50:27.047 DEBUG:teuthology.parallel:result is None 2024-05-09T10:50:27.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.055+0000 7ff299add640 1 -- 172.21.15.98:0/1115161796 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff29406cc70 con 0x7ff2940cf050 2024-05-09T10:50:27.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.056+0000 7ff27bfff640 1 -- 172.21.15.98:0/1115161796 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff288057c40 con 0x7ff2940cf050 2024-05-09T10:50:27.050 INFO:teuthology.orchestra.run.smithi098.stdout:55834574856 2024-05-09T10:50:27.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.057+0000 7f914c84d640 1 -- 172.21.15.98:0/2713684965 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f9148069cb0 con 0x7f914806bbf0 2024-05-09T10:50:27.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.058+0000 7f913effd640 1 -- 172.21.15.98:0/2713684965 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9138058630 con 0x7f914806bbf0 2024-05-09T10:50:27.051 INFO:teuthology.orchestra.run.smithi098.stdout:55834574855 2024-05-09T10:50:27.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.059+0000 7ff299add640 1 -- 172.21.15.98:0/1115161796 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff28006b500 msgr2=0x7ff28006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.059+0000 7ff299add640 1 --2- 172.21.15.98:0/1115161796 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff28006b500 0x7ff28006d9c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff2941531e0 tx=0x7ff28402e000 comp rx=0 tx=0).stop 2024-05-09T10:50:27.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.059+0000 7ff299add640 1 -- 172.21.15.98:0/1115161796 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2940cf050 msgr2=0x7ff2940acf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.059+0000 7ff299add640 1 --2- 172.21.15.98:0/1115161796 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2940cf050 0x7ff2940acf30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff28800c390 tx=0x7ff288001e70 comp rx=0 tx=0).stop 2024-05-09T10:50:27.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.059+0000 7ff299add640 1 -- 172.21.15.98:0/1115161796 shutdown_connections 2024-05-09T10:50:27.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.059+0000 7ff299add640 1 --2- 172.21.15.98:0/1115161796 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff28006b500 0x7ff28006d9c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.059+0000 7ff299add640 1 --2- 172.21.15.98:0/1115161796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2940cf430 0x7ff2940aaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.059+0000 7ff299add640 1 --2- 172.21.15.98:0/1115161796 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2940cf050 0x7ff2940acf30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.059+0000 7ff299add640 1 --2- 172.21.15.98:0/1115161796 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff29406dac0 0x7ff2940cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.059+0000 7ff299add640 1 -- 172.21.15.98:0/1115161796 >> 172.21.15.98:0/1115161796 conn(0x7ff29406abe0 msgr2=0x7ff2940b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:27.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.059+0000 7ff299add640 1 -- 172.21.15.98:0/1115161796 shutdown_connections 2024-05-09T10:50:27.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.059+0000 7ff299add640 1 -- 172.21.15.98:0/1115161796 wait complete. 2024-05-09T10:50:27.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.061+0000 7f914c84d640 1 -- 172.21.15.98:0/2713684965 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f912c06b4d0 msgr2=0x7f912c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.061+0000 7f914c84d640 1 --2- 172.21.15.98:0/2713684965 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f912c06b4d0 0x7f912c06d990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9130001600 tx=0x7f9130001570 comp rx=0 tx=0).stop 2024-05-09T10:50:27.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.061+0000 7f914c84d640 1 -- 172.21.15.98:0/2713684965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f914806bbf0 msgr2=0x7f914815cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.061+0000 7f914c84d640 1 --2- 172.21.15.98:0/2713684965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f914806bbf0 0x7f914815cc50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f913800c500 tx=0x7f913800c9c0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.061+0000 7f914c84d640 1 -- 172.21.15.98:0/2713684965 shutdown_connections 2024-05-09T10:50:27.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.061+0000 7f914c84d640 1 --2- 172.21.15.98:0/2713684965 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f912c06b4d0 0x7f912c06d990 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.061+0000 7f914c84d640 1 --2- 172.21.15.98:0/2713684965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f914815a190 0x7f91480cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.061+0000 7f914c84d640 1 --2- 172.21.15.98:0/2713684965 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f914806c0a0 0x7f91481f32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.061+0000 7f914c84d640 1 --2- 172.21.15.98:0/2713684965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f914806bbf0 0x7f914815cc50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.061+0000 7f914c84d640 1 -- 172.21.15.98:0/2713684965 >> 172.21.15.98:0/2713684965 conn(0x7f914806cae0 msgr2=0x7f91480668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:27.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.061+0000 7f914c84d640 1 -- 172.21.15.98:0/2713684965 shutdown_connections 2024-05-09T10:50:27.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.061+0000 7f914c84d640 1 -- 172.21.15.98:0/2713684965 wait complete. 2024-05-09T10:50:27.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.066+0000 7f0a3c13b640 1 -- 172.21.15.98:0/1771840139 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f0a1c005580 con 0x7f0a3415dea0 2024-05-09T10:50:27.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.067+0000 7f0a2a7fc640 1 -- 172.21.15.98:0/1771840139 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0a2c05c770 con 0x7f0a3415dea0 2024-05-09T10:50:27.060 INFO:teuthology.orchestra.run.smithi098.stdout:55834574855 2024-05-09T10:50:27.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.069+0000 7f3153cf7640 1 -- 172.21.15.98:0/607732709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f314400ad40 con 0x7f31440d4c60 2024-05-09T10:50:27.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.069+0000 7f31427fc640 1 -- 172.21.15.98:0/607732709 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f313c058f80 con 0x7f31440d4c60 2024-05-09T10:50:27.063 INFO:teuthology.orchestra.run.smithi098.stdout:55834574856 2024-05-09T10:50:27.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.070+0000 7f0a3c13b640 1 -- 172.21.15.98:0/1771840139 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0a1806b5a0 msgr2=0x7f0a1806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.070+0000 7f0a3c13b640 1 --2- 172.21.15.98:0/1771840139 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0a1806b5a0 0x7f0a1806da60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0a30034000 tx=0x7f0a3002e000 comp rx=0 tx=0).stop 2024-05-09T10:50:27.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.070+0000 7f0a3c13b640 1 -- 172.21.15.98:0/1771840139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a3415dea0 msgr2=0x7f0a340cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.070+0000 7f0a3c13b640 1 --2- 172.21.15.98:0/1771840139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a3415dea0 0x7f0a340cc4c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00b9e0 tx=0x7f0a2c00bea0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.071+0000 7f0a3c13b640 1 -- 172.21.15.98:0/1771840139 shutdown_connections 2024-05-09T10:50:27.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.071+0000 7f0a3c13b640 1 --2- 172.21.15.98:0/1771840139 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0a1806b5a0 0x7f0a1806da60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.071+0000 7f0a3c13b640 1 --2- 172.21.15.98:0/1771840139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a3415dea0 0x7f0a340cc4c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.071+0000 7f0a3c13b640 1 --2- 172.21.15.98:0/1771840139 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a340d1350 0x7f0a34159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.071+0000 7f0a3c13b640 1 --2- 172.21.15.98:0/1771840139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a340cece0 0x7f0a340cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.071+0000 7f0a3c13b640 1 -- 172.21.15.98:0/1771840139 >> 172.21.15.98:0/1771840139 conn(0x7f0a3409cee0 msgr2=0x7f0a3406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:27.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.071+0000 7f0a3c13b640 1 -- 172.21.15.98:0/1771840139 shutdown_connections 2024-05-09T10:50:27.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.071+0000 7f0a3c13b640 1 -- 172.21.15.98:0/1771840139 wait complete. 2024-05-09T10:50:27.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.072+0000 7f3153cf7640 1 -- 172.21.15.98:0/607732709 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f313006b5d0 msgr2=0x7f313006da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.072+0000 7f3153cf7640 1 --2- 172.21.15.98:0/607732709 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f313006b5d0 0x7f313006da90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3148043000 tx=0x7f3148007f90 comp rx=0 tx=0).stop 2024-05-09T10:50:27.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.072+0000 7f3153cf7640 1 -- 172.21.15.98:0/607732709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31440d4c60 msgr2=0x7f314404e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.072+0000 7f3153cf7640 1 --2- 172.21.15.98:0/607732709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31440d4c60 0x7f314404e8b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f314400dce0 tx=0x7f313c011cf0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.073+0000 7f3153cf7640 1 -- 172.21.15.98:0/607732709 shutdown_connections 2024-05-09T10:50:27.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.073+0000 7f3153cf7640 1 --2- 172.21.15.98:0/607732709 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f313006b5d0 0x7f313006da90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.073+0000 7f3153cf7640 1 --2- 172.21.15.98:0/607732709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31440d4c60 0x7f314404e8b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.073+0000 7f3153cf7640 1 --2- 172.21.15.98:0/607732709 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f31440d80c0 0x7f31440cf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.073+0000 7f3153cf7640 1 --2- 172.21.15.98:0/607732709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3138003b40 0x7f31440cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.073+0000 7f3153cf7640 1 -- 172.21.15.98:0/607732709 >> 172.21.15.98:0/607732709 conn(0x7f314401e6f0 msgr2=0x7f3144006c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:27.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.073+0000 7f3153cf7640 1 -- 172.21.15.98:0/607732709 shutdown_connections 2024-05-09T10:50:27.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.073+0000 7f3153cf7640 1 -- 172.21.15.98:0/607732709 wait complete. 2024-05-09T10:50:27.068 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.0 2024-05-09T10:50:27.068 DEBUG:teuthology.parallel:result is None 2024-05-09T10:50:27.071 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2024-05-09T10:50:27.071 DEBUG:teuthology.parallel:result is None 2024-05-09T10:50:27.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.083+0000 7f1c5d09a640 1 -- 172.21.15.98:0/1577331885 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f1c38005850 con 0x7f1c400051c0 2024-05-09T10:50:27.077 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2024-05-09T10:50:27.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.084+0000 7f1c3f7fe640 1 -- 172.21.15.98:0/1577331885 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1c4805c9b0 con 0x7f1c400051c0 2024-05-09T10:50:27.078 INFO:teuthology.orchestra.run.smithi098.stdout:55834574854 2024-05-09T10:50:27.078 DEBUG:teuthology.parallel:result is None 2024-05-09T10:50:27.079 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.1 2024-05-09T10:50:27.079 DEBUG:teuthology.parallel:result is None 2024-05-09T10:50:27.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.087+0000 7f1c5d09a640 1 -- 172.21.15.98:0/1577331885 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1c4406b520 msgr2=0x7f1c4406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.087+0000 7f1c5d09a640 1 --2- 172.21.15.98:0/1577331885 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1c4406b520 0x7f1c4406d9e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1c580ced60 tx=0x7f1c4c008040 comp rx=0 tx=0).stop 2024-05-09T10:50:27.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.087+0000 7f1c5d09a640 1 -- 172.21.15.98:0/1577331885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c400051c0 msgr2=0x7f1c5815e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.087+0000 7f1c5d09a640 1 --2- 172.21.15.98:0/1577331885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c400051c0 0x7f1c5815e070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1c4800d810 tx=0x7f1c4800dcd0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.087+0000 7f1c5d09a640 1 -- 172.21.15.98:0/1577331885 shutdown_connections 2024-05-09T10:50:27.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.087+0000 7f1c5d09a640 1 --2- 172.21.15.98:0/1577331885 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1c4406b520 0x7f1c4406d9e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.087+0000 7f1c5d09a640 1 --2- 172.21.15.98:0/1577331885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c5809aa80 0x7f1c5809af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.087+0000 7f1c5d09a640 1 --2- 172.21.15.98:0/1577331885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1c5809cd90 0x7f1c5809d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.087+0000 7f1c5d09a640 1 --2- 172.21.15.98:0/1577331885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c400051c0 0x7f1c5815e070 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.087+0000 7f1c5d09a640 1 -- 172.21.15.98:0/1577331885 >> 172.21.15.98:0/1577331885 conn(0x7f1c5806adc0 msgr2=0x7f1c58071f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:27.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.087+0000 7f1c5d09a640 1 -- 172.21.15.98:0/1577331885 shutdown_connections 2024-05-09T10:50:27.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.087+0000 7f1c5d09a640 1 -- 172.21.15.98:0/1577331885 wait complete. 2024-05-09T10:50:27.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.087+0000 7f51125c8640 1 -- 172.21.15.98:0/3599426409 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f510c0b58c0 con 0x7f510c06ed40 2024-05-09T10:50:27.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.087+0000 7f51097fa640 1 -- 172.21.15.98:0/3599426409 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f50fc059ba0 con 0x7f510c06ed40 2024-05-09T10:50:27.082 INFO:teuthology.orchestra.run.smithi098.stdout:55834574856 2024-05-09T10:50:27.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.089+0000 7f51125c8640 1 -- 172.21.15.98:0/3599426409 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f50ec06b520 msgr2=0x7f50ec06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.089+0000 7f51125c8640 1 --2- 172.21.15.98:0/3599426409 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f50ec06b520 0x7f50ec06d9e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f50f4007fd0 tx=0x7f50f4009040 comp rx=0 tx=0).stop 2024-05-09T10:50:27.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.089+0000 7f51125c8640 1 -- 172.21.15.98:0/3599426409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f510c06ed40 msgr2=0x7f510c1566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.089+0000 7f51125c8640 1 --2- 172.21.15.98:0/3599426409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f510c06ed40 0x7f510c1566c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00d830 tx=0x7f50fc00dcf0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.089+0000 7f51125c8640 1 -- 172.21.15.98:0/3599426409 shutdown_connections 2024-05-09T10:50:27.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.089+0000 7f51125c8640 1 --2- 172.21.15.98:0/3599426409 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f50ec06b520 0x7f50ec06d9e0 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f50f4007fd0 tx=0x7f50f4009040 comp rx=0 tx=0).stop 2024-05-09T10:50:27.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.090+0000 7f51125c8640 1 --2- 172.21.15.98:0/3599426409 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f510c155000 0x7f510c142ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.090+0000 7f51125c8640 1 --2- 172.21.15.98:0/3599426409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f510c071930 0x7f510c156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.090+0000 7f51125c8640 1 --2- 172.21.15.98:0/3599426409 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f510c06ed40 0x7f510c1566c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.090+0000 7f51125c8640 1 -- 172.21.15.98:0/3599426409 >> 172.21.15.98:0/3599426409 conn(0x7f510c0b24c0 msgr2=0x7f510c15e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:27.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.090+0000 7f51125c8640 1 -- 172.21.15.98:0/3599426409 shutdown_connections 2024-05-09T10:50:27.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.090+0000 7f51125c8640 1 -- 172.21.15.98:0/3599426409 wait complete. 2024-05-09T10:50:27.092 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-05-09T10:50:27.092 DEBUG:teuthology.parallel:result is None 2024-05-09T10:50:27.094 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2024-05-09T10:50:27.094 DEBUG:teuthology.parallel:result is None 2024-05-09T10:50:27.094 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-09T10:50:27.094 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T10:50:27.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.242+0000 7ffb05c9e640 1 Processor -- start 2024-05-09T10:50:27.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.242+0000 7ffb05c9e640 1 -- start start 2024-05-09T10:50:27.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.242+0000 7ffb05c9e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ffb0015f4e0 0x7ffb000ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:27.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.242+0000 7ffb05c9e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb000b09e0 0x7ffb00165c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:27.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.242+0000 7ffb05c9e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb000aeda0 con 0x7ffb000afe40 2024-05-09T10:50:27.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.242+0000 7ffb05c9e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb000af080 con 0x7ffb0015f4e0 2024-05-09T10:50:27.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.242+0000 7ffb05c9e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb000af360 con 0x7ffb000b09e0 2024-05-09T10:50:27.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.242+0000 7ffaff7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ffb0015f4e0 0x7ffb000ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:27.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.243+0000 7ffaff7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ffb0015f4e0 0x7ffb000ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42266/0 (socket says 172.21.15.98:42266) 2024-05-09T10:50:27.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.243+0000 7ffaff7fe640 1 -- 172.21.15.98:0/4182519752 learned_addr learned my addr 172.21.15.98:0/4182519752 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:27.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.243+0000 7ffafffff640 1 --2- 172.21.15.98:0/4182519752 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb000b09e0 0x7ffb00165c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:27.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.243+0000 7ffaff7fe640 1 -- 172.21.15.98:0/4182519752 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb000b09e0 msgr2=0x7ffb00165c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:27.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.243+0000 7ffaff7fe640 1 --2- 172.21.15.98:0/4182519752 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb000b09e0 0x7ffb00165c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.243+0000 7ffaff7fe640 1 -- 172.21.15.98:0/4182519752 >> v1:172.21.15.105:6789/0 conn(0x7ffb000afe40 legacy=0x7ffb000b02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:27.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.243+0000 7ffaff7fe640 1 -- 172.21.15.98:0/4182519752 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb000af640 con 0x7ffb0015f4e0 2024-05-09T10:50:27.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.243+0000 7ffaff7fe640 1 --2- 172.21.15.98:0/4182519752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb0015f4e0 0x7ffb000ae7c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009870 tx=0x7ffaf002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8e73925a5212e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:27.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.244+0000 7ffafe7fc640 1 -- 172.21.15.98:0/4182519752 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0037070 con 0x7ffb0015f4e0 2024-05-09T10:50:27.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.244+0000 7ffafe7fc640 1 -- 172.21.15.98:0/4182519752 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf0032440 con 0x7ffb0015f4e0 2024-05-09T10:50:27.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.244+0000 7ffafe7fc640 1 -- 172.21.15.98:0/4182519752 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0032810 con 0x7ffb0015f4e0 2024-05-09T10:50:27.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.244+0000 7ffb05c9e640 1 -- 172.21.15.98:0/4182519752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb0015f4e0 msgr2=0x7ffb000ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.244+0000 7ffb05c9e640 1 --2- 172.21.15.98:0/4182519752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb0015f4e0 0x7ffb000ae7c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009870 tx=0x7ffaf002a180 comp rx=0 tx=0).stop 2024-05-09T10:50:27.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.244+0000 7ffb05c9e640 1 -- 172.21.15.98:0/4182519752 shutdown_connections 2024-05-09T10:50:27.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.244+0000 7ffb05c9e640 1 --2- 172.21.15.98:0/4182519752 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb000b09e0 0x7ffb00165c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.244+0000 7ffb05c9e640 1 --2- 172.21.15.98:0/4182519752 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb0015f4e0 0x7ffb000ae7c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.244+0000 7ffb05c9e640 1 -- 172.21.15.98:0/4182519752 >> 172.21.15.98:0/4182519752 conn(0x7ffb000b34e0 msgr2=0x7ffb000b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:27.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.244+0000 7ffb05c9e640 1 -- 172.21.15.98:0/4182519752 shutdown_connections 2024-05-09T10:50:27.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.244+0000 7ffb05c9e640 1 -- 172.21.15.98:0/4182519752 wait complete. 2024-05-09T10:50:27.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.245+0000 7ffb05c9e640 1 Processor -- start 2024-05-09T10:50:27.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.246+0000 7ffb05c9e640 1 -- start start 2024-05-09T10:50:27.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.246+0000 7ffb05c9e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb000afe40 0x7ffb000b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:27.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.246+0000 7ffb05c9e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb000b09e0 0x7ffb000b6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:27.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.246+0000 7ffb05c9e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb0015f4e0 0x7ffb000cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.246+0000 7ffb05c9e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0015f8c0 con 0x7ffb000b09e0 2024-05-09T10:50:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.246+0000 7ffb05c9e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb000cbcf0 con 0x7ffb0015f4e0 2024-05-09T10:50:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.246+0000 7ffb05c9e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb000b0e30 con 0x7ffb000afe40 2024-05-09T10:50:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.246+0000 7ffafeffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb000b09e0 0x7ffb000b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.246+0000 7ffafeffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb000b09e0 0x7ffb000b6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42282/0 (socket says 172.21.15.98:42282) 2024-05-09T10:50:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.246+0000 7ffafeffd640 1 -- 172.21.15.98:0/3762837808 learned_addr learned my addr 172.21.15.98:0/3762837808 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.247+0000 7ffaff7fe640 1 --2- 172.21.15.98:0/3762837808 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb000afe40 0x7ffb000b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.247+0000 7ffafffff640 1 --2- 172.21.15.98:0/3762837808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb0015f4e0 0x7ffb000cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.247+0000 7ffafeffd640 1 -- 172.21.15.98:0/3762837808 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb000afe40 msgr2=0x7ffb000b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.247+0000 7ffafeffd640 1 --2- 172.21.15.98:0/3762837808 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb000afe40 0x7ffb000b38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.247+0000 7ffafeffd640 1 -- 172.21.15.98:0/3762837808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb0015f4e0 msgr2=0x7ffb000cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.247+0000 7ffafeffd640 1 --2- 172.21.15.98:0/3762837808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb0015f4e0 0x7ffb000cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.247+0000 7ffafeffd640 1 -- 172.21.15.98:0/3762837808 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb000ca3a0 con 0x7ffb000b09e0 2024-05-09T10:50:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.247+0000 7ffafeffd640 1 --2- 172.21.15.98:0/3762837808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb000b09e0 0x7ffb000b6730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffae800c430 tx=0x7ffae800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:27.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.247+0000 7ffaf7fff640 1 -- 172.21.15.98:0/3762837808 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8011790 con 0x7ffb000b09e0 2024-05-09T10:50:27.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.247+0000 7ffaf7fff640 1 -- 172.21.15.98:0/3762837808 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae8017070 con 0x7ffb000b09e0 2024-05-09T10:50:27.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.247+0000 7ffaf7fff640 1 -- 172.21.15.98:0/3762837808 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8011bb0 con 0x7ffb000b09e0 2024-05-09T10:50:27.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.247+0000 7ffb05c9e640 1 -- 172.21.15.98:0/3762837808 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb000ca640 con 0x7ffb000b09e0 2024-05-09T10:50:27.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.247+0000 7ffb05c9e640 1 -- 172.21.15.98:0/3762837808 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb000b3ee0 con 0x7ffb000b09e0 2024-05-09T10:50:27.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.248+0000 7ffb05c9e640 1 -- 172.21.15.98:0/3762837808 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb000ae830 con 0x7ffb000b09e0 2024-05-09T10:50:27.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.248+0000 7ffaf7fff640 1 -- 172.21.15.98:0/3762837808 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffae801c460 con 0x7ffb000b09e0 2024-05-09T10:50:27.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.249+0000 7ffaf7fff640 1 --2- 172.21.15.98:0/3762837808 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ffae006b4d0 0x7ffae006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:27.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.249+0000 7ffaf7fff640 1 -- 172.21.15.98:0/3762837808 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffae808a0f0 con 0x7ffb000b09e0 2024-05-09T10:50:27.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.249+0000 7ffaff7fe640 1 --2- 172.21.15.98:0/3762837808 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ffae006b4d0 0x7ffae006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:27.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.249+0000 7ffaff7fe640 1 --2- 172.21.15.98:0/3762837808 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ffae006b4d0 0x7ffae006d990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0034000 tx=0x7ffaf002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:27.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.253+0000 7ffaf7fff640 1 -- 172.21.15.98:0/3762837808 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffae8058f70 con 0x7ffb000b09e0 2024-05-09T10:50:27.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.420+0000 7ffb05c9e640 1 -- 172.21.15.98:0/3762837808 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffb0006fa00 con 0x7ffae006b4d0 2024-05-09T10:50:27.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.426+0000 7ffaf7fff640 1 -- 172.21.15.98:0/3762837808 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251872 (secure 0 0 0) 0x7ffaf007f7f0 con 0x7ffae006b4d0 2024-05-09T10:50:27.422 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:27.423 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-05-09T10:50:27.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.429+0000 7ffb05c9e640 1 -- 172.21.15.98:0/3762837808 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ffae006b4d0 msgr2=0x7ffae006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.429+0000 7ffb05c9e640 1 --2- 172.21.15.98:0/3762837808 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ffae006b4d0 0x7ffae006d990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0034000 tx=0x7ffaf002e000 comp rx=0 tx=0).stop 2024-05-09T10:50:27.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.429+0000 7ffb05c9e640 1 -- 172.21.15.98:0/3762837808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb000b09e0 msgr2=0x7ffb000b6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.429+0000 7ffb05c9e640 1 --2- 172.21.15.98:0/3762837808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb000b09e0 0x7ffb000b6730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffae800c430 tx=0x7ffae800c8f0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.429+0000 7ffb05c9e640 1 -- 172.21.15.98:0/3762837808 shutdown_connections 2024-05-09T10:50:27.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.429+0000 7ffb05c9e640 1 --2- 172.21.15.98:0/3762837808 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ffae006b4d0 0x7ffae006d990 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.429+0000 7ffb05c9e640 1 --2- 172.21.15.98:0/3762837808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb0015f4e0 0x7ffb000cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.429+0000 7ffb05c9e640 1 --2- 172.21.15.98:0/3762837808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb000b09e0 0x7ffb000b6730 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.429+0000 7ffb05c9e640 1 --2- 172.21.15.98:0/3762837808 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffb000afe40 0x7ffb000b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.429+0000 7ffb05c9e640 1 -- 172.21.15.98:0/3762837808 >> 172.21.15.98:0/3762837808 conn(0x7ffb000b34e0 msgr2=0x7ffb0006ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:27.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.429+0000 7ffb05c9e640 1 -- 172.21.15.98:0/3762837808 shutdown_connections 2024-05-09T10:50:27.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.429+0000 7ffb05c9e640 1 -- 172.21.15.98:0/3762837808 wait complete. 2024-05-09T10:50:27.433 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-05-09T10:50:27.367243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001561"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889511+0000","last_change":"2024-05-09T10:50:20.202000+0000","last_active":"2024-05-09T10:50:21.889511+0000","last_peered":"2024-05-09T10:50:21.889511+0000","last_clean":"2024-05-09T10:50:21.889511+0000","last_became_active":"2024-05-09T10:50:20.201722+0000","last_became_peered":"2024-05-09T10:50:20.201722+0000","last_unstale":"2024-05-09T10:50:21.889511+0000","last_undegraded":"2024-05-09T10:50:21.889511+0000","last_fullsized":"2024-05-09T10:50:21.889511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:51:23.834797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182607+0000","last_change":"2024-05-09T10:50:20.185947+0000","last_active":"2024-05-09T10:50:21.182607+0000","last_peered":"2024-05-09T10:50:21.182607+0000","last_clean":"2024-05-09T10:50:21.182607+0000","last_became_active":"2024-05-09T10:50:20.185672+0000","last_became_peered":"2024-05-09T10:50:20.185672+0000","last_unstale":"2024-05-09T10:50:21.182607+0000","last_undegraded":"2024-05-09T10:50:21.182607+0000","last_fullsized":"2024-05-09T10:50:21.182607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:45.067992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182555+0000","last_change":"2024-05-09T10:50:20.186891+0000","last_active":"2024-05-09T10:50:21.182555+0000","last_peered":"2024-05-09T10:50:21.182555+0000","last_clean":"2024-05-09T10:50:21.182555+0000","last_became_active":"2024-05-09T10:50:20.186605+0000","last_became_peered":"2024-05-09T10:50:20.186605+0000","last_unstale":"2024-05-09T10:50:21.182555+0000","last_undegraded":"2024-05-09T10:50:21.182555+0000","last_fullsized":"2024-05-09T10:50:21.182555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:30.799640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.638238+0000","last_change":"2024-05-09T10:50:20.195509+0000","last_active":"2024-05-09T10:50:22.638238+0000","last_peered":"2024-05-09T10:50:22.638238+0000","last_clean":"2024-05-09T10:50:22.638238+0000","last_became_active":"2024-05-09T10:50:20.195045+0000","last_became_peered":"2024-05-09T10:50:20.195045+0000","last_unstale":"2024-05-09T10:50:22.638238+0000","last_undegraded":"2024-05-09T10:50:22.638238+0000","last_fullsized":"2024-05-09T10:50:22.638238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:31:09.517362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889170+0000","last_change":"2024-05-09T10:50:20.198458+0000","last_active":"2024-05-09T10:50:21.889170+0000","last_peered":"2024-05-09T10:50:21.889170+0000","last_clean":"2024-05-09T10:50:21.889170+0000","last_became_active":"2024-05-09T10:50:20.195845+0000","last_became_peered":"2024-05-09T10:50:20.195845+0000","last_unstale":"2024-05-09T10:50:21.889170+0000","last_undegraded":"2024-05-09T10:50:21.889170+0000","last_fullsized":"2024-05-09T10:50:21.889170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:31:09.517362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.637784+0000","last_change":"2024-05-09T10:50:20.179665+0000","last_active":"2024-05-09T10:50:22.637784+0000","last_peered":"2024-05-09T10:50:22.637784+0000","last_clean":"2024-05-09T10:50:22.637784+0000","last_became_active":"2024-05-09T10:50:20.179379+0000","last_became_peered":"2024-05-09T10:50:20.179379+0000","last_unstale":"2024-05-09T10:50:22.637784+0000","last_undegraded":"2024-05-09T10:50:22.637784+0000","last_fullsized":"2024-05-09T10:50:22.637784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:26:34.638740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.888989+0000","last_change":"2024-05-09T10:50:20.199853+0000","last_active":"2024-05-09T10:50:21.888989+0000","last_peered":"2024-05-09T10:50:21.888989+0000","last_clean":"2024-05-09T10:50:21.888989+0000","last_became_active":"2024-05-09T10:50:20.199552+0000","last_became_peered":"2024-05-09T10:50:20.199552+0000","last_unstale":"2024-05-09T10:50:21.888989+0000","last_undegraded":"2024-05-09T10:50:21.888989+0000","last_fullsized":"2024-05-09T10:50:21.888989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:09.785658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843477+0000","last_change":"2024-05-09T10:50:20.191734+0000","last_active":"2024-05-09T10:50:23.843477+0000","last_peered":"2024-05-09T10:50:23.843477+0000","last_clean":"2024-05-09T10:50:23.843477+0000","last_became_active":"2024-05-09T10:50:20.191484+0000","last_became_peered":"2024-05-09T10:50:20.191484+0000","last_unstale":"2024-05-09T10:50:23.843477+0000","last_undegraded":"2024-05-09T10:50:23.843477+0000","last_fullsized":"2024-05-09T10:50:23.843477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:32:42.671953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889473+0000","last_change":"2024-05-09T10:50:20.199646+0000","last_active":"2024-05-09T10:50:21.889473+0000","last_peered":"2024-05-09T10:50:21.889473+0000","last_clean":"2024-05-09T10:50:21.889473+0000","last_became_active":"2024-05-09T10:50:20.199374+0000","last_became_peered":"2024-05-09T10:50:20.199374+0000","last_unstale":"2024-05-09T10:50:21.889473+0000","last_undegraded":"2024-05-09T10:50:21.889473+0000","last_fullsized":"2024-05-09T10:50:21.889473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:27:14.733561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171869+0000","last_change":"2024-05-09T10:50:20.189694+0000","last_active":"2024-05-09T10:50:22.171869+0000","last_peered":"2024-05-09T10:50:22.171869+0000","last_clean":"2024-05-09T10:50:22.171869+0000","last_became_active":"2024-05-09T10:50:20.189429+0000","last_became_peered":"2024-05-09T10:50:20.189429+0000","last_unstale":"2024-05-09T10:50:22.171869+0000","last_undegraded":"2024-05-09T10:50:22.171869+0000","last_fullsized":"2024-05-09T10:50:22.171869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:50:14.919494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171776+0000","last_change":"2024-05-09T10:50:20.189569+0000","last_active":"2024-05-09T10:50:22.171776+0000","last_peered":"2024-05-09T10:50:22.171776+0000","last_clean":"2024-05-09T10:50:22.171776+0000","last_became_active":"2024-05-09T10:50:20.189272+0000","last_became_peered":"2024-05-09T10:50:20.189272+0000","last_unstale":"2024-05-09T10:50:22.171776+0000","last_undegraded":"2024-05-09T10:50:22.171776+0000","last_fullsized":"2024-05-09T10:50:22.171776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:47.420832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183678+0000","last_change":"2024-05-09T10:50:20.191736+0000","last_active":"2024-05-09T10:50:21.183678+0000","last_peered":"2024-05-09T10:50:21.183678+0000","last_clean":"2024-05-09T10:50:21.183678+0000","last_became_active":"2024-05-09T10:50:20.190652+0000","last_became_peered":"2024-05-09T10:50:20.190652+0000","last_unstale":"2024-05-09T10:50:21.183678+0000","last_undegraded":"2024-05-09T10:50:21.183678+0000","last_fullsized":"2024-05-09T10:50:21.183678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:08:00.575142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843704+0000","last_change":"2024-05-09T10:50:20.192354+0000","last_active":"2024-05-09T10:50:23.843704+0000","last_peered":"2024-05-09T10:50:23.843704+0000","last_clean":"2024-05-09T10:50:23.843704+0000","last_became_active":"2024-05-09T10:50:20.191736+0000","last_became_peered":"2024-05-09T10:50:20.191736+0000","last_unstale":"2024-05-09T10:50:23.843704+0000","last_undegraded":"2024-05-09T10:50:23.843704+0000","last_fullsized":"2024-05-09T10:50:23.843704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:50:14.919494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182518+0000","last_change":"2024-05-09T10:50:20.184216+0000","last_active":"2024-05-09T10:50:21.182518+0000","last_peered":"2024-05-09T10:50:21.182518+0000","last_clean":"2024-05-09T10:50:21.182518+0000","last_became_active":"2024-05-09T10:50:20.183920+0000","last_became_peered":"2024-05-09T10:50:20.183920+0000","last_unstale":"2024-05-09T10:50:21.182518+0000","last_undegraded":"2024-05-09T10:50:21.182518+0000","last_fullsized":"2024-05-09T10:50:21.182518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:57:07.297876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183244+0000","last_change":"2024-05-09T10:50:20.189990+0000","last_active":"2024-05-09T10:50:21.183244+0000","last_peered":"2024-05-09T10:50:21.183244+0000","last_clean":"2024-05-09T10:50:21.183244+0000","last_became_active":"2024-05-09T10:50:20.189727+0000","last_became_peered":"2024-05-09T10:50:20.189727+0000","last_unstale":"2024-05-09T10:50:21.183244+0000","last_undegraded":"2024-05-09T10:50:21.183244+0000","last_fullsized":"2024-05-09T10:50:21.183244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:34:20.711637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.637718+0000","last_change":"2024-05-09T10:50:20.179735+0000","last_active":"2024-05-09T10:50:22.637718+0000","last_peered":"2024-05-09T10:50:22.637718+0000","last_clean":"2024-05-09T10:50:22.637718+0000","last_became_active":"2024-05-09T10:50:20.179440+0000","last_became_peered":"2024-05-09T10:50:20.179440+0000","last_unstale":"2024-05-09T10:50:22.637718+0000","last_undegraded":"2024-05-09T10:50:22.637718+0000","last_fullsized":"2024-05-09T10:50:22.637718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:34:20.711637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.638388+0000","last_change":"2024-05-09T10:50:20.194956+0000","last_active":"2024-05-09T10:50:22.638388+0000","last_peered":"2024-05-09T10:50:22.638388+0000","last_clean":"2024-05-09T10:50:22.638388+0000","last_became_active":"2024-05-09T10:50:20.194660+0000","last_became_peered":"2024-05-09T10:50:20.194660+0000","last_unstale":"2024-05-09T10:50:22.638388+0000","last_undegraded":"2024-05-09T10:50:22.638388+0000","last_fullsized":"2024-05-09T10:50:22.638388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:07:48.935082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.264445+0000","last_change":"2024-05-09T10:50:20.188140+0000","last_active":"2024-05-09T10:50:23.264445+0000","last_peered":"2024-05-09T10:50:23.264445+0000","last_clean":"2024-05-09T10:50:23.264445+0000","last_became_active":"2024-05-09T10:50:20.187855+0000","last_became_peered":"2024-05-09T10:50:20.187855+0000","last_unstale":"2024-05-09T10:50:23.264445+0000","last_undegraded":"2024-05-09T10:50:23.264445+0000","last_fullsized":"2024-05-09T10:50:23.264445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:16:53.166077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889478+0000","last_change":"2024-05-09T10:50:20.199567+0000","last_active":"2024-05-09T10:50:21.889478+0000","last_peered":"2024-05-09T10:50:21.889478+0000","last_clean":"2024-05-09T10:50:21.889478+0000","last_became_active":"2024-05-09T10:50:20.198671+0000","last_became_peered":"2024-05-09T10:50:20.198671+0000","last_unstale":"2024-05-09T10:50:21.889478+0000","last_undegraded":"2024-05-09T10:50:21.889478+0000","last_fullsized":"2024-05-09T10:50:21.889478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:03:14.102727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171768+0000","last_change":"2024-05-09T10:50:20.188963+0000","last_active":"2024-05-09T10:50:22.171768+0000","last_peered":"2024-05-09T10:50:22.171768+0000","last_clean":"2024-05-09T10:50:22.171768+0000","last_became_active":"2024-05-09T10:50:20.188654+0000","last_became_peered":"2024-05-09T10:50:20.188654+0000","last_unstale":"2024-05-09T10:50:22.171768+0000","last_undegraded":"2024-05-09T10:50:22.171768+0000","last_fullsized":"2024-05-09T10:50:22.171768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:23.176619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182825+0000","last_change":"2024-05-09T10:50:20.188028+0000","last_active":"2024-05-09T10:50:21.182825+0000","last_peered":"2024-05-09T10:50:21.182825+0000","last_clean":"2024-05-09T10:50:21.182825+0000","last_became_active":"2024-05-09T10:50:20.186949+0000","last_became_peered":"2024-05-09T10:50:20.186949+0000","last_unstale":"2024-05-09T10:50:21.182825+0000","last_undegraded":"2024-05-09T10:50:21.182825+0000","last_fullsized":"2024-05-09T10:50:21.182825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:00:03.184256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.637880+0000","last_change":"2024-05-09T10:50:20.187160+0000","last_active":"2024-05-09T10:50:22.637880+0000","last_peered":"2024-05-09T10:50:22.637880+0000","last_clean":"2024-05-09T10:50:22.637880+0000","last_became_active":"2024-05-09T10:50:20.186840+0000","last_became_peered":"2024-05-09T10:50:20.186840+0000","last_unstale":"2024-05-09T10:50:22.637880+0000","last_undegraded":"2024-05-09T10:50:22.637880+0000","last_fullsized":"2024-05-09T10:50:22.637880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:33:19.903309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843512+0000","last_change":"2024-05-09T10:50:20.188532+0000","last_active":"2024-05-09T10:50:23.843512+0000","last_peered":"2024-05-09T10:50:23.843512+0000","last_clean":"2024-05-09T10:50:23.843512+0000","last_became_active":"2024-05-09T10:50:20.188264+0000","last_became_peered":"2024-05-09T10:50:20.188264+0000","last_unstale":"2024-05-09T10:50:23.843512+0000","last_undegraded":"2024-05-09T10:50:23.843512+0000","last_fullsized":"2024-05-09T10:50:23.843512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:00:03.184256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.637786+0000","last_change":"2024-05-09T10:50:20.194256+0000","last_active":"2024-05-09T10:50:22.637786+0000","last_peered":"2024-05-09T10:50:22.637786+0000","last_clean":"2024-05-09T10:50:22.637786+0000","last_became_active":"2024-05-09T10:50:20.193137+0000","last_became_peered":"2024-05-09T10:50:20.193137+0000","last_unstale":"2024-05-09T10:50:22.637786+0000","last_undegraded":"2024-05-09T10:50:22.637786+0000","last_fullsized":"2024-05-09T10:50:22.637786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:02:11.043966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.844007+0000","last_change":"2024-05-09T10:50:20.189585+0000","last_active":"2024-05-09T10:50:23.844007+0000","last_peered":"2024-05-09T10:50:23.844007+0000","last_clean":"2024-05-09T10:50:23.844007+0000","last_became_active":"2024-05-09T10:50:20.189295+0000","last_became_peered":"2024-05-09T10:50:20.189295+0000","last_unstale":"2024-05-09T10:50:23.844007+0000","last_undegraded":"2024-05-09T10:50:23.844007+0000","last_fullsized":"2024-05-09T10:50:23.844007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:33:19.903309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171966+0000","last_change":"2024-05-09T10:50:20.196303+0000","last_active":"2024-05-09T10:50:22.171966+0000","last_peered":"2024-05-09T10:50:22.171966+0000","last_clean":"2024-05-09T10:50:22.171966+0000","last_became_active":"2024-05-09T10:50:20.195986+0000","last_became_peered":"2024-05-09T10:50:20.195986+0000","last_unstale":"2024-05-09T10:50:22.171966+0000","last_undegraded":"2024-05-09T10:50:22.171966+0000","last_fullsized":"2024-05-09T10:50:22.171966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:34:20.711637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182978+0000","last_change":"2024-05-09T10:50:20.193168+0000","last_active":"2024-05-09T10:50:21.182978+0000","last_peered":"2024-05-09T10:50:21.182978+0000","last_clean":"2024-05-09T10:50:21.182978+0000","last_became_active":"2024-05-09T10:50:20.192898+0000","last_became_peered":"2024-05-09T10:50:20.192898+0000","last_unstale":"2024-05-09T10:50:21.182978+0000","last_undegraded":"2024-05-09T10:50:21.182978+0000","last_fullsized":"2024-05-09T10:50:21.182978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:50:14.919494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843887+0000","last_change":"2024-05-09T10:50:20.188735+0000","last_active":"2024-05-09T10:50:23.843887+0000","last_peered":"2024-05-09T10:50:23.843887+0000","last_clean":"2024-05-09T10:50:23.843887+0000","last_became_active":"2024-05-09T10:50:20.188138+0000","last_became_peered":"2024-05-09T10:50:20.188138+0000","last_unstale":"2024-05-09T10:50:23.843887+0000","last_undegraded":"2024-05-09T10:50:23.843887+0000","last_fullsized":"2024-05-09T10:50:23.843887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:02:11.043966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182945+0000","last_change":"2024-05-09T10:50:20.194279+0000","last_active":"2024-05-09T10:50:21.182945+0000","last_peered":"2024-05-09T10:50:21.182945+0000","last_clean":"2024-05-09T10:50:21.182945+0000","last_became_active":"2024-05-09T10:50:20.193962+0000","last_became_peered":"2024-05-09T10:50:20.193962+0000","last_unstale":"2024-05-09T10:50:21.182945+0000","last_undegraded":"2024-05-09T10:50:21.182945+0000","last_fullsized":"2024-05-09T10:50:21.182945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:23.176619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.638158+0000","last_change":"2024-05-09T10:50:20.193350+0000","last_active":"2024-05-09T10:50:22.638158+0000","last_peered":"2024-05-09T10:50:22.638158+0000","last_clean":"2024-05-09T10:50:22.638158+0000","last_became_active":"2024-05-09T10:50:20.192514+0000","last_became_peered":"2024-05-09T10:50:20.192514+0000","last_unstale":"2024-05-09T10:50:22.638158+0000","last_undegraded":"2024-05-09T10:50:22.638158+0000","last_fullsized":"2024-05-09T10:50:22.638158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:09.785658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.264333+0000","last_change":"2024-05-09T10:50:20.175999+0000","last_active":"2024-05-09T10:50:23.264333+0000","last_peered":"2024-05-09T10:50:23.264333+0000","last_clean":"2024-05-09T10:50:23.264333+0000","last_became_active":"2024-05-09T10:50:20.175695+0000","last_became_peered":"2024-05-09T10:50:20.175695+0000","last_unstale":"2024-05-09T10:50:23.264333+0000","last_undegraded":"2024-05-09T10:50:23.264333+0000","last_fullsized":"2024-05-09T10:50:23.264333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:08:00.575142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171539+0000","last_change":"2024-05-09T10:50:20.196639+0000","last_active":"2024-05-09T10:50:22.171539+0000","last_peered":"2024-05-09T10:50:22.171539+0000","last_clean":"2024-05-09T10:50:22.171539+0000","last_became_active":"2024-05-09T10:50:20.196365+0000","last_became_peered":"2024-05-09T10:50:20.196365+0000","last_unstale":"2024-05-09T10:50:22.171539+0000","last_undegraded":"2024-05-09T10:50:22.171539+0000","last_fullsized":"2024-05-09T10:50:22.171539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:32:42.671953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183207+0000","last_change":"2024-05-09T10:50:20.185161+0000","last_active":"2024-05-09T10:50:21.183207+0000","last_peered":"2024-05-09T10:50:21.183207+0000","last_clean":"2024-05-09T10:50:21.183207+0000","last_became_active":"2024-05-09T10:50:20.184832+0000","last_became_peered":"2024-05-09T10:50:20.184832+0000","last_unstale":"2024-05-09T10:50:21.183207+0000","last_undegraded":"2024-05-09T10:50:21.183207+0000","last_fullsized":"2024-05-09T10:50:21.183207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:21.666208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843791+0000","last_change":"2024-05-09T10:50:19.173345+0000","last_active":"2024-05-09T10:50:23.843791+0000","last_peered":"2024-05-09T10:50:23.843791+0000","last_clean":"2024-05-09T10:50:23.843791+0000","last_became_active":"2024-05-09T10:50:19.173059+0000","last_became_peered":"2024-05-09T10:50:19.173059+0000","last_unstale":"2024-05-09T10:50:23.843791+0000","last_undegraded":"2024-05-09T10:50:23.843791+0000","last_fullsized":"2024-05-09T10:50:23.843791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:53.042012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183068+0000","last_change":"2024-05-09T10:50:20.188071+0000","last_active":"2024-05-09T10:50:21.183068+0000","last_peered":"2024-05-09T10:50:21.183068+0000","last_clean":"2024-05-09T10:50:21.183068+0000","last_became_active":"2024-05-09T10:50:20.187766+0000","last_became_peered":"2024-05-09T10:50:20.187766+0000","last_unstale":"2024-05-09T10:50:21.183068+0000","last_undegraded":"2024-05-09T10:50:21.183068+0000","last_fullsized":"2024-05-09T10:50:21.183068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:47.420832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171634+0000","last_change":"2024-05-09T10:50:19.169316+0000","last_active":"2024-05-09T10:50:22.171634+0000","last_peered":"2024-05-09T10:50:22.171634+0000","last_clean":"2024-05-09T10:50:22.171634+0000","last_became_active":"2024-05-09T10:50:19.169040+0000","last_became_peered":"2024-05-09T10:50:19.169040+0000","last_unstale":"2024-05-09T10:50:22.171634+0000","last_undegraded":"2024-05-09T10:50:22.171634+0000","last_fullsized":"2024-05-09T10:50:22.171634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:39.416778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843566+0000","last_change":"2024-05-09T10:50:19.167276+0000","last_active":"2024-05-09T10:50:23.843566+0000","last_peered":"2024-05-09T10:50:23.843566+0000","last_clean":"2024-05-09T10:50:23.843566+0000","last_became_active":"2024-05-09T10:50:19.166775+0000","last_became_peered":"2024-05-09T10:50:19.166775+0000","last_unstale":"2024-05-09T10:50:23.843566+0000","last_undegraded":"2024-05-09T10:50:23.843566+0000","last_fullsized":"2024-05-09T10:50:23.843566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:41:57.513979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843560+0000","last_change":"2024-05-09T10:50:20.181449+0000","last_active":"2024-05-09T10:50:23.843560+0000","last_peered":"2024-05-09T10:50:23.843560+0000","last_clean":"2024-05-09T10:50:23.843560+0000","last_became_active":"2024-05-09T10:50:20.181137+0000","last_became_peered":"2024-05-09T10:50:20.181137+0000","last_unstale":"2024-05-09T10:50:23.843560+0000","last_undegraded":"2024-05-09T10:50:23.843560+0000","last_fullsized":"2024-05-09T10:50:23.843560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:45.067992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.637867+0000","last_change":"2024-05-09T10:50:19.165083+0000","last_active":"2024-05-09T10:50:22.637867+0000","last_peered":"2024-05-09T10:50:22.637867+0000","last_clean":"2024-05-09T10:50:22.637867+0000","last_became_active":"2024-05-09T10:50:19.164468+0000","last_became_peered":"2024-05-09T10:50:19.164468+0000","last_unstale":"2024-05-09T10:50:22.637867+0000","last_undegraded":"2024-05-09T10:50:22.637867+0000","last_fullsized":"2024-05-09T10:50:22.637867+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:53.042012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182977+0000","last_change":"2024-05-09T10:50:20.193953+0000","last_active":"2024-05-09T10:50:21.182977+0000","last_peered":"2024-05-09T10:50:21.182977+0000","last_clean":"2024-05-09T10:50:21.182977+0000","last_became_active":"2024-05-09T10:50:20.193686+0000","last_became_peered":"2024-05-09T10:50:20.193686+0000","last_unstale":"2024-05-09T10:50:21.182977+0000","last_undegraded":"2024-05-09T10:50:21.182977+0000","last_fullsized":"2024-05-09T10:50:21.182977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:47.420832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171830+0000","last_change":"2024-05-09T10:50:19.164495+0000","last_active":"2024-05-09T10:50:22.171830+0000","last_peered":"2024-05-09T10:50:22.171830+0000","last_clean":"2024-05-09T10:50:22.171830+0000","last_became_active":"2024-05-09T10:50:19.164073+0000","last_became_peered":"2024-05-09T10:50:19.164073+0000","last_unstale":"2024-05-09T10:50:22.171830+0000","last_undegraded":"2024-05-09T10:50:22.171830+0000","last_fullsized":"2024-05-09T10:50:22.171830+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:53.042012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.638230+0000","last_change":"2024-05-09T10:50:20.186184+0000","last_active":"2024-05-09T10:50:22.638230+0000","last_peered":"2024-05-09T10:50:22.638230+0000","last_clean":"2024-05-09T10:50:22.638230+0000","last_became_active":"2024-05-09T10:50:20.185892+0000","last_became_peered":"2024-05-09T10:50:20.185892+0000","last_unstale":"2024-05-09T10:50:22.638230+0000","last_undegraded":"2024-05-09T10:50:22.638230+0000","last_fullsized":"2024-05-09T10:50:22.638230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:35.361899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.638218+0000","last_change":"2024-05-09T10:50:19.166015+0000","last_active":"2024-05-09T10:50:22.638218+0000","last_peered":"2024-05-09T10:50:22.638218+0000","last_clean":"2024-05-09T10:50:22.638218+0000","last_became_active":"2024-05-09T10:50:19.163525+0000","last_became_peered":"2024-05-09T10:50:19.163525+0000","last_unstale":"2024-05-09T10:50:22.638218+0000","last_undegraded":"2024-05-09T10:50:22.638218+0000","last_fullsized":"2024-05-09T10:50:22.638218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:37.762424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183206+0000","last_change":"2024-05-09T10:50:20.183493+0000","last_active":"2024-05-09T10:50:21.183206+0000","last_peered":"2024-05-09T10:50:21.183206+0000","last_clean":"2024-05-09T10:50:21.183206+0000","last_became_active":"2024-05-09T10:50:20.183222+0000","last_became_peered":"2024-05-09T10:50:20.183222+0000","last_unstale":"2024-05-09T10:50:21.183206+0000","last_undegraded":"2024-05-09T10:50:21.183206+0000","last_fullsized":"2024-05-09T10:50:21.183206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:34:20.711637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183073+0000","last_change":"2024-05-09T10:50:19.174608+0000","last_active":"2024-05-09T10:50:21.183073+0000","last_peered":"2024-05-09T10:50:21.183073+0000","last_clean":"2024-05-09T10:50:21.183073+0000","last_became_active":"2024-05-09T10:50:19.174299+0000","last_became_peered":"2024-05-09T10:50:19.174299+0000","last_unstale":"2024-05-09T10:50:21.183073+0000","last_undegraded":"2024-05-09T10:50:21.183073+0000","last_fullsized":"2024-05-09T10:50:21.183073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:49:05.717962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183616+0000","last_change":"2024-05-09T10:50:20.186118+0000","last_active":"2024-05-09T10:50:21.183616+0000","last_peered":"2024-05-09T10:50:21.183616+0000","last_clean":"2024-05-09T10:50:21.183616+0000","last_became_active":"2024-05-09T10:50:20.185818+0000","last_became_peered":"2024-05-09T10:50:20.185818+0000","last_unstale":"2024-05-09T10:50:21.183616+0000","last_undegraded":"2024-05-09T10:50:21.183616+0000","last_fullsized":"2024-05-09T10:50:21.183616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:49:06.723077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171706+0000","last_change":"2024-05-09T10:50:19.170262+0000","last_active":"2024-05-09T10:50:22.171706+0000","last_peered":"2024-05-09T10:50:22.171706+0000","last_clean":"2024-05-09T10:50:22.171706+0000","last_became_active":"2024-05-09T10:50:19.169908+0000","last_became_peered":"2024-05-09T10:50:19.169908+0000","last_unstale":"2024-05-09T10:50:22.171706+0000","last_undegraded":"2024-05-09T10:50:22.171706+0000","last_fullsized":"2024-05-09T10:50:22.171706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:37.762424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889300+0000","last_change":"2024-05-09T10:50:20.199969+0000","last_active":"2024-05-09T10:50:21.889300+0000","last_peered":"2024-05-09T10:50:21.889300+0000","last_clean":"2024-05-09T10:50:21.889300+0000","last_became_active":"2024-05-09T10:50:20.199060+0000","last_became_peered":"2024-05-09T10:50:20.199060+0000","last_unstale":"2024-05-09T10:50:21.889300+0000","last_undegraded":"2024-05-09T10:50:21.889300+0000","last_fullsized":"2024-05-09T10:50:21.889300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:35.361899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183103+0000","last_change":"2024-05-09T10:50:19.176037+0000","last_active":"2024-05-09T10:50:21.183103+0000","last_peered":"2024-05-09T10:50:21.183103+0000","last_clean":"2024-05-09T10:50:21.183103+0000","last_became_active":"2024-05-09T10:50:19.170706+0000","last_became_peered":"2024-05-09T10:50:19.170706+0000","last_unstale":"2024-05-09T10:50:21.183103+0000","last_undegraded":"2024-05-09T10:50:21.183103+0000","last_fullsized":"2024-05-09T10:50:21.183103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:54:23.668155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182651+0000","last_change":"2024-05-09T10:50:20.194758+0000","last_active":"2024-05-09T10:50:21.182651+0000","last_peered":"2024-05-09T10:50:21.182651+0000","last_clean":"2024-05-09T10:50:21.182651+0000","last_became_active":"2024-05-09T10:50:20.194509+0000","last_became_peered":"2024-05-09T10:50:20.194509+0000","last_unstale":"2024-05-09T10:50:21.182651+0000","last_undegraded":"2024-05-09T10:50:21.182651+0000","last_fullsized":"2024-05-09T10:50:21.182651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:32:42.671953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.638065+0000","last_change":"2024-05-09T10:50:19.167487+0000","last_active":"2024-05-09T10:50:22.638065+0000","last_peered":"2024-05-09T10:50:22.638065+0000","last_clean":"2024-05-09T10:50:22.638065+0000","last_became_active":"2024-05-09T10:50:19.167139+0000","last_became_peered":"2024-05-09T10:50:19.167139+0000","last_unstale":"2024-05-09T10:50:22.638065+0000","last_undegraded":"2024-05-09T10:50:22.638065+0000","last_fullsized":"2024-05-09T10:50:22.638065+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:20.661093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889232+0000","last_change":"2024-05-09T10:50:20.198511+0000","last_active":"2024-05-09T10:50:21.889232+0000","last_peered":"2024-05-09T10:50:21.889232+0000","last_clean":"2024-05-09T10:50:21.889232+0000","last_became_active":"2024-05-09T10:50:20.198165+0000","last_became_peered":"2024-05-09T10:50:20.198165+0000","last_unstale":"2024-05-09T10:50:21.889232+0000","last_undegraded":"2024-05-09T10:50:21.889232+0000","last_fullsized":"2024-05-09T10:50:21.889232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:47:01.454064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171521+0000","last_change":"2024-05-09T10:50:19.173828+0000","last_active":"2024-05-09T10:50:22.171521+0000","last_peered":"2024-05-09T10:50:22.171521+0000","last_clean":"2024-05-09T10:50:22.171521+0000","last_became_active":"2024-05-09T10:50:19.173524+0000","last_became_peered":"2024-05-09T10:50:19.173524+0000","last_unstale":"2024-05-09T10:50:22.171521+0000","last_undegraded":"2024-05-09T10:50:22.171521+0000","last_fullsized":"2024-05-09T10:50:22.171521+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:18:17.201844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171530+0000","last_change":"2024-05-09T10:50:20.196631+0000","last_active":"2024-05-09T10:50:22.171530+0000","last_peered":"2024-05-09T10:50:22.171530+0000","last_clean":"2024-05-09T10:50:22.171530+0000","last_became_active":"2024-05-09T10:50:20.196252+0000","last_became_peered":"2024-05-09T10:50:20.196252+0000","last_unstale":"2024-05-09T10:50:22.171530+0000","last_undegraded":"2024-05-09T10:50:22.171530+0000","last_fullsized":"2024-05-09T10:50:22.171530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:00:03.184256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.264260+0000","last_change":"2024-05-09T10:50:19.173070+0000","last_active":"2024-05-09T10:50:23.264260+0000","last_peered":"2024-05-09T10:50:23.264260+0000","last_clean":"2024-05-09T10:50:23.264260+0000","last_became_active":"2024-05-09T10:50:19.172711+0000","last_became_peered":"2024-05-09T10:50:19.172711+0000","last_unstale":"2024-05-09T10:50:23.264260+0000","last_undegraded":"2024-05-09T10:50:23.264260+0000","last_fullsized":"2024-05-09T10:50:23.264260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:53.042012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889056+0000","last_change":"2024-05-09T10:50:20.197201+0000","last_active":"2024-05-09T10:50:21.889056+0000","last_peered":"2024-05-09T10:50:21.889056+0000","last_clean":"2024-05-09T10:50:21.889056+0000","last_became_active":"2024-05-09T10:50:20.196926+0000","last_became_peered":"2024-05-09T10:50:20.196926+0000","last_unstale":"2024-05-09T10:50:21.889056+0000","last_undegraded":"2024-05-09T10:50:21.889056+0000","last_fullsized":"2024-05-09T10:50:21.889056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:39.852439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183547+0000","last_change":"2024-05-09T10:50:19.179678+0000","last_active":"2024-05-09T10:50:21.183547+0000","last_peered":"2024-05-09T10:50:21.183547+0000","last_clean":"2024-05-09T10:50:21.183547+0000","last_became_active":"2024-05-09T10:50:19.179419+0000","last_became_peered":"2024-05-09T10:50:19.179419+0000","last_unstale":"2024-05-09T10:50:21.183547+0000","last_undegraded":"2024-05-09T10:50:21.183547+0000","last_fullsized":"2024-05-09T10:50:21.183547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:38:13.651797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889436+0000","last_change":"2024-05-09T10:50:20.197970+0000","last_active":"2024-05-09T10:50:21.889436+0000","last_peered":"2024-05-09T10:50:21.889436+0000","last_clean":"2024-05-09T10:50:21.889436+0000","last_became_active":"2024-05-09T10:50:20.197653+0000","last_became_peered":"2024-05-09T10:50:20.197653+0000","last_unstale":"2024-05-09T10:50:21.889436+0000","last_undegraded":"2024-05-09T10:50:21.889436+0000","last_fullsized":"2024-05-09T10:50:21.889436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:07:48.935082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.264428+0000","last_change":"2024-05-09T10:50:19.174326+0000","last_active":"2024-05-09T10:50:23.264428+0000","last_peered":"2024-05-09T10:50:23.264428+0000","last_clean":"2024-05-09T10:50:23.264428+0000","last_became_active":"2024-05-09T10:50:19.173957+0000","last_became_peered":"2024-05-09T10:50:19.173957+0000","last_unstale":"2024-05-09T10:50:23.264428+0000","last_undegraded":"2024-05-09T10:50:23.264428+0000","last_fullsized":"2024-05-09T10:50:23.264428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:20.661093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889578+0000","last_change":"2024-05-09T10:50:20.199478+0000","last_active":"2024-05-09T10:50:21.889578+0000","last_peered":"2024-05-09T10:50:21.889578+0000","last_clean":"2024-05-09T10:50:21.889578+0000","last_became_active":"2024-05-09T10:50:20.199083+0000","last_became_peered":"2024-05-09T10:50:20.199083+0000","last_unstale":"2024-05-09T10:50:21.889578+0000","last_undegraded":"2024-05-09T10:50:21.889578+0000","last_fullsized":"2024-05-09T10:50:21.889578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:34:17.252316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183186+0000","last_change":"2024-05-09T10:50:19.173709+0000","last_active":"2024-05-09T10:50:21.183186+0000","last_peered":"2024-05-09T10:50:21.183186+0000","last_clean":"2024-05-09T10:50:21.183186+0000","last_became_active":"2024-05-09T10:50:19.173453+0000","last_became_peered":"2024-05-09T10:50:19.173453+0000","last_unstale":"2024-05-09T10:50:21.183186+0000","last_undegraded":"2024-05-09T10:50:21.183186+0000","last_fullsized":"2024-05-09T10:50:21.183186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:07:59.570026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.638324+0000","last_change":"2024-05-09T10:50:20.193882+0000","last_active":"2024-05-09T10:50:22.638324+0000","last_peered":"2024-05-09T10:50:22.638324+0000","last_clean":"2024-05-09T10:50:22.638324+0000","last_became_active":"2024-05-09T10:50:20.193186+0000","last_became_peered":"2024-05-09T10:50:20.193186+0000","last_unstale":"2024-05-09T10:50:22.638324+0000","last_undegraded":"2024-05-09T10:50:22.638324+0000","last_fullsized":"2024-05-09T10:50:22.638324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:15.043180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.264468+0000","last_change":"2024-05-09T10:50:19.163763+0000","last_active":"2024-05-09T10:50:23.264468+0000","last_peered":"2024-05-09T10:50:23.264468+0000","last_clean":"2024-05-09T10:50:23.264468+0000","last_became_active":"2024-05-09T10:50:19.163429+0000","last_became_peered":"2024-05-09T10:50:19.163429+0000","last_unstale":"2024-05-09T10:50:23.264468+0000","last_undegraded":"2024-05-09T10:50:23.264468+0000","last_fullsized":"2024-05-09T10:50:23.264468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:41:57.513979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889302+0000","last_change":"2024-05-09T10:50:20.200705+0000","last_active":"2024-05-09T10:50:21.889302+0000","last_peered":"2024-05-09T10:50:21.889302+0000","last_clean":"2024-05-09T10:50:21.889302+0000","last_became_active":"2024-05-09T10:50:20.200416+0000","last_became_peered":"2024-05-09T10:50:20.200416+0000","last_unstale":"2024-05-09T10:50:21.889302+0000","last_undegraded":"2024-05-09T10:50:21.889302+0000","last_fullsized":"2024-05-09T10:50:21.889302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:15.043180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.888962+0000","last_change":"2024-05-09T10:50:08.410364+0000","last_active":"2024-05-09T10:50:21.888962+0000","last_peered":"2024-05-09T10:50:21.888962+0000","last_clean":"2024-05-09T10:50:21.888962+0000","last_became_active":"2024-05-09T10:50:08.409457+0000","last_became_peered":"2024-05-09T10:50:08.409457+0000","last_unstale":"2024-05-09T10:50:21.888962+0000","last_undegraded":"2024-05-09T10:50:21.888962+0000","last_fullsized":"2024-05-09T10:50:21.888962+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_clean_scrub_stamp":"2024-05-09T10:50:07.396510+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:03.566698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.637748+0000","last_change":"2024-05-09T10:50:19.158709+0000","last_active":"2024-05-09T10:50:22.637748+0000","last_peered":"2024-05-09T10:50:22.637748+0000","last_clean":"2024-05-09T10:50:22.637748+0000","last_became_active":"2024-05-09T10:50:19.158385+0000","last_became_peered":"2024-05-09T10:50:19.158385+0000","last_unstale":"2024-05-09T10:50:22.637748+0000","last_undegraded":"2024-05-09T10:50:22.637748+0000","last_fullsized":"2024-05-09T10:50:22.637748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:18:17.201844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843642+0000","last_change":"2024-05-09T10:50:20.190695+0000","last_active":"2024-05-09T10:50:23.843642+0000","last_peered":"2024-05-09T10:50:23.843642+0000","last_clean":"2024-05-09T10:50:23.843642+0000","last_became_active":"2024-05-09T10:50:20.190446+0000","last_became_peered":"2024-05-09T10:50:20.190446+0000","last_unstale":"2024-05-09T10:50:23.843642+0000","last_undegraded":"2024-05-09T10:50:23.843642+0000","last_fullsized":"2024-05-09T10:50:23.843642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:23.176619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889057+0000","last_change":"2024-05-09T10:50:19.173033+0000","last_active":"2024-05-09T10:50:21.889057+0000","last_peered":"2024-05-09T10:50:21.889057+0000","last_clean":"2024-05-09T10:50:21.889057+0000","last_became_active":"2024-05-09T10:50:19.172691+0000","last_became_peered":"2024-05-09T10:50:19.172691+0000","last_unstale":"2024-05-09T10:50:21.889057+0000","last_undegraded":"2024-05-09T10:50:21.889057+0000","last_fullsized":"2024-05-09T10:50:21.889057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:49:05.717962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182430+0000","last_change":"2024-05-09T10:50:20.186680+0000","last_active":"2024-05-09T10:50:21.182430+0000","last_peered":"2024-05-09T10:50:21.182430+0000","last_clean":"2024-05-09T10:50:21.182430+0000","last_became_active":"2024-05-09T10:50:20.186413+0000","last_became_peered":"2024-05-09T10:50:20.186413+0000","last_unstale":"2024-05-09T10:50:21.182430+0000","last_undegraded":"2024-05-09T10:50:21.182430+0000","last_fullsized":"2024-05-09T10:50:21.182430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:33:37.505180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.264354+0000","last_change":"2024-05-09T10:50:19.166130+0000","last_active":"2024-05-09T10:50:23.264354+0000","last_peered":"2024-05-09T10:50:23.264354+0000","last_clean":"2024-05-09T10:50:23.264354+0000","last_became_active":"2024-05-09T10:50:19.165820+0000","last_became_peered":"2024-05-09T10:50:19.165820+0000","last_unstale":"2024-05-09T10:50:23.264354+0000","last_undegraded":"2024-05-09T10:50:23.264354+0000","last_fullsized":"2024-05-09T10:50:23.264354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:18:17.201844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182893+0000","last_change":"2024-05-09T10:50:20.190864+0000","last_active":"2024-05-09T10:50:21.182893+0000","last_peered":"2024-05-09T10:50:21.182893+0000","last_clean":"2024-05-09T10:50:21.182893+0000","last_became_active":"2024-05-09T10:50:20.190613+0000","last_became_peered":"2024-05-09T10:50:20.190613+0000","last_unstale":"2024-05-09T10:50:21.182893+0000","last_undegraded":"2024-05-09T10:50:21.182893+0000","last_fullsized":"2024-05-09T10:50:21.182893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:23.176619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.638288+0000","last_change":"2024-05-09T10:50:19.173006+0000","last_active":"2024-05-09T10:50:22.638288+0000","last_peered":"2024-05-09T10:50:22.638288+0000","last_clean":"2024-05-09T10:50:22.638288+0000","last_became_active":"2024-05-09T10:50:19.172607+0000","last_became_peered":"2024-05-09T10:50:19.172607+0000","last_unstale":"2024-05-09T10:50:22.638288+0000","last_undegraded":"2024-05-09T10:50:22.638288+0000","last_fullsized":"2024-05-09T10:50:22.638288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:39.416778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183468+0000","last_change":"2024-05-09T10:50:20.192361+0000","last_active":"2024-05-09T10:50:21.183468+0000","last_peered":"2024-05-09T10:50:21.183468+0000","last_clean":"2024-05-09T10:50:21.183468+0000","last_became_active":"2024-05-09T10:50:20.192116+0000","last_became_peered":"2024-05-09T10:50:20.192116+0000","last_unstale":"2024-05-09T10:50:21.183468+0000","last_undegraded":"2024-05-09T10:50:21.183468+0000","last_fullsized":"2024-05-09T10:50:21.183468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:38.767540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182627+0000","last_change":"2024-05-09T10:50:19.176926+0000","last_active":"2024-05-09T10:50:21.182627+0000","last_peered":"2024-05-09T10:50:21.182627+0000","last_clean":"2024-05-09T10:50:21.182627+0000","last_became_active":"2024-05-09T10:50:19.176627+0000","last_became_peered":"2024-05-09T10:50:19.176627+0000","last_unstale":"2024-05-09T10:50:21.182627+0000","last_undegraded":"2024-05-09T10:50:21.182627+0000","last_fullsized":"2024-05-09T10:50:21.182627+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:37.762424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.638236+0000","last_change":"2024-05-09T10:50:20.193387+0000","last_active":"2024-05-09T10:50:22.638236+0000","last_peered":"2024-05-09T10:50:22.638236+0000","last_clean":"2024-05-09T10:50:22.638236+0000","last_became_active":"2024-05-09T10:50:20.193115+0000","last_became_peered":"2024-05-09T10:50:20.193115+0000","last_unstale":"2024-05-09T10:50:22.638236+0000","last_undegraded":"2024-05-09T10:50:22.638236+0000","last_fullsized":"2024-05-09T10:50:22.638236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:47:01.454064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889536+0000","last_change":"2024-05-09T10:50:19.170801+0000","last_active":"2024-05-09T10:50:21.889536+0000","last_peered":"2024-05-09T10:50:21.889536+0000","last_clean":"2024-05-09T10:50:21.889536+0000","last_became_active":"2024-05-09T10:50:19.170130+0000","last_became_peered":"2024-05-09T10:50:19.170130+0000","last_unstale":"2024-05-09T10:50:21.889536+0000","last_undegraded":"2024-05-09T10:50:21.889536+0000","last_fullsized":"2024-05-09T10:50:21.889536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:04:45.153539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182544+0000","last_change":"2024-05-09T10:50:20.189228+0000","last_active":"2024-05-09T10:50:21.182544+0000","last_peered":"2024-05-09T10:50:21.182544+0000","last_clean":"2024-05-09T10:50:21.182544+0000","last_became_active":"2024-05-09T10:50:20.188969+0000","last_became_peered":"2024-05-09T10:50:20.188969+0000","last_unstale":"2024-05-09T10:50:21.182544+0000","last_undegraded":"2024-05-09T10:50:21.182544+0000","last_fullsized":"2024-05-09T10:50:21.182544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:00:03.184256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.170973+0000","last_change":"2024-05-09T10:50:19.171062+0000","last_active":"2024-05-09T10:50:22.170973+0000","last_peered":"2024-05-09T10:50:22.170973+0000","last_clean":"2024-05-09T10:50:22.170973+0000","last_became_active":"2024-05-09T10:50:19.170760+0000","last_became_peered":"2024-05-09T10:50:19.170760+0000","last_unstale":"2024-05-09T10:50:22.170973+0000","last_undegraded":"2024-05-09T10:50:22.170973+0000","last_fullsized":"2024-05-09T10:50:22.170973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:39.416778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.844039+0000","last_change":"2024-05-09T10:50:20.193239+0000","last_active":"2024-05-09T10:50:23.844039+0000","last_peered":"2024-05-09T10:50:23.844039+0000","last_clean":"2024-05-09T10:50:23.844039+0000","last_became_active":"2024-05-09T10:50:20.193014+0000","last_became_peered":"2024-05-09T10:50:20.193014+0000","last_unstale":"2024-05-09T10:50:23.844039+0000","last_undegraded":"2024-05-09T10:50:23.844039+0000","last_fullsized":"2024-05-09T10:50:23.844039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:33:13.347174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171972+0000","last_change":"2024-05-09T10:50:19.171256+0000","last_active":"2024-05-09T10:50:22.171972+0000","last_peered":"2024-05-09T10:50:22.171972+0000","last_clean":"2024-05-09T10:50:22.171972+0000","last_became_active":"2024-05-09T10:50:19.170964+0000","last_became_peered":"2024-05-09T10:50:19.170964+0000","last_unstale":"2024-05-09T10:50:22.171972+0000","last_undegraded":"2024-05-09T10:50:22.171972+0000","last_fullsized":"2024-05-09T10:50:22.171972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:20.661093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182872+0000","last_change":"2024-05-09T10:50:20.184272+0000","last_active":"2024-05-09T10:50:21.182872+0000","last_peered":"2024-05-09T10:50:21.182872+0000","last_clean":"2024-05-09T10:50:21.182872+0000","last_became_active":"2024-05-09T10:50:20.183978+0000","last_became_peered":"2024-05-09T10:50:20.183978+0000","last_unstale":"2024-05-09T10:50:21.182872+0000","last_undegraded":"2024-05-09T10:50:21.182872+0000","last_fullsized":"2024-05-09T10:50:21.182872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:45.067992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.888875+0000","last_change":"2024-05-09T10:50:19.171403+0000","last_active":"2024-05-09T10:50:21.888875+0000","last_peered":"2024-05-09T10:50:21.888875+0000","last_clean":"2024-05-09T10:50:21.888875+0000","last_became_active":"2024-05-09T10:50:19.171106+0000","last_became_peered":"2024-05-09T10:50:19.171106+0000","last_unstale":"2024-05-09T10:50:21.888875+0000","last_undegraded":"2024-05-09T10:50:21.888875+0000","last_fullsized":"2024-05-09T10:50:21.888875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:39.416778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183306+0000","last_change":"2024-05-09T10:50:20.189694+0000","last_active":"2024-05-09T10:50:21.183306+0000","last_peered":"2024-05-09T10:50:21.183306+0000","last_clean":"2024-05-09T10:50:21.183306+0000","last_became_active":"2024-05-09T10:50:20.189423+0000","last_became_peered":"2024-05-09T10:50:20.189423+0000","last_unstale":"2024-05-09T10:50:21.183306+0000","last_undegraded":"2024-05-09T10:50:21.183306+0000","last_fullsized":"2024-05-09T10:50:21.183306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:40.421893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183073+0000","last_change":"2024-05-09T10:50:19.174912+0000","last_active":"2024-05-09T10:50:21.183073+0000","last_peered":"2024-05-09T10:50:21.183073+0000","last_clean":"2024-05-09T10:50:21.183073+0000","last_became_active":"2024-05-09T10:50:19.174640+0000","last_became_peered":"2024-05-09T10:50:19.174640+0000","last_unstale":"2024-05-09T10:50:21.183073+0000","last_undegraded":"2024-05-09T10:50:21.183073+0000","last_fullsized":"2024-05-09T10:50:21.183073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:53.042012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843648+0000","last_change":"2024-05-09T10:50:20.190046+0000","last_active":"2024-05-09T10:50:23.843648+0000","last_peered":"2024-05-09T10:50:23.843648+0000","last_clean":"2024-05-09T10:50:23.843648+0000","last_became_active":"2024-05-09T10:50:20.189778+0000","last_became_peered":"2024-05-09T10:50:20.189778+0000","last_unstale":"2024-05-09T10:50:23.843648+0000","last_undegraded":"2024-05-09T10:50:23.843648+0000","last_fullsized":"2024-05-09T10:50:23.843648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:47.420832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182837+0000","last_change":"2024-05-09T10:50:19.172738+0000","last_active":"2024-05-09T10:50:21.182837+0000","last_peered":"2024-05-09T10:50:21.182837+0000","last_clean":"2024-05-09T10:50:21.182837+0000","last_became_active":"2024-05-09T10:50:19.172449+0000","last_became_peered":"2024-05-09T10:50:19.172449+0000","last_unstale":"2024-05-09T10:50:21.182837+0000","last_undegraded":"2024-05-09T10:50:21.182837+0000","last_fullsized":"2024-05-09T10:50:21.182837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:20.661093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843762+0000","last_change":"2024-05-09T10:50:20.193228+0000","last_active":"2024-05-09T10:50:23.843762+0000","last_peered":"2024-05-09T10:50:23.843762+0000","last_clean":"2024-05-09T10:50:23.843762+0000","last_became_active":"2024-05-09T10:50:20.192454+0000","last_became_peered":"2024-05-09T10:50:20.192454+0000","last_unstale":"2024-05-09T10:50:23.843762+0000","last_undegraded":"2024-05-09T10:50:23.843762+0000","last_fullsized":"2024-05-09T10:50:23.843762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:34:20.711637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889326+0000","last_change":"2024-05-09T10:50:19.174967+0000","last_active":"2024-05-09T10:50:21.889326+0000","last_peered":"2024-05-09T10:50:21.889326+0000","last_clean":"2024-05-09T10:50:21.889326+0000","last_became_active":"2024-05-09T10:50:19.173603+0000","last_became_peered":"2024-05-09T10:50:19.173603+0000","last_unstale":"2024-05-09T10:50:21.889326+0000","last_undegraded":"2024-05-09T10:50:21.889326+0000","last_fullsized":"2024-05-09T10:50:21.889326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:16:52.160962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171552+0000","last_change":"2024-05-09T10:50:20.187976+0000","last_active":"2024-05-09T10:50:22.171552+0000","last_peered":"2024-05-09T10:50:22.171552+0000","last_clean":"2024-05-09T10:50:22.171552+0000","last_became_active":"2024-05-09T10:50:20.187661+0000","last_became_peered":"2024-05-09T10:50:20.187661+0000","last_unstale":"2024-05-09T10:50:22.171552+0000","last_undegraded":"2024-05-09T10:50:22.171552+0000","last_fullsized":"2024-05-09T10:50:22.171552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:45.067992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.638210+0000","last_change":"2024-05-09T10:50:20.181807+0000","last_active":"2024-05-09T10:50:22.638210+0000","last_peered":"2024-05-09T10:50:22.638210+0000","last_clean":"2024-05-09T10:50:22.638210+0000","last_became_active":"2024-05-09T10:50:20.181212+0000","last_became_peered":"2024-05-09T10:50:20.181212+0000","last_unstale":"2024-05-09T10:50:22.638210+0000","last_undegraded":"2024-05-09T10:50:22.638210+0000","last_fullsized":"2024-05-09T10:50:22.638210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:39.852439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889187+0000","last_change":"2024-05-09T10:50:19.175398+0000","last_active":"2024-05-09T10:50:21.889187+0000","last_peered":"2024-05-09T10:50:21.889187+0000","last_clean":"2024-05-09T10:50:21.889187+0000","last_became_active":"2024-05-09T10:50:19.174893+0000","last_became_peered":"2024-05-09T10:50:19.174893+0000","last_unstale":"2024-05-09T10:50:21.889187+0000","last_undegraded":"2024-05-09T10:50:21.889187+0000","last_fullsized":"2024-05-09T10:50:21.889187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:07:59.570026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.638044+0000","last_change":"2024-05-09T10:50:20.183971+0000","last_active":"2024-05-09T10:50:22.638044+0000","last_peered":"2024-05-09T10:50:22.638044+0000","last_clean":"2024-05-09T10:50:22.638044+0000","last_became_active":"2024-05-09T10:50:20.183666+0000","last_became_peered":"2024-05-09T10:50:20.183666+0000","last_unstale":"2024-05-09T10:50:22.638044+0000","last_undegraded":"2024-05-09T10:50:22.638044+0000","last_fullsized":"2024-05-09T10:50:22.638044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:33:13.347174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889464+0000","last_change":"2024-05-09T10:50:19.169800+0000","last_active":"2024-05-09T10:50:21.889464+0000","last_peered":"2024-05-09T10:50:21.889464+0000","last_clean":"2024-05-09T10:50:21.889464+0000","last_became_active":"2024-05-09T10:50:19.169521+0000","last_became_peered":"2024-05-09T10:50:19.169521+0000","last_unstale":"2024-05-09T10:50:21.889464+0000","last_undegraded":"2024-05-09T10:50:21.889464+0000","last_fullsized":"2024-05-09T10:50:21.889464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:37.762424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183081+0000","last_change":"2024-05-09T10:50:20.181912+0000","last_active":"2024-05-09T10:50:21.183081+0000","last_peered":"2024-05-09T10:50:21.183081+0000","last_clean":"2024-05-09T10:50:21.183081+0000","last_became_active":"2024-05-09T10:50:20.181615+0000","last_became_peered":"2024-05-09T10:50:20.181615+0000","last_unstale":"2024-05-09T10:50:21.183081+0000","last_undegraded":"2024-05-09T10:50:21.183081+0000","last_fullsized":"2024-05-09T10:50:21.183081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:50:14.919494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843989+0000","last_change":"2024-05-09T10:50:19.165320+0000","last_active":"2024-05-09T10:50:23.843989+0000","last_peered":"2024-05-09T10:50:23.843989+0000","last_clean":"2024-05-09T10:50:23.843989+0000","last_became_active":"2024-05-09T10:50:19.165033+0000","last_became_peered":"2024-05-09T10:50:19.165033+0000","last_unstale":"2024-05-09T10:50:23.843989+0000","last_undegraded":"2024-05-09T10:50:23.843989+0000","last_fullsized":"2024-05-09T10:50:23.843989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:11:05.623356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843981+0000","last_change":"2024-05-09T10:50:20.188606+0000","last_active":"2024-05-09T10:50:23.843981+0000","last_peered":"2024-05-09T10:50:23.843981+0000","last_clean":"2024-05-09T10:50:23.843981+0000","last_became_active":"2024-05-09T10:50:20.188300+0000","last_became_peered":"2024-05-09T10:50:20.188300+0000","last_unstale":"2024-05-09T10:50:23.843981+0000","last_undegraded":"2024-05-09T10:50:23.843981+0000","last_fullsized":"2024-05-09T10:50:23.843981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:26:34.638740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183175+0000","last_change":"2024-05-09T10:50:19.168206+0000","last_active":"2024-05-09T10:50:21.183175+0000","last_peered":"2024-05-09T10:50:21.183175+0000","last_clean":"2024-05-09T10:50:21.183175+0000","last_became_active":"2024-05-09T10:50:19.167924+0000","last_became_peered":"2024-05-09T10:50:19.167924+0000","last_unstale":"2024-05-09T10:50:21.183175+0000","last_undegraded":"2024-05-09T10:50:21.183175+0000","last_fullsized":"2024-05-09T10:50:21.183175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:11:05.623356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T10:50:27.437 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T10:50:27.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.580+0000 7f370f73b640 1 Processor -- start 2024-05-09T10:50:27.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.580+0000 7f370f73b640 1 -- start start 2024-05-09T10:50:27.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.581+0000 7f370f73b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f370815f4e0 0x7f37080ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:27.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.581+0000 7f370f73b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f37080b09e0 0x7f3708165c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:27.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.581+0000 7f370f73b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37080aeda0 con 0x7f37080afe40 2024-05-09T10:50:27.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.581+0000 7f370f73b640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37080af080 con 0x7f370815f4e0 2024-05-09T10:50:27.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.581+0000 7f370f73b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37080af360 con 0x7f37080b09e0 2024-05-09T10:50:27.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.581+0000 7f370d4d7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f370815f4e0 0x7f37080ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:27.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.581+0000 7f370d4d7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f370815f4e0 0x7f37080ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42286/0 (socket says 172.21.15.98:42286) 2024-05-09T10:50:27.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.581+0000 7f370d4d7640 1 -- 172.21.15.98:0/3797637505 learned_addr learned my addr 172.21.15.98:0/3797637505 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:27.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.581+0000 7f370dcd8640 1 --2- 172.21.15.98:0/3797637505 >> v2:172.21.15.105:3300/0 conn(0x7f37080b09e0 0x7f3708165c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:27.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.581+0000 7f370d4d7640 1 -- 172.21.15.98:0/3797637505 >> v2:172.21.15.105:3300/0 conn(0x7f37080b09e0 msgr2=0x7f3708165c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:27.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.581+0000 7f370d4d7640 1 --2- 172.21.15.98:0/3797637505 >> v2:172.21.15.105:3300/0 conn(0x7f37080b09e0 0x7f3708165c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.581+0000 7f370d4d7640 1 -- 172.21.15.98:0/3797637505 >> v1:172.21.15.105:6789/0 conn(0x7f37080afe40 legacy=0x7f37080b02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:27.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.581+0000 7f370d4d7640 1 -- 172.21.15.98:0/3797637505 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37080af640 con 0x7f370815f4e0 2024-05-09T10:50:27.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.581+0000 7f370dcd8640 1 --2- 172.21.15.98:0/3797637505 >> v2:172.21.15.105:3300/0 conn(0x7f37080b09e0 0x7f3708165c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:27.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.582+0000 7f370d4d7640 1 --2- 172.21.15.98:0/3797637505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f370815f4e0 0x7f37080ae7c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f370806b810 tx=0x7f370402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70200e9f50c193dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:27.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.582+0000 7f36fffff640 1 -- 172.21.15.98:0/3797637505 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3704037070 con 0x7f370815f4e0 2024-05-09T10:50:27.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.582+0000 7f36fffff640 1 -- 172.21.15.98:0/3797637505 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3704032440 con 0x7f370815f4e0 2024-05-09T10:50:27.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.582+0000 7f36fffff640 1 -- 172.21.15.98:0/3797637505 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3704032810 con 0x7f370815f4e0 2024-05-09T10:50:27.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.583+0000 7f370f73b640 1 -- 172.21.15.98:0/3797637505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f370815f4e0 msgr2=0x7f37080ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.583+0000 7f370f73b640 1 --2- 172.21.15.98:0/3797637505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f370815f4e0 0x7f37080ae7c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f370806b810 tx=0x7f370402a1e0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.583+0000 7f370f73b640 1 -- 172.21.15.98:0/3797637505 shutdown_connections 2024-05-09T10:50:27.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.583+0000 7f370f73b640 1 --2- 172.21.15.98:0/3797637505 >> v2:172.21.15.105:3300/0 conn(0x7f37080b09e0 0x7f3708165c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.583+0000 7f370f73b640 1 --2- 172.21.15.98:0/3797637505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f370815f4e0 0x7f37080ae7c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.583+0000 7f370f73b640 1 -- 172.21.15.98:0/3797637505 >> 172.21.15.98:0/3797637505 conn(0x7f37080b34e0 msgr2=0x7f37080b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:27.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.583+0000 7f370f73b640 1 -- 172.21.15.98:0/3797637505 shutdown_connections 2024-05-09T10:50:27.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.583+0000 7f370f73b640 1 -- 172.21.15.98:0/3797637505 wait complete. 2024-05-09T10:50:27.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.584+0000 7f370f73b640 1 Processor -- start 2024-05-09T10:50:27.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.584+0000 7f370f73b640 1 -- start start 2024-05-09T10:50:27.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.585+0000 7f370f73b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37080afe40 0x7f37080b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:27.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.585+0000 7f370f73b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f37080b09e0 0x7f3708156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:27.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.585+0000 7f370f73b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f370815f4e0 0x7f370815b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:27.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.585+0000 7f370f73b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370815f8c0 con 0x7f37080afe40 2024-05-09T10:50:27.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.585+0000 7f370f73b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37080cbcf0 con 0x7f370815f4e0 2024-05-09T10:50:27.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.585+0000 7f370f73b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37080b0df0 con 0x7f37080b09e0 2024-05-09T10:50:27.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.585+0000 7f370d4d7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37080afe40 0x7f37080b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:27.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.585+0000 7f370d4d7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37080afe40 0x7f37080b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42288/0 (socket says 172.21.15.98:42288) 2024-05-09T10:50:27.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.585+0000 7f370d4d7640 1 -- 172.21.15.98:0/3507681030 learned_addr learned my addr 172.21.15.98:0/3507681030 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:27.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.585+0000 7f370d4d7640 1 -- 172.21.15.98:0/3507681030 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f37080b09e0 msgr2=0x7f3708156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.585+0000 7f370d4d7640 1 --2- 172.21.15.98:0/3507681030 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f37080b09e0 0x7f3708156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.585+0000 7f370d4d7640 1 -- 172.21.15.98:0/3507681030 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f370815f4e0 msgr2=0x7f370815b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.585+0000 7f370d4d7640 1 --2- 172.21.15.98:0/3507681030 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f370815f4e0 0x7f370815b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.585+0000 7f370d4d7640 1 -- 172.21.15.98:0/3507681030 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370806c8a0 con 0x7f37080afe40 2024-05-09T10:50:27.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.585+0000 7f370d4d7640 1 --2- 172.21.15.98:0/3507681030 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37080afe40 0x7f37080b4d50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3704000c00 tx=0x7f37040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:27.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.585+0000 7f36fdffb640 1 -- 172.21.15.98:0/3507681030 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3704037040 con 0x7f37080afe40 2024-05-09T10:50:27.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.585+0000 7f36fdffb640 1 -- 172.21.15.98:0/3507681030 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3704042070 con 0x7f37080afe40 2024-05-09T10:50:27.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.585+0000 7f370f73b640 1 -- 172.21.15.98:0/3507681030 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370806cb80 con 0x7f37080afe40 2024-05-09T10:50:27.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.586+0000 7f36fdffb640 1 -- 172.21.15.98:0/3507681030 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3704030bb0 con 0x7f37080afe40 2024-05-09T10:50:27.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.586+0000 7f370f73b640 1 -- 172.21.15.98:0/3507681030 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37080c9c20 con 0x7f37080afe40 2024-05-09T10:50:27.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.587+0000 7f370f73b640 1 -- 172.21.15.98:0/3507681030 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f370815e880 con 0x7f37080afe40 2024-05-09T10:50:27.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.587+0000 7f36fdffb640 1 -- 172.21.15.98:0/3507681030 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3704047460 con 0x7f37080afe40 2024-05-09T10:50:27.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.588+0000 7f36fdffb640 1 --2- 172.21.15.98:0/3507681030 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f36ec06b4d0 0x7f36ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:27.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.588+0000 7f370ccd6640 1 --2- 172.21.15.98:0/3507681030 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f36ec06b4d0 0x7f36ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:27.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.588+0000 7f36fdffb640 1 -- 172.21.15.98:0/3507681030 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37040aade0 con 0x7f37080afe40 2024-05-09T10:50:27.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.588+0000 7f370ccd6640 1 --2- 172.21.15.98:0/3507681030 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f36ec06b4d0 0x7f36ec06d990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f36f8000f60 tx=0x7f36f80079d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:27.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.592+0000 7f36fdffb640 1 -- 172.21.15.98:0/3507681030 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3704079cf0 con 0x7f37080afe40 2024-05-09T10:50:27.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.756+0000 7f370f73b640 1 -- 172.21.15.98:0/3507681030 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f370806d0a0 con 0x7f36ec06b4d0 2024-05-09T10:50:27.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.762+0000 7f36fdffb640 1 -- 172.21.15.98:0/3507681030 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251872 (secure 0 0 0) 0x7f370806d0a0 con 0x7f36ec06b4d0 2024-05-09T10:50:27.758 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:27.760 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-05-09T10:50:27.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.765+0000 7f370f73b640 1 -- 172.21.15.98:0/3507681030 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f36ec06b4d0 msgr2=0x7f36ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.765+0000 7f370f73b640 1 --2- 172.21.15.98:0/3507681030 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f36ec06b4d0 0x7f36ec06d990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f36f8000f60 tx=0x7f36f80079d0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.765+0000 7f370f73b640 1 -- 172.21.15.98:0/3507681030 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37080afe40 msgr2=0x7f37080b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.765+0000 7f370f73b640 1 --2- 172.21.15.98:0/3507681030 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37080afe40 0x7f37080b4d50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3704000c00 tx=0x7f37040046c0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.765+0000 7f370f73b640 1 -- 172.21.15.98:0/3507681030 shutdown_connections 2024-05-09T10:50:27.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.765+0000 7f370f73b640 1 --2- 172.21.15.98:0/3507681030 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f36ec06b4d0 0x7f36ec06d990 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.765+0000 7f370f73b640 1 --2- 172.21.15.98:0/3507681030 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f370815f4e0 0x7f370815b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.765+0000 7f370f73b640 1 --2- 172.21.15.98:0/3507681030 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f37080b09e0 0x7f3708156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.765+0000 7f370f73b640 1 --2- 172.21.15.98:0/3507681030 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37080afe40 0x7f37080b4d50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.765+0000 7f370f73b640 1 -- 172.21.15.98:0/3507681030 >> 172.21.15.98:0/3507681030 conn(0x7f37080b34e0 msgr2=0x7f370815cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:27.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.765+0000 7f370f73b640 1 -- 172.21.15.98:0/3507681030 shutdown_connections 2024-05-09T10:50:27.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.765+0000 7f370f73b640 1 -- 172.21.15.98:0/3507681030 wait complete. 2024-05-09T10:50:27.769 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-05-09T10:50:27.367243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001561"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889511+0000","last_change":"2024-05-09T10:50:20.202000+0000","last_active":"2024-05-09T10:50:21.889511+0000","last_peered":"2024-05-09T10:50:21.889511+0000","last_clean":"2024-05-09T10:50:21.889511+0000","last_became_active":"2024-05-09T10:50:20.201722+0000","last_became_peered":"2024-05-09T10:50:20.201722+0000","last_unstale":"2024-05-09T10:50:21.889511+0000","last_undegraded":"2024-05-09T10:50:21.889511+0000","last_fullsized":"2024-05-09T10:50:21.889511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:51:23.834797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182607+0000","last_change":"2024-05-09T10:50:20.185947+0000","last_active":"2024-05-09T10:50:21.182607+0000","last_peered":"2024-05-09T10:50:21.182607+0000","last_clean":"2024-05-09T10:50:21.182607+0000","last_became_active":"2024-05-09T10:50:20.185672+0000","last_became_peered":"2024-05-09T10:50:20.185672+0000","last_unstale":"2024-05-09T10:50:21.182607+0000","last_undegraded":"2024-05-09T10:50:21.182607+0000","last_fullsized":"2024-05-09T10:50:21.182607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:45.067992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182555+0000","last_change":"2024-05-09T10:50:20.186891+0000","last_active":"2024-05-09T10:50:21.182555+0000","last_peered":"2024-05-09T10:50:21.182555+0000","last_clean":"2024-05-09T10:50:21.182555+0000","last_became_active":"2024-05-09T10:50:20.186605+0000","last_became_peered":"2024-05-09T10:50:20.186605+0000","last_unstale":"2024-05-09T10:50:21.182555+0000","last_undegraded":"2024-05-09T10:50:21.182555+0000","last_fullsized":"2024-05-09T10:50:21.182555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:37:30.799640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.638238+0000","last_change":"2024-05-09T10:50:20.195509+0000","last_active":"2024-05-09T10:50:22.638238+0000","last_peered":"2024-05-09T10:50:22.638238+0000","last_clean":"2024-05-09T10:50:22.638238+0000","last_became_active":"2024-05-09T10:50:20.195045+0000","last_became_peered":"2024-05-09T10:50:20.195045+0000","last_unstale":"2024-05-09T10:50:22.638238+0000","last_undegraded":"2024-05-09T10:50:22.638238+0000","last_fullsized":"2024-05-09T10:50:22.638238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:31:09.517362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889170+0000","last_change":"2024-05-09T10:50:20.198458+0000","last_active":"2024-05-09T10:50:21.889170+0000","last_peered":"2024-05-09T10:50:21.889170+0000","last_clean":"2024-05-09T10:50:21.889170+0000","last_became_active":"2024-05-09T10:50:20.195845+0000","last_became_peered":"2024-05-09T10:50:20.195845+0000","last_unstale":"2024-05-09T10:50:21.889170+0000","last_undegraded":"2024-05-09T10:50:21.889170+0000","last_fullsized":"2024-05-09T10:50:21.889170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:31:09.517362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.637784+0000","last_change":"2024-05-09T10:50:20.179665+0000","last_active":"2024-05-09T10:50:22.637784+0000","last_peered":"2024-05-09T10:50:22.637784+0000","last_clean":"2024-05-09T10:50:22.637784+0000","last_became_active":"2024-05-09T10:50:20.179379+0000","last_became_peered":"2024-05-09T10:50:20.179379+0000","last_unstale":"2024-05-09T10:50:22.637784+0000","last_undegraded":"2024-05-09T10:50:22.637784+0000","last_fullsized":"2024-05-09T10:50:22.637784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:26:34.638740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.888989+0000","last_change":"2024-05-09T10:50:20.199853+0000","last_active":"2024-05-09T10:50:21.888989+0000","last_peered":"2024-05-09T10:50:21.888989+0000","last_clean":"2024-05-09T10:50:21.888989+0000","last_became_active":"2024-05-09T10:50:20.199552+0000","last_became_peered":"2024-05-09T10:50:20.199552+0000","last_unstale":"2024-05-09T10:50:21.888989+0000","last_undegraded":"2024-05-09T10:50:21.888989+0000","last_fullsized":"2024-05-09T10:50:21.888989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:09.785658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843477+0000","last_change":"2024-05-09T10:50:20.191734+0000","last_active":"2024-05-09T10:50:23.843477+0000","last_peered":"2024-05-09T10:50:23.843477+0000","last_clean":"2024-05-09T10:50:23.843477+0000","last_became_active":"2024-05-09T10:50:20.191484+0000","last_became_peered":"2024-05-09T10:50:20.191484+0000","last_unstale":"2024-05-09T10:50:23.843477+0000","last_undegraded":"2024-05-09T10:50:23.843477+0000","last_fullsized":"2024-05-09T10:50:23.843477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:32:42.671953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889473+0000","last_change":"2024-05-09T10:50:20.199646+0000","last_active":"2024-05-09T10:50:21.889473+0000","last_peered":"2024-05-09T10:50:21.889473+0000","last_clean":"2024-05-09T10:50:21.889473+0000","last_became_active":"2024-05-09T10:50:20.199374+0000","last_became_peered":"2024-05-09T10:50:20.199374+0000","last_unstale":"2024-05-09T10:50:21.889473+0000","last_undegraded":"2024-05-09T10:50:21.889473+0000","last_fullsized":"2024-05-09T10:50:21.889473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:27:14.733561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171869+0000","last_change":"2024-05-09T10:50:20.189694+0000","last_active":"2024-05-09T10:50:22.171869+0000","last_peered":"2024-05-09T10:50:22.171869+0000","last_clean":"2024-05-09T10:50:22.171869+0000","last_became_active":"2024-05-09T10:50:20.189429+0000","last_became_peered":"2024-05-09T10:50:20.189429+0000","last_unstale":"2024-05-09T10:50:22.171869+0000","last_undegraded":"2024-05-09T10:50:22.171869+0000","last_fullsized":"2024-05-09T10:50:22.171869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:50:14.919494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171776+0000","last_change":"2024-05-09T10:50:20.189569+0000","last_active":"2024-05-09T10:50:22.171776+0000","last_peered":"2024-05-09T10:50:22.171776+0000","last_clean":"2024-05-09T10:50:22.171776+0000","last_became_active":"2024-05-09T10:50:20.189272+0000","last_became_peered":"2024-05-09T10:50:20.189272+0000","last_unstale":"2024-05-09T10:50:22.171776+0000","last_undegraded":"2024-05-09T10:50:22.171776+0000","last_fullsized":"2024-05-09T10:50:22.171776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:47.420832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183678+0000","last_change":"2024-05-09T10:50:20.191736+0000","last_active":"2024-05-09T10:50:21.183678+0000","last_peered":"2024-05-09T10:50:21.183678+0000","last_clean":"2024-05-09T10:50:21.183678+0000","last_became_active":"2024-05-09T10:50:20.190652+0000","last_became_peered":"2024-05-09T10:50:20.190652+0000","last_unstale":"2024-05-09T10:50:21.183678+0000","last_undegraded":"2024-05-09T10:50:21.183678+0000","last_fullsized":"2024-05-09T10:50:21.183678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:08:00.575142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843704+0000","last_change":"2024-05-09T10:50:20.192354+0000","last_active":"2024-05-09T10:50:23.843704+0000","last_peered":"2024-05-09T10:50:23.843704+0000","last_clean":"2024-05-09T10:50:23.843704+0000","last_became_active":"2024-05-09T10:50:20.191736+0000","last_became_peered":"2024-05-09T10:50:20.191736+0000","last_unstale":"2024-05-09T10:50:23.843704+0000","last_undegraded":"2024-05-09T10:50:23.843704+0000","last_fullsized":"2024-05-09T10:50:23.843704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:50:14.919494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182518+0000","last_change":"2024-05-09T10:50:20.184216+0000","last_active":"2024-05-09T10:50:21.182518+0000","last_peered":"2024-05-09T10:50:21.182518+0000","last_clean":"2024-05-09T10:50:21.182518+0000","last_became_active":"2024-05-09T10:50:20.183920+0000","last_became_peered":"2024-05-09T10:50:20.183920+0000","last_unstale":"2024-05-09T10:50:21.182518+0000","last_undegraded":"2024-05-09T10:50:21.182518+0000","last_fullsized":"2024-05-09T10:50:21.182518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:57:07.297876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183244+0000","last_change":"2024-05-09T10:50:20.189990+0000","last_active":"2024-05-09T10:50:21.183244+0000","last_peered":"2024-05-09T10:50:21.183244+0000","last_clean":"2024-05-09T10:50:21.183244+0000","last_became_active":"2024-05-09T10:50:20.189727+0000","last_became_peered":"2024-05-09T10:50:20.189727+0000","last_unstale":"2024-05-09T10:50:21.183244+0000","last_undegraded":"2024-05-09T10:50:21.183244+0000","last_fullsized":"2024-05-09T10:50:21.183244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:34:20.711637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.637718+0000","last_change":"2024-05-09T10:50:20.179735+0000","last_active":"2024-05-09T10:50:22.637718+0000","last_peered":"2024-05-09T10:50:22.637718+0000","last_clean":"2024-05-09T10:50:22.637718+0000","last_became_active":"2024-05-09T10:50:20.179440+0000","last_became_peered":"2024-05-09T10:50:20.179440+0000","last_unstale":"2024-05-09T10:50:22.637718+0000","last_undegraded":"2024-05-09T10:50:22.637718+0000","last_fullsized":"2024-05-09T10:50:22.637718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:34:20.711637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.638388+0000","last_change":"2024-05-09T10:50:20.194956+0000","last_active":"2024-05-09T10:50:22.638388+0000","last_peered":"2024-05-09T10:50:22.638388+0000","last_clean":"2024-05-09T10:50:22.638388+0000","last_became_active":"2024-05-09T10:50:20.194660+0000","last_became_peered":"2024-05-09T10:50:20.194660+0000","last_unstale":"2024-05-09T10:50:22.638388+0000","last_undegraded":"2024-05-09T10:50:22.638388+0000","last_fullsized":"2024-05-09T10:50:22.638388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:07:48.935082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.264445+0000","last_change":"2024-05-09T10:50:20.188140+0000","last_active":"2024-05-09T10:50:23.264445+0000","last_peered":"2024-05-09T10:50:23.264445+0000","last_clean":"2024-05-09T10:50:23.264445+0000","last_became_active":"2024-05-09T10:50:20.187855+0000","last_became_peered":"2024-05-09T10:50:20.187855+0000","last_unstale":"2024-05-09T10:50:23.264445+0000","last_undegraded":"2024-05-09T10:50:23.264445+0000","last_fullsized":"2024-05-09T10:50:23.264445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:16:53.166077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889478+0000","last_change":"2024-05-09T10:50:20.199567+0000","last_active":"2024-05-09T10:50:21.889478+0000","last_peered":"2024-05-09T10:50:21.889478+0000","last_clean":"2024-05-09T10:50:21.889478+0000","last_became_active":"2024-05-09T10:50:20.198671+0000","last_became_peered":"2024-05-09T10:50:20.198671+0000","last_unstale":"2024-05-09T10:50:21.889478+0000","last_undegraded":"2024-05-09T10:50:21.889478+0000","last_fullsized":"2024-05-09T10:50:21.889478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:03:14.102727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171768+0000","last_change":"2024-05-09T10:50:20.188963+0000","last_active":"2024-05-09T10:50:22.171768+0000","last_peered":"2024-05-09T10:50:22.171768+0000","last_clean":"2024-05-09T10:50:22.171768+0000","last_became_active":"2024-05-09T10:50:20.188654+0000","last_became_peered":"2024-05-09T10:50:20.188654+0000","last_unstale":"2024-05-09T10:50:22.171768+0000","last_undegraded":"2024-05-09T10:50:22.171768+0000","last_fullsized":"2024-05-09T10:50:22.171768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:23.176619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182825+0000","last_change":"2024-05-09T10:50:20.188028+0000","last_active":"2024-05-09T10:50:21.182825+0000","last_peered":"2024-05-09T10:50:21.182825+0000","last_clean":"2024-05-09T10:50:21.182825+0000","last_became_active":"2024-05-09T10:50:20.186949+0000","last_became_peered":"2024-05-09T10:50:20.186949+0000","last_unstale":"2024-05-09T10:50:21.182825+0000","last_undegraded":"2024-05-09T10:50:21.182825+0000","last_fullsized":"2024-05-09T10:50:21.182825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:00:03.184256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.637880+0000","last_change":"2024-05-09T10:50:20.187160+0000","last_active":"2024-05-09T10:50:22.637880+0000","last_peered":"2024-05-09T10:50:22.637880+0000","last_clean":"2024-05-09T10:50:22.637880+0000","last_became_active":"2024-05-09T10:50:20.186840+0000","last_became_peered":"2024-05-09T10:50:20.186840+0000","last_unstale":"2024-05-09T10:50:22.637880+0000","last_undegraded":"2024-05-09T10:50:22.637880+0000","last_fullsized":"2024-05-09T10:50:22.637880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:33:19.903309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843512+0000","last_change":"2024-05-09T10:50:20.188532+0000","last_active":"2024-05-09T10:50:23.843512+0000","last_peered":"2024-05-09T10:50:23.843512+0000","last_clean":"2024-05-09T10:50:23.843512+0000","last_became_active":"2024-05-09T10:50:20.188264+0000","last_became_peered":"2024-05-09T10:50:20.188264+0000","last_unstale":"2024-05-09T10:50:23.843512+0000","last_undegraded":"2024-05-09T10:50:23.843512+0000","last_fullsized":"2024-05-09T10:50:23.843512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:00:03.184256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.637786+0000","last_change":"2024-05-09T10:50:20.194256+0000","last_active":"2024-05-09T10:50:22.637786+0000","last_peered":"2024-05-09T10:50:22.637786+0000","last_clean":"2024-05-09T10:50:22.637786+0000","last_became_active":"2024-05-09T10:50:20.193137+0000","last_became_peered":"2024-05-09T10:50:20.193137+0000","last_unstale":"2024-05-09T10:50:22.637786+0000","last_undegraded":"2024-05-09T10:50:22.637786+0000","last_fullsized":"2024-05-09T10:50:22.637786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:02:11.043966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.844007+0000","last_change":"2024-05-09T10:50:20.189585+0000","last_active":"2024-05-09T10:50:23.844007+0000","last_peered":"2024-05-09T10:50:23.844007+0000","last_clean":"2024-05-09T10:50:23.844007+0000","last_became_active":"2024-05-09T10:50:20.189295+0000","last_became_peered":"2024-05-09T10:50:20.189295+0000","last_unstale":"2024-05-09T10:50:23.844007+0000","last_undegraded":"2024-05-09T10:50:23.844007+0000","last_fullsized":"2024-05-09T10:50:23.844007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:33:19.903309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171966+0000","last_change":"2024-05-09T10:50:20.196303+0000","last_active":"2024-05-09T10:50:22.171966+0000","last_peered":"2024-05-09T10:50:22.171966+0000","last_clean":"2024-05-09T10:50:22.171966+0000","last_became_active":"2024-05-09T10:50:20.195986+0000","last_became_peered":"2024-05-09T10:50:20.195986+0000","last_unstale":"2024-05-09T10:50:22.171966+0000","last_undegraded":"2024-05-09T10:50:22.171966+0000","last_fullsized":"2024-05-09T10:50:22.171966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:34:20.711637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182978+0000","last_change":"2024-05-09T10:50:20.193168+0000","last_active":"2024-05-09T10:50:21.182978+0000","last_peered":"2024-05-09T10:50:21.182978+0000","last_clean":"2024-05-09T10:50:21.182978+0000","last_became_active":"2024-05-09T10:50:20.192898+0000","last_became_peered":"2024-05-09T10:50:20.192898+0000","last_unstale":"2024-05-09T10:50:21.182978+0000","last_undegraded":"2024-05-09T10:50:21.182978+0000","last_fullsized":"2024-05-09T10:50:21.182978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:50:14.919494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843887+0000","last_change":"2024-05-09T10:50:20.188735+0000","last_active":"2024-05-09T10:50:23.843887+0000","last_peered":"2024-05-09T10:50:23.843887+0000","last_clean":"2024-05-09T10:50:23.843887+0000","last_became_active":"2024-05-09T10:50:20.188138+0000","last_became_peered":"2024-05-09T10:50:20.188138+0000","last_unstale":"2024-05-09T10:50:23.843887+0000","last_undegraded":"2024-05-09T10:50:23.843887+0000","last_fullsized":"2024-05-09T10:50:23.843887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:02:11.043966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182945+0000","last_change":"2024-05-09T10:50:20.194279+0000","last_active":"2024-05-09T10:50:21.182945+0000","last_peered":"2024-05-09T10:50:21.182945+0000","last_clean":"2024-05-09T10:50:21.182945+0000","last_became_active":"2024-05-09T10:50:20.193962+0000","last_became_peered":"2024-05-09T10:50:20.193962+0000","last_unstale":"2024-05-09T10:50:21.182945+0000","last_undegraded":"2024-05-09T10:50:21.182945+0000","last_fullsized":"2024-05-09T10:50:21.182945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:23.176619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.638158+0000","last_change":"2024-05-09T10:50:20.193350+0000","last_active":"2024-05-09T10:50:22.638158+0000","last_peered":"2024-05-09T10:50:22.638158+0000","last_clean":"2024-05-09T10:50:22.638158+0000","last_became_active":"2024-05-09T10:50:20.192514+0000","last_became_peered":"2024-05-09T10:50:20.192514+0000","last_unstale":"2024-05-09T10:50:22.638158+0000","last_undegraded":"2024-05-09T10:50:22.638158+0000","last_fullsized":"2024-05-09T10:50:22.638158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:09.785658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.264333+0000","last_change":"2024-05-09T10:50:20.175999+0000","last_active":"2024-05-09T10:50:23.264333+0000","last_peered":"2024-05-09T10:50:23.264333+0000","last_clean":"2024-05-09T10:50:23.264333+0000","last_became_active":"2024-05-09T10:50:20.175695+0000","last_became_peered":"2024-05-09T10:50:20.175695+0000","last_unstale":"2024-05-09T10:50:23.264333+0000","last_undegraded":"2024-05-09T10:50:23.264333+0000","last_fullsized":"2024-05-09T10:50:23.264333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:08:00.575142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171539+0000","last_change":"2024-05-09T10:50:20.196639+0000","last_active":"2024-05-09T10:50:22.171539+0000","last_peered":"2024-05-09T10:50:22.171539+0000","last_clean":"2024-05-09T10:50:22.171539+0000","last_became_active":"2024-05-09T10:50:20.196365+0000","last_became_peered":"2024-05-09T10:50:20.196365+0000","last_unstale":"2024-05-09T10:50:22.171539+0000","last_undegraded":"2024-05-09T10:50:22.171539+0000","last_fullsized":"2024-05-09T10:50:22.171539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:32:42.671953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183207+0000","last_change":"2024-05-09T10:50:20.185161+0000","last_active":"2024-05-09T10:50:21.183207+0000","last_peered":"2024-05-09T10:50:21.183207+0000","last_clean":"2024-05-09T10:50:21.183207+0000","last_became_active":"2024-05-09T10:50:20.184832+0000","last_became_peered":"2024-05-09T10:50:20.184832+0000","last_unstale":"2024-05-09T10:50:21.183207+0000","last_undegraded":"2024-05-09T10:50:21.183207+0000","last_fullsized":"2024-05-09T10:50:21.183207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:21.666208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843791+0000","last_change":"2024-05-09T10:50:19.173345+0000","last_active":"2024-05-09T10:50:23.843791+0000","last_peered":"2024-05-09T10:50:23.843791+0000","last_clean":"2024-05-09T10:50:23.843791+0000","last_became_active":"2024-05-09T10:50:19.173059+0000","last_became_peered":"2024-05-09T10:50:19.173059+0000","last_unstale":"2024-05-09T10:50:23.843791+0000","last_undegraded":"2024-05-09T10:50:23.843791+0000","last_fullsized":"2024-05-09T10:50:23.843791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:53.042012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183068+0000","last_change":"2024-05-09T10:50:20.188071+0000","last_active":"2024-05-09T10:50:21.183068+0000","last_peered":"2024-05-09T10:50:21.183068+0000","last_clean":"2024-05-09T10:50:21.183068+0000","last_became_active":"2024-05-09T10:50:20.187766+0000","last_became_peered":"2024-05-09T10:50:20.187766+0000","last_unstale":"2024-05-09T10:50:21.183068+0000","last_undegraded":"2024-05-09T10:50:21.183068+0000","last_fullsized":"2024-05-09T10:50:21.183068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:47.420832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171634+0000","last_change":"2024-05-09T10:50:19.169316+0000","last_active":"2024-05-09T10:50:22.171634+0000","last_peered":"2024-05-09T10:50:22.171634+0000","last_clean":"2024-05-09T10:50:22.171634+0000","last_became_active":"2024-05-09T10:50:19.169040+0000","last_became_peered":"2024-05-09T10:50:19.169040+0000","last_unstale":"2024-05-09T10:50:22.171634+0000","last_undegraded":"2024-05-09T10:50:22.171634+0000","last_fullsized":"2024-05-09T10:50:22.171634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:39.416778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843566+0000","last_change":"2024-05-09T10:50:19.167276+0000","last_active":"2024-05-09T10:50:23.843566+0000","last_peered":"2024-05-09T10:50:23.843566+0000","last_clean":"2024-05-09T10:50:23.843566+0000","last_became_active":"2024-05-09T10:50:19.166775+0000","last_became_peered":"2024-05-09T10:50:19.166775+0000","last_unstale":"2024-05-09T10:50:23.843566+0000","last_undegraded":"2024-05-09T10:50:23.843566+0000","last_fullsized":"2024-05-09T10:50:23.843566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:41:57.513979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843560+0000","last_change":"2024-05-09T10:50:20.181449+0000","last_active":"2024-05-09T10:50:23.843560+0000","last_peered":"2024-05-09T10:50:23.843560+0000","last_clean":"2024-05-09T10:50:23.843560+0000","last_became_active":"2024-05-09T10:50:20.181137+0000","last_became_peered":"2024-05-09T10:50:20.181137+0000","last_unstale":"2024-05-09T10:50:23.843560+0000","last_undegraded":"2024-05-09T10:50:23.843560+0000","last_fullsized":"2024-05-09T10:50:23.843560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:45.067992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.637867+0000","last_change":"2024-05-09T10:50:19.165083+0000","last_active":"2024-05-09T10:50:22.637867+0000","last_peered":"2024-05-09T10:50:22.637867+0000","last_clean":"2024-05-09T10:50:22.637867+0000","last_became_active":"2024-05-09T10:50:19.164468+0000","last_became_peered":"2024-05-09T10:50:19.164468+0000","last_unstale":"2024-05-09T10:50:22.637867+0000","last_undegraded":"2024-05-09T10:50:22.637867+0000","last_fullsized":"2024-05-09T10:50:22.637867+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:53.042012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182977+0000","last_change":"2024-05-09T10:50:20.193953+0000","last_active":"2024-05-09T10:50:21.182977+0000","last_peered":"2024-05-09T10:50:21.182977+0000","last_clean":"2024-05-09T10:50:21.182977+0000","last_became_active":"2024-05-09T10:50:20.193686+0000","last_became_peered":"2024-05-09T10:50:20.193686+0000","last_unstale":"2024-05-09T10:50:21.182977+0000","last_undegraded":"2024-05-09T10:50:21.182977+0000","last_fullsized":"2024-05-09T10:50:21.182977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:47.420832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171830+0000","last_change":"2024-05-09T10:50:19.164495+0000","last_active":"2024-05-09T10:50:22.171830+0000","last_peered":"2024-05-09T10:50:22.171830+0000","last_clean":"2024-05-09T10:50:22.171830+0000","last_became_active":"2024-05-09T10:50:19.164073+0000","last_became_peered":"2024-05-09T10:50:19.164073+0000","last_unstale":"2024-05-09T10:50:22.171830+0000","last_undegraded":"2024-05-09T10:50:22.171830+0000","last_fullsized":"2024-05-09T10:50:22.171830+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:53.042012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.638230+0000","last_change":"2024-05-09T10:50:20.186184+0000","last_active":"2024-05-09T10:50:22.638230+0000","last_peered":"2024-05-09T10:50:22.638230+0000","last_clean":"2024-05-09T10:50:22.638230+0000","last_became_active":"2024-05-09T10:50:20.185892+0000","last_became_peered":"2024-05-09T10:50:20.185892+0000","last_unstale":"2024-05-09T10:50:22.638230+0000","last_undegraded":"2024-05-09T10:50:22.638230+0000","last_fullsized":"2024-05-09T10:50:22.638230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:35.361899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.638218+0000","last_change":"2024-05-09T10:50:19.166015+0000","last_active":"2024-05-09T10:50:22.638218+0000","last_peered":"2024-05-09T10:50:22.638218+0000","last_clean":"2024-05-09T10:50:22.638218+0000","last_became_active":"2024-05-09T10:50:19.163525+0000","last_became_peered":"2024-05-09T10:50:19.163525+0000","last_unstale":"2024-05-09T10:50:22.638218+0000","last_undegraded":"2024-05-09T10:50:22.638218+0000","last_fullsized":"2024-05-09T10:50:22.638218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:37.762424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183206+0000","last_change":"2024-05-09T10:50:20.183493+0000","last_active":"2024-05-09T10:50:21.183206+0000","last_peered":"2024-05-09T10:50:21.183206+0000","last_clean":"2024-05-09T10:50:21.183206+0000","last_became_active":"2024-05-09T10:50:20.183222+0000","last_became_peered":"2024-05-09T10:50:20.183222+0000","last_unstale":"2024-05-09T10:50:21.183206+0000","last_undegraded":"2024-05-09T10:50:21.183206+0000","last_fullsized":"2024-05-09T10:50:21.183206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:34:20.711637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183073+0000","last_change":"2024-05-09T10:50:19.174608+0000","last_active":"2024-05-09T10:50:21.183073+0000","last_peered":"2024-05-09T10:50:21.183073+0000","last_clean":"2024-05-09T10:50:21.183073+0000","last_became_active":"2024-05-09T10:50:19.174299+0000","last_became_peered":"2024-05-09T10:50:19.174299+0000","last_unstale":"2024-05-09T10:50:21.183073+0000","last_undegraded":"2024-05-09T10:50:21.183073+0000","last_fullsized":"2024-05-09T10:50:21.183073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:49:05.717962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183616+0000","last_change":"2024-05-09T10:50:20.186118+0000","last_active":"2024-05-09T10:50:21.183616+0000","last_peered":"2024-05-09T10:50:21.183616+0000","last_clean":"2024-05-09T10:50:21.183616+0000","last_became_active":"2024-05-09T10:50:20.185818+0000","last_became_peered":"2024-05-09T10:50:20.185818+0000","last_unstale":"2024-05-09T10:50:21.183616+0000","last_undegraded":"2024-05-09T10:50:21.183616+0000","last_fullsized":"2024-05-09T10:50:21.183616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:49:06.723077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171706+0000","last_change":"2024-05-09T10:50:19.170262+0000","last_active":"2024-05-09T10:50:22.171706+0000","last_peered":"2024-05-09T10:50:22.171706+0000","last_clean":"2024-05-09T10:50:22.171706+0000","last_became_active":"2024-05-09T10:50:19.169908+0000","last_became_peered":"2024-05-09T10:50:19.169908+0000","last_unstale":"2024-05-09T10:50:22.171706+0000","last_undegraded":"2024-05-09T10:50:22.171706+0000","last_fullsized":"2024-05-09T10:50:22.171706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:37.762424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889300+0000","last_change":"2024-05-09T10:50:20.199969+0000","last_active":"2024-05-09T10:50:21.889300+0000","last_peered":"2024-05-09T10:50:21.889300+0000","last_clean":"2024-05-09T10:50:21.889300+0000","last_became_active":"2024-05-09T10:50:20.199060+0000","last_became_peered":"2024-05-09T10:50:20.199060+0000","last_unstale":"2024-05-09T10:50:21.889300+0000","last_undegraded":"2024-05-09T10:50:21.889300+0000","last_fullsized":"2024-05-09T10:50:21.889300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:35.361899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183103+0000","last_change":"2024-05-09T10:50:19.176037+0000","last_active":"2024-05-09T10:50:21.183103+0000","last_peered":"2024-05-09T10:50:21.183103+0000","last_clean":"2024-05-09T10:50:21.183103+0000","last_became_active":"2024-05-09T10:50:19.170706+0000","last_became_peered":"2024-05-09T10:50:19.170706+0000","last_unstale":"2024-05-09T10:50:21.183103+0000","last_undegraded":"2024-05-09T10:50:21.183103+0000","last_fullsized":"2024-05-09T10:50:21.183103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:54:23.668155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182651+0000","last_change":"2024-05-09T10:50:20.194758+0000","last_active":"2024-05-09T10:50:21.182651+0000","last_peered":"2024-05-09T10:50:21.182651+0000","last_clean":"2024-05-09T10:50:21.182651+0000","last_became_active":"2024-05-09T10:50:20.194509+0000","last_became_peered":"2024-05-09T10:50:20.194509+0000","last_unstale":"2024-05-09T10:50:21.182651+0000","last_undegraded":"2024-05-09T10:50:21.182651+0000","last_fullsized":"2024-05-09T10:50:21.182651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:32:42.671953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.638065+0000","last_change":"2024-05-09T10:50:19.167487+0000","last_active":"2024-05-09T10:50:22.638065+0000","last_peered":"2024-05-09T10:50:22.638065+0000","last_clean":"2024-05-09T10:50:22.638065+0000","last_became_active":"2024-05-09T10:50:19.167139+0000","last_became_peered":"2024-05-09T10:50:19.167139+0000","last_unstale":"2024-05-09T10:50:22.638065+0000","last_undegraded":"2024-05-09T10:50:22.638065+0000","last_fullsized":"2024-05-09T10:50:22.638065+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:20.661093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889232+0000","last_change":"2024-05-09T10:50:20.198511+0000","last_active":"2024-05-09T10:50:21.889232+0000","last_peered":"2024-05-09T10:50:21.889232+0000","last_clean":"2024-05-09T10:50:21.889232+0000","last_became_active":"2024-05-09T10:50:20.198165+0000","last_became_peered":"2024-05-09T10:50:20.198165+0000","last_unstale":"2024-05-09T10:50:21.889232+0000","last_undegraded":"2024-05-09T10:50:21.889232+0000","last_fullsized":"2024-05-09T10:50:21.889232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:47:01.454064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171521+0000","last_change":"2024-05-09T10:50:19.173828+0000","last_active":"2024-05-09T10:50:22.171521+0000","last_peered":"2024-05-09T10:50:22.171521+0000","last_clean":"2024-05-09T10:50:22.171521+0000","last_became_active":"2024-05-09T10:50:19.173524+0000","last_became_peered":"2024-05-09T10:50:19.173524+0000","last_unstale":"2024-05-09T10:50:22.171521+0000","last_undegraded":"2024-05-09T10:50:22.171521+0000","last_fullsized":"2024-05-09T10:50:22.171521+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:18:17.201844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171530+0000","last_change":"2024-05-09T10:50:20.196631+0000","last_active":"2024-05-09T10:50:22.171530+0000","last_peered":"2024-05-09T10:50:22.171530+0000","last_clean":"2024-05-09T10:50:22.171530+0000","last_became_active":"2024-05-09T10:50:20.196252+0000","last_became_peered":"2024-05-09T10:50:20.196252+0000","last_unstale":"2024-05-09T10:50:22.171530+0000","last_undegraded":"2024-05-09T10:50:22.171530+0000","last_fullsized":"2024-05-09T10:50:22.171530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:00:03.184256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.264260+0000","last_change":"2024-05-09T10:50:19.173070+0000","last_active":"2024-05-09T10:50:23.264260+0000","last_peered":"2024-05-09T10:50:23.264260+0000","last_clean":"2024-05-09T10:50:23.264260+0000","last_became_active":"2024-05-09T10:50:19.172711+0000","last_became_peered":"2024-05-09T10:50:19.172711+0000","last_unstale":"2024-05-09T10:50:23.264260+0000","last_undegraded":"2024-05-09T10:50:23.264260+0000","last_fullsized":"2024-05-09T10:50:23.264260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:53.042012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889056+0000","last_change":"2024-05-09T10:50:20.197201+0000","last_active":"2024-05-09T10:50:21.889056+0000","last_peered":"2024-05-09T10:50:21.889056+0000","last_clean":"2024-05-09T10:50:21.889056+0000","last_became_active":"2024-05-09T10:50:20.196926+0000","last_became_peered":"2024-05-09T10:50:20.196926+0000","last_unstale":"2024-05-09T10:50:21.889056+0000","last_undegraded":"2024-05-09T10:50:21.889056+0000","last_fullsized":"2024-05-09T10:50:21.889056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:39.852439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183547+0000","last_change":"2024-05-09T10:50:19.179678+0000","last_active":"2024-05-09T10:50:21.183547+0000","last_peered":"2024-05-09T10:50:21.183547+0000","last_clean":"2024-05-09T10:50:21.183547+0000","last_became_active":"2024-05-09T10:50:19.179419+0000","last_became_peered":"2024-05-09T10:50:19.179419+0000","last_unstale":"2024-05-09T10:50:21.183547+0000","last_undegraded":"2024-05-09T10:50:21.183547+0000","last_fullsized":"2024-05-09T10:50:21.183547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:38:13.651797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889436+0000","last_change":"2024-05-09T10:50:20.197970+0000","last_active":"2024-05-09T10:50:21.889436+0000","last_peered":"2024-05-09T10:50:21.889436+0000","last_clean":"2024-05-09T10:50:21.889436+0000","last_became_active":"2024-05-09T10:50:20.197653+0000","last_became_peered":"2024-05-09T10:50:20.197653+0000","last_unstale":"2024-05-09T10:50:21.889436+0000","last_undegraded":"2024-05-09T10:50:21.889436+0000","last_fullsized":"2024-05-09T10:50:21.889436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:07:48.935082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.264428+0000","last_change":"2024-05-09T10:50:19.174326+0000","last_active":"2024-05-09T10:50:23.264428+0000","last_peered":"2024-05-09T10:50:23.264428+0000","last_clean":"2024-05-09T10:50:23.264428+0000","last_became_active":"2024-05-09T10:50:19.173957+0000","last_became_peered":"2024-05-09T10:50:19.173957+0000","last_unstale":"2024-05-09T10:50:23.264428+0000","last_undegraded":"2024-05-09T10:50:23.264428+0000","last_fullsized":"2024-05-09T10:50:23.264428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:20.661093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889578+0000","last_change":"2024-05-09T10:50:20.199478+0000","last_active":"2024-05-09T10:50:21.889578+0000","last_peered":"2024-05-09T10:50:21.889578+0000","last_clean":"2024-05-09T10:50:21.889578+0000","last_became_active":"2024-05-09T10:50:20.199083+0000","last_became_peered":"2024-05-09T10:50:20.199083+0000","last_unstale":"2024-05-09T10:50:21.889578+0000","last_undegraded":"2024-05-09T10:50:21.889578+0000","last_fullsized":"2024-05-09T10:50:21.889578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:34:17.252316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183186+0000","last_change":"2024-05-09T10:50:19.173709+0000","last_active":"2024-05-09T10:50:21.183186+0000","last_peered":"2024-05-09T10:50:21.183186+0000","last_clean":"2024-05-09T10:50:21.183186+0000","last_became_active":"2024-05-09T10:50:19.173453+0000","last_became_peered":"2024-05-09T10:50:19.173453+0000","last_unstale":"2024-05-09T10:50:21.183186+0000","last_undegraded":"2024-05-09T10:50:21.183186+0000","last_fullsized":"2024-05-09T10:50:21.183186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:07:59.570026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.638324+0000","last_change":"2024-05-09T10:50:20.193882+0000","last_active":"2024-05-09T10:50:22.638324+0000","last_peered":"2024-05-09T10:50:22.638324+0000","last_clean":"2024-05-09T10:50:22.638324+0000","last_became_active":"2024-05-09T10:50:20.193186+0000","last_became_peered":"2024-05-09T10:50:20.193186+0000","last_unstale":"2024-05-09T10:50:22.638324+0000","last_undegraded":"2024-05-09T10:50:22.638324+0000","last_fullsized":"2024-05-09T10:50:22.638324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:15.043180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.264468+0000","last_change":"2024-05-09T10:50:19.163763+0000","last_active":"2024-05-09T10:50:23.264468+0000","last_peered":"2024-05-09T10:50:23.264468+0000","last_clean":"2024-05-09T10:50:23.264468+0000","last_became_active":"2024-05-09T10:50:19.163429+0000","last_became_peered":"2024-05-09T10:50:19.163429+0000","last_unstale":"2024-05-09T10:50:23.264468+0000","last_undegraded":"2024-05-09T10:50:23.264468+0000","last_fullsized":"2024-05-09T10:50:23.264468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:41:57.513979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889302+0000","last_change":"2024-05-09T10:50:20.200705+0000","last_active":"2024-05-09T10:50:21.889302+0000","last_peered":"2024-05-09T10:50:21.889302+0000","last_clean":"2024-05-09T10:50:21.889302+0000","last_became_active":"2024-05-09T10:50:20.200416+0000","last_became_peered":"2024-05-09T10:50:20.200416+0000","last_unstale":"2024-05-09T10:50:21.889302+0000","last_undegraded":"2024-05-09T10:50:21.889302+0000","last_fullsized":"2024-05-09T10:50:21.889302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:15.043180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.888962+0000","last_change":"2024-05-09T10:50:08.410364+0000","last_active":"2024-05-09T10:50:21.888962+0000","last_peered":"2024-05-09T10:50:21.888962+0000","last_clean":"2024-05-09T10:50:21.888962+0000","last_became_active":"2024-05-09T10:50:08.409457+0000","last_became_peered":"2024-05-09T10:50:08.409457+0000","last_unstale":"2024-05-09T10:50:21.888962+0000","last_undegraded":"2024-05-09T10:50:21.888962+0000","last_fullsized":"2024-05-09T10:50:21.888962+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_clean_scrub_stamp":"2024-05-09T10:50:07.396510+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:03.566698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.637748+0000","last_change":"2024-05-09T10:50:19.158709+0000","last_active":"2024-05-09T10:50:22.637748+0000","last_peered":"2024-05-09T10:50:22.637748+0000","last_clean":"2024-05-09T10:50:22.637748+0000","last_became_active":"2024-05-09T10:50:19.158385+0000","last_became_peered":"2024-05-09T10:50:19.158385+0000","last_unstale":"2024-05-09T10:50:22.637748+0000","last_undegraded":"2024-05-09T10:50:22.637748+0000","last_fullsized":"2024-05-09T10:50:22.637748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:18:17.201844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843642+0000","last_change":"2024-05-09T10:50:20.190695+0000","last_active":"2024-05-09T10:50:23.843642+0000","last_peered":"2024-05-09T10:50:23.843642+0000","last_clean":"2024-05-09T10:50:23.843642+0000","last_became_active":"2024-05-09T10:50:20.190446+0000","last_became_peered":"2024-05-09T10:50:20.190446+0000","last_unstale":"2024-05-09T10:50:23.843642+0000","last_undegraded":"2024-05-09T10:50:23.843642+0000","last_fullsized":"2024-05-09T10:50:23.843642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:23.176619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889057+0000","last_change":"2024-05-09T10:50:19.173033+0000","last_active":"2024-05-09T10:50:21.889057+0000","last_peered":"2024-05-09T10:50:21.889057+0000","last_clean":"2024-05-09T10:50:21.889057+0000","last_became_active":"2024-05-09T10:50:19.172691+0000","last_became_peered":"2024-05-09T10:50:19.172691+0000","last_unstale":"2024-05-09T10:50:21.889057+0000","last_undegraded":"2024-05-09T10:50:21.889057+0000","last_fullsized":"2024-05-09T10:50:21.889057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:49:05.717962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182430+0000","last_change":"2024-05-09T10:50:20.186680+0000","last_active":"2024-05-09T10:50:21.182430+0000","last_peered":"2024-05-09T10:50:21.182430+0000","last_clean":"2024-05-09T10:50:21.182430+0000","last_became_active":"2024-05-09T10:50:20.186413+0000","last_became_peered":"2024-05-09T10:50:20.186413+0000","last_unstale":"2024-05-09T10:50:21.182430+0000","last_undegraded":"2024-05-09T10:50:21.182430+0000","last_fullsized":"2024-05-09T10:50:21.182430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:33:37.505180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.264354+0000","last_change":"2024-05-09T10:50:19.166130+0000","last_active":"2024-05-09T10:50:23.264354+0000","last_peered":"2024-05-09T10:50:23.264354+0000","last_clean":"2024-05-09T10:50:23.264354+0000","last_became_active":"2024-05-09T10:50:19.165820+0000","last_became_peered":"2024-05-09T10:50:19.165820+0000","last_unstale":"2024-05-09T10:50:23.264354+0000","last_undegraded":"2024-05-09T10:50:23.264354+0000","last_fullsized":"2024-05-09T10:50:23.264354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:18:17.201844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182893+0000","last_change":"2024-05-09T10:50:20.190864+0000","last_active":"2024-05-09T10:50:21.182893+0000","last_peered":"2024-05-09T10:50:21.182893+0000","last_clean":"2024-05-09T10:50:21.182893+0000","last_became_active":"2024-05-09T10:50:20.190613+0000","last_became_peered":"2024-05-09T10:50:20.190613+0000","last_unstale":"2024-05-09T10:50:21.182893+0000","last_undegraded":"2024-05-09T10:50:21.182893+0000","last_fullsized":"2024-05-09T10:50:21.182893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:23.176619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.638288+0000","last_change":"2024-05-09T10:50:19.173006+0000","last_active":"2024-05-09T10:50:22.638288+0000","last_peered":"2024-05-09T10:50:22.638288+0000","last_clean":"2024-05-09T10:50:22.638288+0000","last_became_active":"2024-05-09T10:50:19.172607+0000","last_became_peered":"2024-05-09T10:50:19.172607+0000","last_unstale":"2024-05-09T10:50:22.638288+0000","last_undegraded":"2024-05-09T10:50:22.638288+0000","last_fullsized":"2024-05-09T10:50:22.638288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:39.416778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183468+0000","last_change":"2024-05-09T10:50:20.192361+0000","last_active":"2024-05-09T10:50:21.183468+0000","last_peered":"2024-05-09T10:50:21.183468+0000","last_clean":"2024-05-09T10:50:21.183468+0000","last_became_active":"2024-05-09T10:50:20.192116+0000","last_became_peered":"2024-05-09T10:50:20.192116+0000","last_unstale":"2024-05-09T10:50:21.183468+0000","last_undegraded":"2024-05-09T10:50:21.183468+0000","last_fullsized":"2024-05-09T10:50:21.183468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:38.767540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182627+0000","last_change":"2024-05-09T10:50:19.176926+0000","last_active":"2024-05-09T10:50:21.182627+0000","last_peered":"2024-05-09T10:50:21.182627+0000","last_clean":"2024-05-09T10:50:21.182627+0000","last_became_active":"2024-05-09T10:50:19.176627+0000","last_became_peered":"2024-05-09T10:50:19.176627+0000","last_unstale":"2024-05-09T10:50:21.182627+0000","last_undegraded":"2024-05-09T10:50:21.182627+0000","last_fullsized":"2024-05-09T10:50:21.182627+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:37.762424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.638236+0000","last_change":"2024-05-09T10:50:20.193387+0000","last_active":"2024-05-09T10:50:22.638236+0000","last_peered":"2024-05-09T10:50:22.638236+0000","last_clean":"2024-05-09T10:50:22.638236+0000","last_became_active":"2024-05-09T10:50:20.193115+0000","last_became_peered":"2024-05-09T10:50:20.193115+0000","last_unstale":"2024-05-09T10:50:22.638236+0000","last_undegraded":"2024-05-09T10:50:22.638236+0000","last_fullsized":"2024-05-09T10:50:22.638236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:47:01.454064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889536+0000","last_change":"2024-05-09T10:50:19.170801+0000","last_active":"2024-05-09T10:50:21.889536+0000","last_peered":"2024-05-09T10:50:21.889536+0000","last_clean":"2024-05-09T10:50:21.889536+0000","last_became_active":"2024-05-09T10:50:19.170130+0000","last_became_peered":"2024-05-09T10:50:19.170130+0000","last_unstale":"2024-05-09T10:50:21.889536+0000","last_undegraded":"2024-05-09T10:50:21.889536+0000","last_fullsized":"2024-05-09T10:50:21.889536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:04:45.153539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182544+0000","last_change":"2024-05-09T10:50:20.189228+0000","last_active":"2024-05-09T10:50:21.182544+0000","last_peered":"2024-05-09T10:50:21.182544+0000","last_clean":"2024-05-09T10:50:21.182544+0000","last_became_active":"2024-05-09T10:50:20.188969+0000","last_became_peered":"2024-05-09T10:50:20.188969+0000","last_unstale":"2024-05-09T10:50:21.182544+0000","last_undegraded":"2024-05-09T10:50:21.182544+0000","last_fullsized":"2024-05-09T10:50:21.182544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:00:03.184256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.170973+0000","last_change":"2024-05-09T10:50:19.171062+0000","last_active":"2024-05-09T10:50:22.170973+0000","last_peered":"2024-05-09T10:50:22.170973+0000","last_clean":"2024-05-09T10:50:22.170973+0000","last_became_active":"2024-05-09T10:50:19.170760+0000","last_became_peered":"2024-05-09T10:50:19.170760+0000","last_unstale":"2024-05-09T10:50:22.170973+0000","last_undegraded":"2024-05-09T10:50:22.170973+0000","last_fullsized":"2024-05-09T10:50:22.170973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:39.416778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.844039+0000","last_change":"2024-05-09T10:50:20.193239+0000","last_active":"2024-05-09T10:50:23.844039+0000","last_peered":"2024-05-09T10:50:23.844039+0000","last_clean":"2024-05-09T10:50:23.844039+0000","last_became_active":"2024-05-09T10:50:20.193014+0000","last_became_peered":"2024-05-09T10:50:20.193014+0000","last_unstale":"2024-05-09T10:50:23.844039+0000","last_undegraded":"2024-05-09T10:50:23.844039+0000","last_fullsized":"2024-05-09T10:50:23.844039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:33:13.347174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171972+0000","last_change":"2024-05-09T10:50:19.171256+0000","last_active":"2024-05-09T10:50:22.171972+0000","last_peered":"2024-05-09T10:50:22.171972+0000","last_clean":"2024-05-09T10:50:22.171972+0000","last_became_active":"2024-05-09T10:50:19.170964+0000","last_became_peered":"2024-05-09T10:50:19.170964+0000","last_unstale":"2024-05-09T10:50:22.171972+0000","last_undegraded":"2024-05-09T10:50:22.171972+0000","last_fullsized":"2024-05-09T10:50:22.171972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:20.661093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182872+0000","last_change":"2024-05-09T10:50:20.184272+0000","last_active":"2024-05-09T10:50:21.182872+0000","last_peered":"2024-05-09T10:50:21.182872+0000","last_clean":"2024-05-09T10:50:21.182872+0000","last_became_active":"2024-05-09T10:50:20.183978+0000","last_became_peered":"2024-05-09T10:50:20.183978+0000","last_unstale":"2024-05-09T10:50:21.182872+0000","last_undegraded":"2024-05-09T10:50:21.182872+0000","last_fullsized":"2024-05-09T10:50:21.182872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:45.067992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.888875+0000","last_change":"2024-05-09T10:50:19.171403+0000","last_active":"2024-05-09T10:50:21.888875+0000","last_peered":"2024-05-09T10:50:21.888875+0000","last_clean":"2024-05-09T10:50:21.888875+0000","last_became_active":"2024-05-09T10:50:19.171106+0000","last_became_peered":"2024-05-09T10:50:19.171106+0000","last_unstale":"2024-05-09T10:50:21.888875+0000","last_undegraded":"2024-05-09T10:50:21.888875+0000","last_fullsized":"2024-05-09T10:50:21.888875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:39.416778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183306+0000","last_change":"2024-05-09T10:50:20.189694+0000","last_active":"2024-05-09T10:50:21.183306+0000","last_peered":"2024-05-09T10:50:21.183306+0000","last_clean":"2024-05-09T10:50:21.183306+0000","last_became_active":"2024-05-09T10:50:20.189423+0000","last_became_peered":"2024-05-09T10:50:20.189423+0000","last_unstale":"2024-05-09T10:50:21.183306+0000","last_undegraded":"2024-05-09T10:50:21.183306+0000","last_fullsized":"2024-05-09T10:50:21.183306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:40.421893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183073+0000","last_change":"2024-05-09T10:50:19.174912+0000","last_active":"2024-05-09T10:50:21.183073+0000","last_peered":"2024-05-09T10:50:21.183073+0000","last_clean":"2024-05-09T10:50:21.183073+0000","last_became_active":"2024-05-09T10:50:19.174640+0000","last_became_peered":"2024-05-09T10:50:19.174640+0000","last_unstale":"2024-05-09T10:50:21.183073+0000","last_undegraded":"2024-05-09T10:50:21.183073+0000","last_fullsized":"2024-05-09T10:50:21.183073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:33:53.042012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843648+0000","last_change":"2024-05-09T10:50:20.190046+0000","last_active":"2024-05-09T10:50:23.843648+0000","last_peered":"2024-05-09T10:50:23.843648+0000","last_clean":"2024-05-09T10:50:23.843648+0000","last_became_active":"2024-05-09T10:50:20.189778+0000","last_became_peered":"2024-05-09T10:50:20.189778+0000","last_unstale":"2024-05-09T10:50:23.843648+0000","last_undegraded":"2024-05-09T10:50:23.843648+0000","last_fullsized":"2024-05-09T10:50:23.843648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:47.420832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.182837+0000","last_change":"2024-05-09T10:50:19.172738+0000","last_active":"2024-05-09T10:50:21.182837+0000","last_peered":"2024-05-09T10:50:21.182837+0000","last_clean":"2024-05-09T10:50:21.182837+0000","last_became_active":"2024-05-09T10:50:19.172449+0000","last_became_peered":"2024-05-09T10:50:19.172449+0000","last_unstale":"2024-05-09T10:50:21.182837+0000","last_undegraded":"2024-05-09T10:50:21.182837+0000","last_fullsized":"2024-05-09T10:50:21.182837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:20.661093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843762+0000","last_change":"2024-05-09T10:50:20.193228+0000","last_active":"2024-05-09T10:50:23.843762+0000","last_peered":"2024-05-09T10:50:23.843762+0000","last_clean":"2024-05-09T10:50:23.843762+0000","last_became_active":"2024-05-09T10:50:20.192454+0000","last_became_peered":"2024-05-09T10:50:20.192454+0000","last_unstale":"2024-05-09T10:50:23.843762+0000","last_undegraded":"2024-05-09T10:50:23.843762+0000","last_fullsized":"2024-05-09T10:50:23.843762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:34:20.711637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889326+0000","last_change":"2024-05-09T10:50:19.174967+0000","last_active":"2024-05-09T10:50:21.889326+0000","last_peered":"2024-05-09T10:50:21.889326+0000","last_clean":"2024-05-09T10:50:21.889326+0000","last_became_active":"2024-05-09T10:50:19.173603+0000","last_became_peered":"2024-05-09T10:50:19.173603+0000","last_unstale":"2024-05-09T10:50:21.889326+0000","last_undegraded":"2024-05-09T10:50:21.889326+0000","last_fullsized":"2024-05-09T10:50:21.889326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:16:52.160962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.171552+0000","last_change":"2024-05-09T10:50:20.187976+0000","last_active":"2024-05-09T10:50:22.171552+0000","last_peered":"2024-05-09T10:50:22.171552+0000","last_clean":"2024-05-09T10:50:22.171552+0000","last_became_active":"2024-05-09T10:50:20.187661+0000","last_became_peered":"2024-05-09T10:50:20.187661+0000","last_unstale":"2024-05-09T10:50:22.171552+0000","last_undegraded":"2024-05-09T10:50:22.171552+0000","last_fullsized":"2024-05-09T10:50:22.171552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:45.067992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.638210+0000","last_change":"2024-05-09T10:50:20.181807+0000","last_active":"2024-05-09T10:50:22.638210+0000","last_peered":"2024-05-09T10:50:22.638210+0000","last_clean":"2024-05-09T10:50:22.638210+0000","last_became_active":"2024-05-09T10:50:20.181212+0000","last_became_peered":"2024-05-09T10:50:20.181212+0000","last_unstale":"2024-05-09T10:50:22.638210+0000","last_undegraded":"2024-05-09T10:50:22.638210+0000","last_fullsized":"2024-05-09T10:50:22.638210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:39.852439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889187+0000","last_change":"2024-05-09T10:50:19.175398+0000","last_active":"2024-05-09T10:50:21.889187+0000","last_peered":"2024-05-09T10:50:21.889187+0000","last_clean":"2024-05-09T10:50:21.889187+0000","last_became_active":"2024-05-09T10:50:19.174893+0000","last_became_peered":"2024-05-09T10:50:19.174893+0000","last_unstale":"2024-05-09T10:50:21.889187+0000","last_undegraded":"2024-05-09T10:50:21.889187+0000","last_fullsized":"2024-05-09T10:50:21.889187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:07:59.570026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:22.638044+0000","last_change":"2024-05-09T10:50:20.183971+0000","last_active":"2024-05-09T10:50:22.638044+0000","last_peered":"2024-05-09T10:50:22.638044+0000","last_clean":"2024-05-09T10:50:22.638044+0000","last_became_active":"2024-05-09T10:50:20.183666+0000","last_became_peered":"2024-05-09T10:50:20.183666+0000","last_unstale":"2024-05-09T10:50:22.638044+0000","last_undegraded":"2024-05-09T10:50:22.638044+0000","last_fullsized":"2024-05-09T10:50:22.638044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:33:13.347174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.889464+0000","last_change":"2024-05-09T10:50:19.169800+0000","last_active":"2024-05-09T10:50:21.889464+0000","last_peered":"2024-05-09T10:50:21.889464+0000","last_clean":"2024-05-09T10:50:21.889464+0000","last_became_active":"2024-05-09T10:50:19.169521+0000","last_became_peered":"2024-05-09T10:50:19.169521+0000","last_unstale":"2024-05-09T10:50:21.889464+0000","last_undegraded":"2024-05-09T10:50:21.889464+0000","last_fullsized":"2024-05-09T10:50:21.889464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:37.762424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183081+0000","last_change":"2024-05-09T10:50:20.181912+0000","last_active":"2024-05-09T10:50:21.183081+0000","last_peered":"2024-05-09T10:50:21.183081+0000","last_clean":"2024-05-09T10:50:21.183081+0000","last_became_active":"2024-05-09T10:50:20.181615+0000","last_became_peered":"2024-05-09T10:50:20.181615+0000","last_unstale":"2024-05-09T10:50:21.183081+0000","last_undegraded":"2024-05-09T10:50:21.183081+0000","last_fullsized":"2024-05-09T10:50:21.183081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:50:14.919494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843989+0000","last_change":"2024-05-09T10:50:19.165320+0000","last_active":"2024-05-09T10:50:23.843989+0000","last_peered":"2024-05-09T10:50:23.843989+0000","last_clean":"2024-05-09T10:50:23.843989+0000","last_became_active":"2024-05-09T10:50:19.165033+0000","last_became_peered":"2024-05-09T10:50:19.165033+0000","last_unstale":"2024-05-09T10:50:23.843989+0000","last_undegraded":"2024-05-09T10:50:23.843989+0000","last_fullsized":"2024-05-09T10:50:23.843989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:11:05.623356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:23.843981+0000","last_change":"2024-05-09T10:50:20.188606+0000","last_active":"2024-05-09T10:50:23.843981+0000","last_peered":"2024-05-09T10:50:23.843981+0000","last_clean":"2024-05-09T10:50:23.843981+0000","last_became_active":"2024-05-09T10:50:20.188300+0000","last_became_peered":"2024-05-09T10:50:20.188300+0000","last_unstale":"2024-05-09T10:50:23.843981+0000","last_undegraded":"2024-05-09T10:50:23.843981+0000","last_fullsized":"2024-05-09T10:50:23.843981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:19.147433+0000","last_clean_scrub_stamp":"2024-05-09T10:50:19.147433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:26:34.638740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T10:50:21.183175+0000","last_change":"2024-05-09T10:50:19.168206+0000","last_active":"2024-05-09T10:50:21.183175+0000","last_peered":"2024-05-09T10:50:21.183175+0000","last_clean":"2024-05-09T10:50:21.183175+0000","last_became_active":"2024-05-09T10:50:19.167924+0000","last_became_peered":"2024-05-09T10:50:19.167924+0000","last_unstale":"2024-05-09T10:50:21.183175+0000","last_undegraded":"2024-05-09T10:50:21.183175+0000","last_fullsized":"2024-05-09T10:50:21.183175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:18.142317+0000","last_clean_scrub_stamp":"2024-05-09T10:50:18.142317+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:11:05.623356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T10:50:27.773 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-09T10:50:27.773 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-09T10:50:27.773 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-09T10:50:27.773 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-09T10:50:27.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.914+0000 7fa880812640 1 Processor -- start 2024-05-09T10:50:27.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.914+0000 7fa880812640 1 -- start start 2024-05-09T10:50:27.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.914+0000 7fa880812640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa87c0ce240 0x7fa87c06abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:27.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa880812640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa87c0d0390 0x7fa87c06b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:27.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa880812640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa87c06e4e0 con 0x7fa87c15bce0 2024-05-09T10:50:27.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa880812640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa87c06d920 con 0x7fa87c0d0390 2024-05-09T10:50:27.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa880812640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c0721a0 con 0x7fa87c0ce240 2024-05-09T10:50:27.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa87a59c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa87c15bce0 0x7fa87c06ddd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56046/0 (socket says 172.21.15.98:56046) 2024-05-09T10:50:27.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa87a59c640 1 -- 172.21.15.98:0/1343567095 learned_addr learned my addr 172.21.15.98:0/1343567095 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:27.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa87ad9d640 1 --2- 172.21.15.98:0/1343567095 >> v2:172.21.15.105:3300/0 conn(0x7fa87c0d0390 0x7fa87c06b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:27.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa871d9b640 1 --2- 172.21.15.98:0/1343567095 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa87c0ce240 0x7fa87c06abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:27.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa879d9b640 1 -- 172.21.15.98:0/1343567095 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3854029850 0 0) 0x7fa87c06e4e0 con 0x7fa87c15bce0 2024-05-09T10:50:27.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa879d9b640 1 -- 172.21.15.98:0/1343567095 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa868003520 con 0x7fa87c15bce0 2024-05-09T10:50:27.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa879d9b640 1 -- 172.21.15.98:0/1343567095 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3900123981 0 0) 0x7fa868003520 con 0x7fa87c15bce0 2024-05-09T10:50:27.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa879d9b640 1 -- 172.21.15.98:0/1343567095 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa87c0ce240 msgr2=0x7fa87c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:27.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa879d9b640 1 --2- 172.21.15.98:0/1343567095 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa87c0ce240 0x7fa87c06abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa879d9b640 1 -- 172.21.15.98:0/1343567095 >> v2:172.21.15.105:3300/0 conn(0x7fa87c0d0390 msgr2=0x7fa87c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:27.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa879d9b640 1 --2- 172.21.15.98:0/1343567095 >> v2:172.21.15.105:3300/0 conn(0x7fa87c0d0390 0x7fa87c06b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa879d9b640 1 -- 172.21.15.98:0/1343567095 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c072480 con 0x7fa87c15bce0 2024-05-09T10:50:27.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa879d9b640 1 -- 172.21.15.98:0/1343567095 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7fa86c001670 con 0x7fa87c15bce0 2024-05-09T10:50:27.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa879d9b640 1 -- 172.21.15.98:0/1343567095 >> v1:172.21.15.98:6789/0 conn(0x7fa87c15bce0 legacy=0x7fa87c06ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa879d9b640 1 --2- 172.21.15.98:0/1343567095 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8680039b0 0x7fa868003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:27.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa879d9b640 1 --2- 172.21.15.98:0/1343567095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa868004310 0x7fa868004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:27.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa879d9b640 1 --2- 172.21.15.98:0/1343567095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa868004e50 0x7fa8680052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:27.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa879d9b640 1 -- 172.21.15.98:0/1343567095 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c06d920 con 0x7fa868004e50 2024-05-09T10:50:27.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa879d9b640 1 -- 172.21.15.98:0/1343567095 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c0721a0 con 0x7fa868004310 2024-05-09T10:50:27.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa879d9b640 1 -- 172.21.15.98:0/1343567095 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86800bde0 con 0x7fa8680039b0 2024-05-09T10:50:27.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa87ad9d640 1 --2- 172.21.15.98:0/1343567095 >> v2:172.21.15.105:3300/0 conn(0x7fa87c0d0390 0x7fa87c06b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:50:27.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa87a59c640 1 --2- 172.21.15.98:0/1343567095 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8680039b0 0x7fa868003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:27.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa87ad9d640 1 --2- 172.21.15.98:0/1343567095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa868004e50 0x7fa8680052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:27.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.915+0000 7fa871d9b640 1 --2- 172.21.15.98:0/1343567095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa868004310 0x7fa868004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:27.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.916+0000 7fa87ad9d640 1 -- 172.21.15.98:0/1343567095 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8680039b0 msgr2=0x7fa868003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:27.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.916+0000 7fa87ad9d640 1 --2- 172.21.15.98:0/1343567095 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8680039b0 0x7fa868003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.916+0000 7fa87ad9d640 1 -- 172.21.15.98:0/1343567095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa868004310 msgr2=0x7fa868004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:27.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.916+0000 7fa87ad9d640 1 --2- 172.21.15.98:0/1343567095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa868004310 0x7fa868004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.916+0000 7fa87ad9d640 1 -- 172.21.15.98:0/1343567095 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa868003520 con 0x7fa868004e50 2024-05-09T10:50:27.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.916+0000 7fa87ad9d640 1 --2- 172.21.15.98:0/1343567095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa868004e50 0x7fa8680052d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa87400e320 tx=0x7fa868005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe5e78655f2847d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:27.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.916+0000 7fa879d9b640 1 -- 172.21.15.98:0/1343567095 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa874009de0 con 0x7fa868004e50 2024-05-09T10:50:27.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.916+0000 7fa879d9b640 1 -- 172.21.15.98:0/1343567095 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87402e620 con 0x7fa868004e50 2024-05-09T10:50:27.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.916+0000 7fa879d9b640 1 -- 172.21.15.98:0/1343567095 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87402ea10 con 0x7fa868004e50 2024-05-09T10:50:27.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.917+0000 7fa880812640 1 -- 172.21.15.98:0/1343567095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa868004e50 msgr2=0x7fa8680052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.917+0000 7fa880812640 1 --2- 172.21.15.98:0/1343567095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa868004e50 0x7fa8680052d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa87400e320 tx=0x7fa868005830 comp rx=0 tx=0).stop 2024-05-09T10:50:27.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.917+0000 7fa87ad9d640 1 -- 172.21.15.98:0/1343567095 reap_dead start 2024-05-09T10:50:27.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.917+0000 7fa880812640 1 -- 172.21.15.98:0/1343567095 shutdown_connections 2024-05-09T10:50:27.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.917+0000 7fa880812640 1 -- 172.21.15.98:0/1343567095 >> 172.21.15.98:0/1343567095 conn(0x7fa87c066f20 msgr2=0x7fa87c06bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:27.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.917+0000 7fa880812640 1 -- 172.21.15.98:0/1343567095 shutdown_connections 2024-05-09T10:50:27.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.917+0000 7fa880812640 1 -- 172.21.15.98:0/1343567095 wait complete. 2024-05-09T10:50:27.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.918+0000 7fa880812640 1 Processor -- start 2024-05-09T10:50:27.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.918+0000 7fa880812640 1 -- start start 2024-05-09T10:50:27.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.919+0000 7fa880812640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa868004e50 0x7fa87c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:27.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.919+0000 7fa880812640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa87c0c9ff0 0x7fa87c1f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:27.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.919+0000 7fa880812640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa87c0d0d80 0x7fa87c0d1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:27.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.919+0000 7fa880812640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c06ad60 con 0x7fa87c0d0d80 2024-05-09T10:50:27.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.919+0000 7fa880812640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c06abe0 con 0x7fa87c0c9ff0 2024-05-09T10:50:27.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.919+0000 7fa880812640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c06aee0 con 0x7fa868004e50 2024-05-09T10:50:27.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.919+0000 7fa87ad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa87c0d0d80 0x7fa87c0d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:27.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.919+0000 7fa87ad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa87c0d0d80 0x7fa87c0d1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42298/0 (socket says 172.21.15.98:42298) 2024-05-09T10:50:27.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.919+0000 7fa87ad9d640 1 -- 172.21.15.98:0/2425894828 learned_addr learned my addr 172.21.15.98:0/2425894828 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:27.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.919+0000 7fa87a59c640 1 --2- 172.21.15.98:0/2425894828 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa868004e50 0x7fa87c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:27.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.919+0000 7fa871d9b640 1 --2- 172.21.15.98:0/2425894828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa87c0c9ff0 0x7fa87c1f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:27.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.919+0000 7fa87ad9d640 1 -- 172.21.15.98:0/2425894828 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa868004e50 msgr2=0x7fa87c0b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.919+0000 7fa87ad9d640 1 --2- 172.21.15.98:0/2425894828 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa868004e50 0x7fa87c0b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.920+0000 7fa87ad9d640 1 -- 172.21.15.98:0/2425894828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa87c0c9ff0 msgr2=0x7fa87c1f2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:27.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.920+0000 7fa87ad9d640 1 --2- 172.21.15.98:0/2425894828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa87c0c9ff0 0x7fa87c1f2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:27.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.920+0000 7fa87ad9d640 1 -- 172.21.15.98:0/2425894828 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c0cb5e0 con 0x7fa87c0d0d80 2024-05-09T10:50:27.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.920+0000 7fa87a59c640 1 --2- 172.21.15.98:0/2425894828 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa868004e50 0x7fa87c0b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:27.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.920+0000 7fa87ad9d640 1 --2- 172.21.15.98:0/2425894828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa87c0d0d80 0x7fa87c0d1200 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa87402b0a0 tx=0x7fa874009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:27.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.920+0000 7fa8737fe640 1 -- 172.21.15.98:0/2425894828 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa874009de0 con 0x7fa87c0d0d80 2024-05-09T10:50:27.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.920+0000 7fa8737fe640 1 -- 172.21.15.98:0/2425894828 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87402e420 con 0x7fa87c0d0d80 2024-05-09T10:50:27.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.920+0000 7fa880812640 1 -- 172.21.15.98:0/2425894828 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87c0ca540 con 0x7fa87c0d0d80 2024-05-09T10:50:27.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.920+0000 7fa880812640 1 -- 172.21.15.98:0/2425894828 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa87c0cab00 con 0x7fa87c0d0d80 2024-05-09T10:50:27.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.920+0000 7fa8737fe640 1 -- 172.21.15.98:0/2425894828 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa874038250 con 0x7fa87c0d0d80 2024-05-09T10:50:27.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.921+0000 7fa880812640 1 -- 172.21.15.98:0/2425894828 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa87c069cb0 con 0x7fa87c0d0d80 2024-05-09T10:50:27.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.921+0000 7fa8737fe640 1 -- 172.21.15.98:0/2425894828 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa874038860 con 0x7fa87c0d0d80 2024-05-09T10:50:27.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.922+0000 7fa8737fe640 1 --2- 172.21.15.98:0/2425894828 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa85c06b4d0 0x7fa85c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:27.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.922+0000 7fa8737fe640 1 -- 172.21.15.98:0/2425894828 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8740aa7b0 con 0x7fa87c0d0d80 2024-05-09T10:50:27.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.922+0000 7fa87a59c640 1 --2- 172.21.15.98:0/2425894828 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa85c06b4d0 0x7fa85c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:27.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.922+0000 7fa87a59c640 1 --2- 172.21.15.98:0/2425894828 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa85c06b4d0 0x7fa85c06d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0cff70 tx=0x7fa86c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:27.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:27.926+0000 7fa8737fe640 1 -- 172.21.15.98:0/2425894828 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa874032030 con 0x7fa87c0d0d80 2024-05-09T10:50:28.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.132+0000 7fa880812640 1 -- 172.21.15.98:0/2425894828 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa87c06c050 con 0x7fa87c0d0d80 2024-05-09T10:50:28.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.133+0000 7fa8737fe640 1 -- 172.21.15.98:0/2425894828 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa874079630 con 0x7fa87c0d0d80 2024-05-09T10:50:28.126 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:28.127 INFO:teuthology.orchestra.run.smithi098.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-09T10:50:28.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.135+0000 7fa880812640 1 -- 172.21.15.98:0/2425894828 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa85c06b4d0 msgr2=0x7fa85c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:28.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.135+0000 7fa880812640 1 --2- 172.21.15.98:0/2425894828 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa85c06b4d0 0x7fa85c06d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0cff70 tx=0x7fa86c009040 comp rx=0 tx=0).stop 2024-05-09T10:50:28.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.135+0000 7fa880812640 1 -- 172.21.15.98:0/2425894828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa87c0d0d80 msgr2=0x7fa87c0d1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:28.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.135+0000 7fa880812640 1 --2- 172.21.15.98:0/2425894828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa87c0d0d80 0x7fa87c0d1200 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa87402b0a0 tx=0x7fa874009750 comp rx=0 tx=0).stop 2024-05-09T10:50:28.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.135+0000 7fa880812640 1 -- 172.21.15.98:0/2425894828 shutdown_connections 2024-05-09T10:50:28.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.135+0000 7fa880812640 1 --2- 172.21.15.98:0/2425894828 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa85c06b4d0 0x7fa85c06d990 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.135+0000 7fa880812640 1 --2- 172.21.15.98:0/2425894828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa87c0d0d80 0x7fa87c0d1200 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.135+0000 7fa880812640 1 --2- 172.21.15.98:0/2425894828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa87c0c9ff0 0x7fa87c1f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.135+0000 7fa880812640 1 --2- 172.21.15.98:0/2425894828 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa868004e50 0x7fa87c0b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.135+0000 7fa880812640 1 -- 172.21.15.98:0/2425894828 >> 172.21.15.98:0/2425894828 conn(0x7fa87c066f20 msgr2=0x7fa87c15c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:28.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.135+0000 7fa880812640 1 -- 172.21.15.98:0/2425894828 shutdown_connections 2024-05-09T10:50:28.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.135+0000 7fa880812640 1 -- 172.21.15.98:0/2425894828 wait complete. 2024-05-09T10:50:28.139 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-09T10:50:28.139 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:28.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.248+0000 7fc38e9a5640 1 Processor -- start 2024-05-09T10:50:28.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.248+0000 7fc38e9a5640 1 -- start start 2024-05-09T10:50:28.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.249+0000 7fc38e9a5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc3880b40b0 0x7fc38815de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:28.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.249+0000 7fc38e9a5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc388071ad0 con 0x7fc38815a3f0 2024-05-09T10:50:28.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.249+0000 7fc38e9a5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc388071db0 con 0x7fc38806dc50 2024-05-09T10:50:28.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.249+0000 7fc38e9a5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3880718a0 con 0x7fc3880b40b0 2024-05-09T10:50:28.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.249+0000 7fc387fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc38815a3f0 0x7fc38806d540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56054/0 (socket says 172.21.15.98:56054) 2024-05-09T10:50:28.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.249+0000 7fc387fff640 1 -- 172.21.15.98:0/3031097744 learned_addr learned my addr 172.21.15.98:0/3031097744 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:28.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.249+0000 7fc3877fe640 1 --2- 172.21.15.98:0/3031097744 >> v2:172.21.15.105:3300/0 conn(0x7fc3880b40b0 0x7fc38815de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:28.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.249+0000 7fc386ffd640 1 -- 172.21.15.98:0/3031097744 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 495599259 0 0) 0x7fc388071ad0 con 0x7fc38815a3f0 2024-05-09T10:50:28.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.249+0000 7fc386ffd640 1 -- 172.21.15.98:0/3031097744 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc370003520 con 0x7fc38815a3f0 2024-05-09T10:50:28.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.250+0000 7fc386ffd640 1 -- 172.21.15.98:0/3031097744 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1641656761 0 0) 0x7fc370003520 con 0x7fc38815a3f0 2024-05-09T10:50:28.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.250+0000 7fc386ffd640 1 -- 172.21.15.98:0/3031097744 >> v2:172.21.15.105:3300/0 conn(0x7fc3880b40b0 msgr2=0x7fc38815de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:28.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.250+0000 7fc386ffd640 1 --2- 172.21.15.98:0/3031097744 >> v2:172.21.15.105:3300/0 conn(0x7fc3880b40b0 0x7fc38815de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.250+0000 7fc386ffd640 1 -- 172.21.15.98:0/3031097744 >> v1:172.21.15.105:6789/0 conn(0x7fc38806dc50 legacy=0x7fc38806c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:28.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.250+0000 7fc386ffd640 1 -- 172.21.15.98:0/3031097744 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc388072090 con 0x7fc38815a3f0 2024-05-09T10:50:28.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.250+0000 7fc386ffd640 1 -- 172.21.15.98:0/3031097744 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7fc378001670 con 0x7fc38815a3f0 2024-05-09T10:50:28.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.250+0000 7fc386ffd640 1 -- 172.21.15.98:0/3031097744 >> v1:172.21.15.98:6789/0 conn(0x7fc38815a3f0 legacy=0x7fc38806d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:28.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.250+0000 7fc386ffd640 1 --2- 172.21.15.98:0/3031097744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3700039b0 0x7fc370003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:28.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.250+0000 7fc386ffd640 1 --2- 172.21.15.98:0/3031097744 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3700043b0 0x7fc370004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:28.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.250+0000 7fc386ffd640 1 --2- 172.21.15.98:0/3031097744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc370005120 0x7fc37000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:28.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.250+0000 7fc386ffd640 1 -- 172.21.15.98:0/3031097744 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37000c030 con 0x7fc370005120 2024-05-09T10:50:28.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.250+0000 7fc386ffd640 1 -- 172.21.15.98:0/3031097744 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37000c1e0 con 0x7fc3700039b0 2024-05-09T10:50:28.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.250+0000 7fc386ffd640 1 -- 172.21.15.98:0/3031097744 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37000c3e0 con 0x7fc3700043b0 2024-05-09T10:50:28.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.250+0000 7fc38cf42640 1 --2- 172.21.15.98:0/3031097744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc370005120 0x7fc37000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:28.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.250+0000 7fc387fff640 1 --2- 172.21.15.98:0/3031097744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3700039b0 0x7fc370003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:28.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.250+0000 7fc3877fe640 1 --2- 172.21.15.98:0/3031097744 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3700043b0 0x7fc370004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:28.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.250+0000 7fc38cf42640 1 -- 172.21.15.98:0/3031097744 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3700043b0 msgr2=0x7fc370004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:28.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.250+0000 7fc38cf42640 1 --2- 172.21.15.98:0/3031097744 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3700043b0 0x7fc370004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.250+0000 7fc38cf42640 1 -- 172.21.15.98:0/3031097744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3700039b0 msgr2=0x7fc370003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:28.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.250+0000 7fc38cf42640 1 --2- 172.21.15.98:0/3031097744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3700039b0 0x7fc370003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.250+0000 7fc38cf42640 1 -- 172.21.15.98:0/3031097744 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc370003520 con 0x7fc370005120 2024-05-09T10:50:28.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.251+0000 7fc38cf42640 1 -- 172.21.15.98:0/3031097744 reap_dead start 2024-05-09T10:50:28.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.251+0000 7fc38cf42640 1 --2- 172.21.15.98:0/3031097744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc370005120 0x7fc37000b9c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc380001f60 tx=0x7fc370005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf4d35e9cf07ddb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:28.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.251+0000 7fc386ffd640 1 -- 172.21.15.98:0/3031097744 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380001b30 con 0x7fc370005120 2024-05-09T10:50:28.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.251+0000 7fc386ffd640 1 -- 172.21.15.98:0/3031097744 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc380003b90 con 0x7fc370005120 2024-05-09T10:50:28.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.251+0000 7fc386ffd640 1 -- 172.21.15.98:0/3031097744 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380005110 con 0x7fc370005120 2024-05-09T10:50:28.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.251+0000 7fc38e9a5640 1 -- 172.21.15.98:0/3031097744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc370005120 msgr2=0x7fc37000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:28.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.251+0000 7fc38e9a5640 1 --2- 172.21.15.98:0/3031097744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc370005120 0x7fc37000b9c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc380001f60 tx=0x7fc370005830 comp rx=0 tx=0).stop 2024-05-09T10:50:28.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.251+0000 7fc38e9a5640 1 -- 172.21.15.98:0/3031097744 shutdown_connections 2024-05-09T10:50:28.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.251+0000 7fc38e9a5640 1 --2- 172.21.15.98:0/3031097744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc370005120 0x7fc37000b9c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.251+0000 7fc38e9a5640 1 -- 172.21.15.98:0/3031097744 >> 172.21.15.98:0/3031097744 conn(0x7fc38806abe0 msgr2=0x7fc38806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:28.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.251+0000 7fc38e9a5640 1 -- 172.21.15.98:0/3031097744 shutdown_connections 2024-05-09T10:50:28.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.251+0000 7fc38e9a5640 1 -- 172.21.15.98:0/3031097744 wait complete. 2024-05-09T10:50:28.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.252+0000 7fc38e9a5640 1 Processor -- start 2024-05-09T10:50:28.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.253+0000 7fc38e9a5640 1 -- start start 2024-05-09T10:50:28.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.253+0000 7fc38e9a5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc370005120 0x7fc3881481d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:28.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.253+0000 7fc38e9a5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3880b2220 0x7fc3880b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:28.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.253+0000 7fc38e9a5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc38815b9e0 0x7fc38815be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:28.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.253+0000 7fc38e9a5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3880af120 con 0x7fc38815b9e0 2024-05-09T10:50:28.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.253+0000 7fc38e9a5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3880aee30 con 0x7fc370005120 2024-05-09T10:50:28.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.253+0000 7fc38e9a5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc388072400 con 0x7fc3880b2220 2024-05-09T10:50:28.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.253+0000 7fc38cf42640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc38815b9e0 0x7fc38815be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:28.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.253+0000 7fc38cf42640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc38815b9e0 0x7fc38815be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42322/0 (socket says 172.21.15.98:42322) 2024-05-09T10:50:28.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.253+0000 7fc38cf42640 1 -- 172.21.15.98:0/2513674908 learned_addr learned my addr 172.21.15.98:0/2513674908 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:28.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.254+0000 7fc387fff640 1 --2- 172.21.15.98:0/2513674908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc370005120 0x7fc3881481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:28.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.254+0000 7fc3877fe640 1 --2- 172.21.15.98:0/2513674908 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3880b2220 0x7fc3880b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:28.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.254+0000 7fc38cf42640 1 -- 172.21.15.98:0/2513674908 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3880b2220 msgr2=0x7fc3880b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:28.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.254+0000 7fc38cf42640 1 --2- 172.21.15.98:0/2513674908 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3880b2220 0x7fc3880b26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.254+0000 7fc38cf42640 1 -- 172.21.15.98:0/2513674908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc370005120 msgr2=0x7fc3881481d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:28.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.254+0000 7fc38cf42640 1 --2- 172.21.15.98:0/2513674908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc370005120 0x7fc3881481d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.254+0000 7fc38cf42640 1 -- 172.21.15.98:0/2513674908 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc388148710 con 0x7fc38815b9e0 2024-05-09T10:50:28.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.254+0000 7fc38cf42640 1 --2- 172.21.15.98:0/2513674908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc38815b9e0 0x7fc38815be40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc380005970 tx=0x7fc3800015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:28.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.254+0000 7fc384ff9640 1 -- 172.21.15.98:0/2513674908 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380006b50 con 0x7fc38815b9e0 2024-05-09T10:50:28.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.254+0000 7fc384ff9640 1 -- 172.21.15.98:0/2513674908 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc380006cf0 con 0x7fc38815b9e0 2024-05-09T10:50:28.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.254+0000 7fc38e9a5640 1 -- 172.21.15.98:0/2513674908 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3880b2d70 con 0x7fc38815b9e0 2024-05-09T10:50:28.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.254+0000 7fc384ff9640 1 -- 172.21.15.98:0/2513674908 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38003e670 con 0x7fc38815b9e0 2024-05-09T10:50:28.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.254+0000 7fc38e9a5640 1 -- 172.21.15.98:0/2513674908 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3880b3330 con 0x7fc38815b9e0 2024-05-09T10:50:28.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.255+0000 7fc38e9a5640 1 -- 172.21.15.98:0/2513674908 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc38806afc0 con 0x7fc38815b9e0 2024-05-09T10:50:28.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.255+0000 7fc384ff9640 1 -- 172.21.15.98:0/2513674908 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc380030070 con 0x7fc38815b9e0 2024-05-09T10:50:28.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.256+0000 7fc384ff9640 1 --2- 172.21.15.98:0/2513674908 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc37406b4d0 0x7fc37406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:28.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.256+0000 7fc384ff9640 1 -- 172.21.15.98:0/2513674908 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3800a9fa0 con 0x7fc38815b9e0 2024-05-09T10:50:28.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.256+0000 7fc387fff640 1 --2- 172.21.15.98:0/2513674908 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc37406b4d0 0x7fc37406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:28.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.256+0000 7fc387fff640 1 --2- 172.21.15.98:0/2513674908 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc37406b4d0 0x7fc37406d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc378000c00 tx=0x7fc378001b80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:28.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.260+0000 7fc384ff9640 1 -- 172.21.15.98:0/2513674908 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc38806afc0 con 0x7fc38815b9e0 2024-05-09T10:50:28.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.431+0000 7fc38e9a5640 1 -- 172.21.15.98:0/2513674908 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc38806d770 con 0x7fc38815b9e0 2024-05-09T10:50:28.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.432+0000 7fc384ff9640 1 -- 172.21.15.98:0/2513674908 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7fc38806d770 con 0x7fc38815b9e0 2024-05-09T10:50:28.426 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:28.426 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":20,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:21.176419+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:50:17.323444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T10:50:18.482315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:28.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.434+0000 7fc38e9a5640 1 -- 172.21.15.98:0/2513674908 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc37406b4d0 msgr2=0x7fc37406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:28.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.434+0000 7fc38e9a5640 1 --2- 172.21.15.98:0/2513674908 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc37406b4d0 0x7fc37406d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc378000c00 tx=0x7fc378001b80 comp rx=0 tx=0).stop 2024-05-09T10:50:28.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.434+0000 7fc38e9a5640 1 -- 172.21.15.98:0/2513674908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc38815b9e0 msgr2=0x7fc38815be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:28.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.434+0000 7fc38e9a5640 1 --2- 172.21.15.98:0/2513674908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc38815b9e0 0x7fc38815be40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc380005970 tx=0x7fc3800015b0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.434+0000 7fc38e9a5640 1 -- 172.21.15.98:0/2513674908 shutdown_connections 2024-05-09T10:50:28.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.434+0000 7fc38e9a5640 1 --2- 172.21.15.98:0/2513674908 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc37406b4d0 0x7fc37406d990 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.434+0000 7fc38e9a5640 1 --2- 172.21.15.98:0/2513674908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc38815b9e0 0x7fc38815be40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.434+0000 7fc38e9a5640 1 --2- 172.21.15.98:0/2513674908 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc3880b2220 0x7fc3880b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.434+0000 7fc38e9a5640 1 --2- 172.21.15.98:0/2513674908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc370005120 0x7fc3881481d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.434+0000 7fc38e9a5640 1 -- 172.21.15.98:0/2513674908 >> 172.21.15.98:0/2513674908 conn(0x7fc38806abe0 msgr2=0x7fc3880cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:28.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.434+0000 7fc38e9a5640 1 -- 172.21.15.98:0/2513674908 shutdown_connections 2024-05-09T10:50:28.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.434+0000 7fc38e9a5640 1 -- 172.21.15.98:0/2513674908 wait complete. 2024-05-09T10:50:28.438 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T10:50:07.396423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T10:50:17.323444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T10:50:18.482315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T10:50:28.439 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T10:50:28.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.548+0000 7f06d6522640 1 Processor -- start 2024-05-09T10:50:28.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.548+0000 7f06d6522640 1 -- start start 2024-05-09T10:50:28.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.549+0000 7f06d6522640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f06d00cfbb0 0x7f06d0161b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:28.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.549+0000 7f06d6522640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f06d0162070 0x7f06d015d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:28.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.549+0000 7f06d6522640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d00d0090 con 0x7f06d00b2d40 2024-05-09T10:50:28.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.549+0000 7f06d6522640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d01624a0 con 0x7f06d00cfbb0 2024-05-09T10:50:28.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.549+0000 7f06d6522640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d00d0370 con 0x7f06d0162070 2024-05-09T10:50:28.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.549+0000 7f06c7fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f06d00cfbb0 0x7f06d0161b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:28.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.549+0000 7f06cffff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f06d00b2d40 0x7f06d006c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56058/0 (socket says 172.21.15.98:56058) 2024-05-09T10:50:28.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.549+0000 7f06cffff640 1 -- 172.21.15.98:0/2538829694 learned_addr learned my addr 172.21.15.98:0/2538829694 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:28.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.549+0000 7f06d4abf640 1 --2- 172.21.15.98:0/2538829694 >> v2:172.21.15.105:3300/0 conn(0x7f06d0162070 0x7f06d015d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:28.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.550+0000 7f06c7fff640 1 -- 172.21.15.98:0/2538829694 >> v2:172.21.15.105:3300/0 conn(0x7f06d0162070 msgr2=0x7f06d015d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:28.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.550+0000 7f06c7fff640 1 --2- 172.21.15.98:0/2538829694 >> v2:172.21.15.105:3300/0 conn(0x7f06d0162070 0x7f06d015d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.550+0000 7f06c7fff640 1 -- 172.21.15.98:0/2538829694 >> v1:172.21.15.98:6789/0 conn(0x7f06d00b2d40 legacy=0x7f06d006c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:28.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.550+0000 7f06cf7fe640 1 -- 172.21.15.98:0/2538829694 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3786170591 0 0) 0x7f06d00d0090 con 0x7f06d00b2d40 2024-05-09T10:50:28.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.550+0000 7f06c7fff640 1 -- 172.21.15.98:0/2538829694 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d00d0650 con 0x7f06d00cfbb0 2024-05-09T10:50:28.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.550+0000 7f06c7fff640 1 --2- 172.21.15.98:0/2538829694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06d00cfbb0 0x7f06d0161b30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f06d00d2950 tx=0x7f06b802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb35bc4f3d0b1d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:28.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.550+0000 7f06cf7fe640 1 -- 172.21.15.98:0/2538829694 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b8037070 con 0x7f06d00cfbb0 2024-05-09T10:50:28.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.550+0000 7f06cf7fe640 1 -- 172.21.15.98:0/2538829694 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b8032440 con 0x7f06d00cfbb0 2024-05-09T10:50:28.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.550+0000 7f06cf7fe640 1 -- 172.21.15.98:0/2538829694 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b802ae60 con 0x7f06d00cfbb0 2024-05-09T10:50:28.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.551+0000 7f06d6522640 1 -- 172.21.15.98:0/2538829694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06d00cfbb0 msgr2=0x7f06d0161b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:28.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.551+0000 7f06d6522640 1 --2- 172.21.15.98:0/2538829694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06d00cfbb0 0x7f06d0161b30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f06d00d2950 tx=0x7f06b802a110 comp rx=0 tx=0).stop 2024-05-09T10:50:28.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.551+0000 7f06d6522640 1 -- 172.21.15.98:0/2538829694 shutdown_connections 2024-05-09T10:50:28.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.551+0000 7f06d6522640 1 --2- 172.21.15.98:0/2538829694 >> v2:172.21.15.105:3300/0 conn(0x7f06d0162070 0x7f06d015d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.551+0000 7f06d6522640 1 --2- 172.21.15.98:0/2538829694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06d00cfbb0 0x7f06d0161b30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.551+0000 7f06d6522640 1 -- 172.21.15.98:0/2538829694 >> 172.21.15.98:0/2538829694 conn(0x7f06d00d1970 msgr2=0x7f06d00d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:28.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.551+0000 7f06d6522640 1 -- 172.21.15.98:0/2538829694 shutdown_connections 2024-05-09T10:50:28.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.551+0000 7f06d6522640 1 -- 172.21.15.98:0/2538829694 wait complete. 2024-05-09T10:50:28.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.552+0000 7f06d6522640 1 Processor -- start 2024-05-09T10:50:28.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.552+0000 7f06d6522640 1 -- start start 2024-05-09T10:50:28.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.553+0000 7f06d6522640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d00b2d40 0x7f06d01f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:28.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.553+0000 7f06d6522640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06d00cfbb0 0x7f06d0157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:28.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.553+0000 7f06d6522640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06d0162070 0x7f06d01e7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:28.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.553+0000 7f06d6522640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d006d020 con 0x7f06d0162070 2024-05-09T10:50:28.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.553+0000 7f06d6522640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d00d0c10 con 0x7f06d00cfbb0 2024-05-09T10:50:28.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.553+0000 7f06d6522640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d00b3120 con 0x7f06d00b2d40 2024-05-09T10:50:28.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.553+0000 7f06d4abf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06d0162070 0x7f06d01e7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:28.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.553+0000 7f06d4abf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06d0162070 0x7f06d01e7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42334/0 (socket says 172.21.15.98:42334) 2024-05-09T10:50:28.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.553+0000 7f06d4abf640 1 -- 172.21.15.98:0/1382872875 learned_addr learned my addr 172.21.15.98:0/1382872875 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:28.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.553+0000 7f06cffff640 1 --2- 172.21.15.98:0/1382872875 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d00b2d40 0x7f06d01f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:28.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.553+0000 7f06c7fff640 1 --2- 172.21.15.98:0/1382872875 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06d00cfbb0 0x7f06d0157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:28.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.553+0000 7f06d4abf640 1 -- 172.21.15.98:0/1382872875 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d00b2d40 msgr2=0x7f06d01f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:28.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.553+0000 7f06d4abf640 1 --2- 172.21.15.98:0/1382872875 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d00b2d40 0x7f06d01f3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.553+0000 7f06d4abf640 1 -- 172.21.15.98:0/1382872875 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06d00cfbb0 msgr2=0x7f06d0157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:28.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.553+0000 7f06d4abf640 1 --2- 172.21.15.98:0/1382872875 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06d00cfbb0 0x7f06d0157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.553+0000 7f06d4abf640 1 -- 172.21.15.98:0/1382872875 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d015b490 con 0x7f06d0162070 2024-05-09T10:50:28.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.553+0000 7f06cffff640 1 --2- 172.21.15.98:0/1382872875 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d00b2d40 0x7f06d01f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:28.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.553+0000 7f06c7fff640 1 --2- 172.21.15.98:0/1382872875 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06d00cfbb0 0x7f06d0157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:28.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.554+0000 7f06d4abf640 1 --2- 172.21.15.98:0/1382872875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06d0162070 0x7f06d01e7370 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f06c8007e40 tx=0x7f06c800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:28.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.554+0000 7f06cd7fa640 1 -- 172.21.15.98:0/1382872875 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c8014620 con 0x7f06d0162070 2024-05-09T10:50:28.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.554+0000 7f06cd7fa640 1 -- 172.21.15.98:0/1382872875 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c8019070 con 0x7f06d0162070 2024-05-09T10:50:28.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.554+0000 7f06d6522640 1 -- 172.21.15.98:0/1382872875 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d0157340 con 0x7f06d0162070 2024-05-09T10:50:28.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.554+0000 7f06cd7fa640 1 -- 172.21.15.98:0/1382872875 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c8014a20 con 0x7f06d0162070 2024-05-09T10:50:28.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.554+0000 7f06d6522640 1 -- 172.21.15.98:0/1382872875 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d01f3cd0 con 0x7f06d0162070 2024-05-09T10:50:28.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.555+0000 7f06d6522640 1 -- 172.21.15.98:0/1382872875 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d006ba50 con 0x7f06d0162070 2024-05-09T10:50:28.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.555+0000 7f06cd7fa640 1 -- 172.21.15.98:0/1382872875 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f06c8014bc0 con 0x7f06d0162070 2024-05-09T10:50:28.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.556+0000 7f06cd7fa640 1 --2- 172.21.15.98:0/1382872875 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f06b006b4d0 0x7f06b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:28.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.556+0000 7f06cd7fa640 1 -- 172.21.15.98:0/1382872875 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06c808aa00 con 0x7f06d0162070 2024-05-09T10:50:28.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.556+0000 7f06cffff640 1 --2- 172.21.15.98:0/1382872875 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f06b006b4d0 0x7f06b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:28.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.556+0000 7f06cffff640 1 --2- 172.21.15.98:0/1382872875 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f06b006b4d0 0x7f06b006d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f06c0000f60 tx=0x7f06c0007ca0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:28.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.560+0000 7f06cd7fa640 1 -- 172.21.15.98:0/1382872875 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f06c8059910 con 0x7f06d0162070 2024-05-09T10:50:28.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.728+0000 7f06d6522640 1 -- 172.21.15.98:0/1382872875 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f06d009fa40 con 0x7f06d0162070 2024-05-09T10:50:28.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.728+0000 7f06cd7fa640 1 -- 172.21.15.98:0/1382872875 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f06c8059210 con 0x7f06d0162070 2024-05-09T10:50:28.722 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-05-09T10:50:28.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.730+0000 7f06d6522640 1 -- 172.21.15.98:0/1382872875 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f06b006b4d0 msgr2=0x7f06b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:28.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.730+0000 7f06d6522640 1 --2- 172.21.15.98:0/1382872875 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f06b006b4d0 0x7f06b006d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f06c0000f60 tx=0x7f06c0007ca0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.730+0000 7f06d6522640 1 -- 172.21.15.98:0/1382872875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06d0162070 msgr2=0x7f06d01e7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:28.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.730+0000 7f06d6522640 1 --2- 172.21.15.98:0/1382872875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06d0162070 0x7f06d01e7370 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f06c8007e40 tx=0x7f06c800e4b0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.730+0000 7f06d6522640 1 -- 172.21.15.98:0/1382872875 shutdown_connections 2024-05-09T10:50:28.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.730+0000 7f06d6522640 1 --2- 172.21.15.98:0/1382872875 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f06b006b4d0 0x7f06b006d990 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.730+0000 7f06d6522640 1 --2- 172.21.15.98:0/1382872875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06d0162070 0x7f06d01e7370 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.730+0000 7f06d6522640 1 --2- 172.21.15.98:0/1382872875 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06d00cfbb0 0x7f06d0157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.730+0000 7f06d6522640 1 --2- 172.21.15.98:0/1382872875 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f06d00b2d40 0x7f06d01f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.730+0000 7f06d6522640 1 -- 172.21.15.98:0/1382872875 >> 172.21.15.98:0/1382872875 conn(0x7f06d00d1970 msgr2=0x7f06d006e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:28.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.730+0000 7f06d6522640 1 -- 172.21.15.98:0/1382872875 shutdown_connections 2024-05-09T10:50:28.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.730+0000 7f06d6522640 1 -- 172.21.15.98:0/1382872875 wait complete. 2024-05-09T10:50:28.735 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T10:50:28.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.843+0000 7f3d46934640 1 Processor -- start 2024-05-09T10:50:28.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.843+0000 7f3d46934640 1 -- start start 2024-05-09T10:50:28.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.843+0000 7f3d46934640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3d40159740 0x7f3d400c6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:28.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.843+0000 7f3d46934640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3d4015c9a0 0x7f3d400ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:28.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.843+0000 7f3d46934640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3d4015ce50 0x7f3d400aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:28.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.843+0000 7f3d46934640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d4006af30 con 0x7f3d40159740 2024-05-09T10:50:28.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.843+0000 7f3d46934640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d4006b210 con 0x7f3d4015c9a0 2024-05-09T10:50:28.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.843+0000 7f3d46934640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4006b4f0 con 0x7f3d4015ce50 2024-05-09T10:50:28.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.844+0000 7f3d3ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3d40159740 0x7f3d400c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:28.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.844+0000 7f3d3ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3d40159740 0x7f3d400c6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42336/0 (socket says 172.21.15.98:42336) 2024-05-09T10:50:28.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.844+0000 7f3d3ffff640 1 -- 172.21.15.98:0/1845744911 learned_addr learned my addr 172.21.15.98:0/1845744911 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:28.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.844+0000 7f3d44ed1640 1 --2- 172.21.15.98:0/1845744911 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3d4015ce50 0x7f3d400aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:28.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.844+0000 7f3d3f7fe640 1 --2- 172.21.15.98:0/1845744911 >> v2:172.21.15.105:3300/0 conn(0x7f3d4015c9a0 0x7f3d400ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:28.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.844+0000 7f3d3ffff640 1 -- 172.21.15.98:0/1845744911 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3d4015ce50 msgr2=0x7f3d400aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:28.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.844+0000 7f3d3ffff640 1 --2- 172.21.15.98:0/1845744911 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3d4015ce50 0x7f3d400aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.844+0000 7f3d3ffff640 1 -- 172.21.15.98:0/1845744911 >> v2:172.21.15.105:3300/0 conn(0x7f3d4015c9a0 msgr2=0x7f3d400ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:28.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.844+0000 7f3d3ffff640 1 --2- 172.21.15.98:0/1845744911 >> v2:172.21.15.105:3300/0 conn(0x7f3d4015c9a0 0x7f3d400ae6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.844+0000 7f3d3ffff640 1 -- 172.21.15.98:0/1845744911 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4015c480 con 0x7f3d40159740 2024-05-09T10:50:28.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.844+0000 7f3d3f7fe640 1 --2- 172.21.15.98:0/1845744911 >> v2:172.21.15.105:3300/0 conn(0x7f3d4015c9a0 0x7f3d400ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:28.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.845+0000 7f3d3ffff640 1 --2- 172.21.15.98:0/1845744911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d40159740 0x7f3d400c6080 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3d30009870 tx=0x7f3d3002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8988182c6961a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:28.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.845+0000 7f3d3effd640 1 -- 172.21.15.98:0/1845744911 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d30037070 con 0x7f3d40159740 2024-05-09T10:50:28.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.845+0000 7f3d3effd640 1 -- 172.21.15.98:0/1845744911 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d30032440 con 0x7f3d40159740 2024-05-09T10:50:28.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.845+0000 7f3d3effd640 1 -- 172.21.15.98:0/1845744911 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d30032810 con 0x7f3d40159740 2024-05-09T10:50:28.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.845+0000 7f3d46934640 1 -- 172.21.15.98:0/1845744911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d40159740 msgr2=0x7f3d400c6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:28.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.845+0000 7f3d46934640 1 --2- 172.21.15.98:0/1845744911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d40159740 0x7f3d400c6080 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3d30009870 tx=0x7f3d3002a210 comp rx=0 tx=0).stop 2024-05-09T10:50:28.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.845+0000 7f3d46934640 1 -- 172.21.15.98:0/1845744911 shutdown_connections 2024-05-09T10:50:28.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.845+0000 7f3d46934640 1 --2- 172.21.15.98:0/1845744911 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3d4015ce50 0x7f3d400aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.845+0000 7f3d46934640 1 --2- 172.21.15.98:0/1845744911 >> v2:172.21.15.105:3300/0 conn(0x7f3d4015c9a0 0x7f3d400ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.845+0000 7f3d46934640 1 --2- 172.21.15.98:0/1845744911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d40159740 0x7f3d400c6080 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.845+0000 7f3d46934640 1 -- 172.21.15.98:0/1845744911 >> 172.21.15.98:0/1845744911 conn(0x7f3d400cf760 msgr2=0x7f3d400cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:28.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.845+0000 7f3d46934640 1 -- 172.21.15.98:0/1845744911 shutdown_connections 2024-05-09T10:50:28.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.845+0000 7f3d46934640 1 -- 172.21.15.98:0/1845744911 wait complete. 2024-05-09T10:50:28.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.846+0000 7f3d46934640 1 Processor -- start 2024-05-09T10:50:28.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.847+0000 7f3d46934640 1 -- start start 2024-05-09T10:50:28.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.847+0000 7f3d46934640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d40159740 0x7f3d4014a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:28.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.847+0000 7f3d46934640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3d4015c9a0 0x7f3d400ad880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:28.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.847+0000 7f3d46934640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d4015ce50 0x7f3d400addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:28.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.847+0000 7f3d46934640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4015c170 con 0x7f3d4015ce50 2024-05-09T10:50:28.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.847+0000 7f3d46934640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d40159b20 con 0x7f3d40159740 2024-05-09T10:50:28.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.847+0000 7f3d46934640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4006abe0 con 0x7f3d4015c9a0 2024-05-09T10:50:28.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.847+0000 7f3d44ed1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d4015ce50 0x7f3d400addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:28.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.848+0000 7f3d44ed1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d4015ce50 0x7f3d400addc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42340/0 (socket says 172.21.15.98:42340) 2024-05-09T10:50:28.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.848+0000 7f3d44ed1640 1 -- 172.21.15.98:0/1875543015 learned_addr learned my addr 172.21.15.98:0/1875543015 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:28.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.848+0000 7f3d3ffff640 1 --2- 172.21.15.98:0/1875543015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d40159740 0x7f3d4014a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:28.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.848+0000 7f3d3f7fe640 1 --2- 172.21.15.98:0/1875543015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3d4015c9a0 0x7f3d400ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:28.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.848+0000 7f3d44ed1640 1 -- 172.21.15.98:0/1875543015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3d4015c9a0 msgr2=0x7f3d400ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:28.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.848+0000 7f3d44ed1640 1 --2- 172.21.15.98:0/1875543015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3d4015c9a0 0x7f3d400ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.848+0000 7f3d44ed1640 1 -- 172.21.15.98:0/1875543015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d40159740 msgr2=0x7f3d4014a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:28.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.848+0000 7f3d44ed1640 1 --2- 172.21.15.98:0/1875543015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d40159740 0x7f3d4014a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:28.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.848+0000 7f3d44ed1640 1 -- 172.21.15.98:0/1875543015 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4015b750 con 0x7f3d4015ce50 2024-05-09T10:50:28.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.848+0000 7f3d44ed1640 1 --2- 172.21.15.98:0/1875543015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d4015ce50 0x7f3d400addc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3d38007d80 tx=0x7f3d3800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:28.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.848+0000 7f3d3cff9640 1 -- 172.21.15.98:0/1875543015 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d38014420 con 0x7f3d4015ce50 2024-05-09T10:50:28.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.848+0000 7f3d3cff9640 1 -- 172.21.15.98:0/1875543015 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d38019070 con 0x7f3d4015ce50 2024-05-09T10:50:28.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.848+0000 7f3d46934640 1 -- 172.21.15.98:0/1875543015 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d4015ba30 con 0x7f3d4015ce50 2024-05-09T10:50:28.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.848+0000 7f3d3cff9640 1 -- 172.21.15.98:0/1875543015 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d38014800 con 0x7f3d4015ce50 2024-05-09T10:50:28.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.848+0000 7f3d46934640 1 -- 172.21.15.98:0/1875543015 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d4015aa00 con 0x7f3d4015ce50 2024-05-09T10:50:28.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.849+0000 7f3d46934640 1 -- 172.21.15.98:0/1875543015 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d40072c80 con 0x7f3d4015ce50 2024-05-09T10:50:28.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.850+0000 7f3d3cff9640 1 -- 172.21.15.98:0/1875543015 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d380149a0 con 0x7f3d4015ce50 2024-05-09T10:50:28.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.850+0000 7f3d3cff9640 1 --2- 172.21.15.98:0/1875543015 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3d2c06b4d0 0x7f3d2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:28.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.850+0000 7f3d3cff9640 1 -- 172.21.15.98:0/1875543015 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d3808a870 con 0x7f3d4015ce50 2024-05-09T10:50:28.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.850+0000 7f3d3ffff640 1 --2- 172.21.15.98:0/1875543015 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3d2c06b4d0 0x7f3d2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:28.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.851+0000 7f3d3ffff640 1 --2- 172.21.15.98:0/1875543015 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3d2c06b4d0 0x7f3d2c06d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3d30034000 tx=0x7f3d3002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:28.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:28.854+0000 7f3d3cff9640 1 -- 172.21.15.98:0/1875543015 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3d380590a0 con 0x7f3d4015ce50 2024-05-09T10:50:29.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.025+0000 7f3d46934640 1 -- 172.21.15.98:0/1875543015 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3d4015bf50 con 0x7f3d4015ce50 2024-05-09T10:50:29.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.026+0000 7f3d3cff9640 1 -- 172.21.15.98:0/1875543015 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3d3805d140 con 0x7f3d4015ce50 2024-05-09T10:50:29.019 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2024-05-09T10:50:29.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.028+0000 7f3d46934640 1 -- 172.21.15.98:0/1875543015 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3d2c06b4d0 msgr2=0x7f3d2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:29.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.028+0000 7f3d46934640 1 --2- 172.21.15.98:0/1875543015 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3d2c06b4d0 0x7f3d2c06d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3d30034000 tx=0x7f3d3002e000 comp rx=0 tx=0).stop 2024-05-09T10:50:29.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.028+0000 7f3d46934640 1 -- 172.21.15.98:0/1875543015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d4015ce50 msgr2=0x7f3d400addc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:29.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.028+0000 7f3d46934640 1 --2- 172.21.15.98:0/1875543015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d4015ce50 0x7f3d400addc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3d38007d80 tx=0x7f3d3800e4b0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.028+0000 7f3d46934640 1 -- 172.21.15.98:0/1875543015 shutdown_connections 2024-05-09T10:50:29.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.028+0000 7f3d46934640 1 --2- 172.21.15.98:0/1875543015 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3d2c06b4d0 0x7f3d2c06d990 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.028+0000 7f3d46934640 1 --2- 172.21.15.98:0/1875543015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d4015ce50 0x7f3d400addc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.028+0000 7f3d46934640 1 --2- 172.21.15.98:0/1875543015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3d4015c9a0 0x7f3d400ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.028+0000 7f3d46934640 1 --2- 172.21.15.98:0/1875543015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d40159740 0x7f3d4014a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.028+0000 7f3d46934640 1 -- 172.21.15.98:0/1875543015 >> 172.21.15.98:0/1875543015 conn(0x7f3d400cf760 msgr2=0x7f3d400c4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:29.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.028+0000 7f3d46934640 1 -- 172.21.15.98:0/1875543015 shutdown_connections 2024-05-09T10:50:29.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.028+0000 7f3d46934640 1 -- 172.21.15.98:0/1875543015 wait complete. 2024-05-09T10:50:29.032 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T10:50:29.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.141+0000 7ff2895b6640 1 Processor -- start 2024-05-09T10:50:29.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.141+0000 7ff2895b6640 1 -- start start 2024-05-09T10:50:29.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.142+0000 7ff2895b6640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff2840d1be0 0x7ff2840d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:29.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.142+0000 7ff2895b6640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2840d1fc0 0x7ff28415cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:29.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.142+0000 7ff2895b6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2840d1710 con 0x7ff2840cf950 2024-05-09T10:50:29.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.142+0000 7ff2895b6640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2840b2bb0 con 0x7ff2840d1be0 2024-05-09T10:50:29.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.142+0000 7ff2895b6640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2840b2e90 con 0x7ff2840d1fc0 2024-05-09T10:50:29.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.142+0000 7ff282ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff2840d1be0 0x7ff2840d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:29.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.142+0000 7ff282ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff2840d1be0 0x7ff2840d11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42346/0 (socket says 172.21.15.98:42346) 2024-05-09T10:50:29.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.142+0000 7ff282ffd640 1 -- 172.21.15.98:0/4067259967 learned_addr learned my addr 172.21.15.98:0/4067259967 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:29.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.143+0000 7ff2837fe640 1 --2- 172.21.15.98:0/4067259967 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2840d1fc0 0x7ff28415cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:29.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.143+0000 7ff282ffd640 1 -- 172.21.15.98:0/4067259967 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2840d1fc0 msgr2=0x7ff28415cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:29.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.143+0000 7ff282ffd640 1 --2- 172.21.15.98:0/4067259967 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2840d1fc0 0x7ff28415cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.143+0000 7ff282ffd640 1 -- 172.21.15.98:0/4067259967 >> v1:172.21.15.105:6789/0 conn(0x7ff2840cf950 legacy=0x7ff2840ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:29.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.143+0000 7ff282ffd640 1 -- 172.21.15.98:0/4067259967 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2840b3170 con 0x7ff2840d1be0 2024-05-09T10:50:29.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.143+0000 7ff282ffd640 1 --2- 172.21.15.98:0/4067259967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2840d1be0 0x7ff2840d11d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff284070d60 tx=0x7ff27402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3ce1c24520deed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:29.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.143+0000 7ff281ffb640 1 -- 172.21.15.98:0/4067259967 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff274037070 con 0x7ff2840d1be0 2024-05-09T10:50:29.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.143+0000 7ff281ffb640 1 -- 172.21.15.98:0/4067259967 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27402ae50 con 0x7ff2840d1be0 2024-05-09T10:50:29.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.143+0000 7ff281ffb640 1 -- 172.21.15.98:0/4067259967 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff274032660 con 0x7ff2840d1be0 2024-05-09T10:50:29.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.144+0000 7ff2895b6640 1 -- 172.21.15.98:0/4067259967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2840d1be0 msgr2=0x7ff2840d11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:29.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.144+0000 7ff2895b6640 1 --2- 172.21.15.98:0/4067259967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2840d1be0 0x7ff2840d11d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff284070d60 tx=0x7ff27402a140 comp rx=0 tx=0).stop 2024-05-09T10:50:29.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.144+0000 7ff2895b6640 1 -- 172.21.15.98:0/4067259967 shutdown_connections 2024-05-09T10:50:29.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.144+0000 7ff2895b6640 1 --2- 172.21.15.98:0/4067259967 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2840d1fc0 0x7ff28415cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.144+0000 7ff2895b6640 1 --2- 172.21.15.98:0/4067259967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2840d1be0 0x7ff2840d11d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.144+0000 7ff2895b6640 1 -- 172.21.15.98:0/4067259967 >> 172.21.15.98:0/4067259967 conn(0x7ff2840b6db0 msgr2=0x7ff2840b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:29.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.144+0000 7ff2895b6640 1 -- 172.21.15.98:0/4067259967 shutdown_connections 2024-05-09T10:50:29.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.144+0000 7ff2895b6640 1 -- 172.21.15.98:0/4067259967 wait complete. 2024-05-09T10:50:29.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.145+0000 7ff2895b6640 1 Processor -- start 2024-05-09T10:50:29.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.145+0000 7ff2895b6640 1 -- start start 2024-05-09T10:50:29.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.146+0000 7ff2895b6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2840cf950 0x7ff2841f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:29.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.146+0000 7ff2895b6640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2840d1be0 0x7ff2840afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:29.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.146+0000 7ff2895b6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2840d1fc0 0x7ff28414aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:29.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.146+0000 7ff2895b6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28415d0e0 con 0x7ff2840cf950 2024-05-09T10:50:29.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.146+0000 7ff2895b6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2840b2190 con 0x7ff2840d1fc0 2024-05-09T10:50:29.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.146+0000 7ff2895b6640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28415d260 con 0x7ff2840d1be0 2024-05-09T10:50:29.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.146+0000 7ff282ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2840cf950 0x7ff2841f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:29.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.146+0000 7ff282ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2840cf950 0x7ff2841f7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42356/0 (socket says 172.21.15.98:42356) 2024-05-09T10:50:29.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.146+0000 7ff282ffd640 1 -- 172.21.15.98:0/1131204097 learned_addr learned my addr 172.21.15.98:0/1131204097 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:29.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.146+0000 7ff2837fe640 1 --2- 172.21.15.98:0/1131204097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2840d1fc0 0x7ff28414aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:29.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.146+0000 7ff282ffd640 1 -- 172.21.15.98:0/1131204097 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2840d1be0 msgr2=0x7ff2840afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:29.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.146+0000 7ff282ffd640 1 --2- 172.21.15.98:0/1131204097 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2840d1be0 0x7ff2840afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.146+0000 7ff282ffd640 1 -- 172.21.15.98:0/1131204097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2840d1fc0 msgr2=0x7ff28414aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:29.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.146+0000 7ff282ffd640 1 --2- 172.21.15.98:0/1131204097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2840d1fc0 0x7ff28414aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.146+0000 7ff282ffd640 1 -- 172.21.15.98:0/1131204097 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2841f8050 con 0x7ff2840cf950 2024-05-09T10:50:29.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.146+0000 7ff2837fe640 1 --2- 172.21.15.98:0/1131204097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2840d1fc0 0x7ff28414aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:29.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.147+0000 7ff282ffd640 1 --2- 172.21.15.98:0/1131204097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2840cf950 0x7ff2841f7b10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff274000c00 tx=0x7ff2740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:29.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.147+0000 7ff26b7fe640 1 -- 172.21.15.98:0/1131204097 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff274041070 con 0x7ff2840cf950 2024-05-09T10:50:29.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.147+0000 7ff26b7fe640 1 -- 172.21.15.98:0/1131204097 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27403c670 con 0x7ff2840cf950 2024-05-09T10:50:29.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.147+0000 7ff2895b6640 1 -- 172.21.15.98:0/1131204097 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff284159790 con 0x7ff2840cf950 2024-05-09T10:50:29.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.147+0000 7ff26b7fe640 1 -- 172.21.15.98:0/1131204097 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff274037040 con 0x7ff2840cf950 2024-05-09T10:50:29.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.147+0000 7ff2895b6640 1 -- 172.21.15.98:0/1131204097 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff284156580 con 0x7ff2840cf950 2024-05-09T10:50:29.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.148+0000 7ff2895b6640 1 -- 172.21.15.98:0/1131204097 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff28409fbb0 con 0x7ff2840cf950 2024-05-09T10:50:29.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.148+0000 7ff26b7fe640 1 -- 172.21.15.98:0/1131204097 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff274044070 con 0x7ff2840cf950 2024-05-09T10:50:29.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.148+0000 7ff26b7fe640 1 --2- 172.21.15.98:0/1131204097 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff27006b4d0 0x7ff27006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:29.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.149+0000 7ff26b7fe640 1 -- 172.21.15.98:0/1131204097 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2740aa6a0 con 0x7ff2840cf950 2024-05-09T10:50:29.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.149+0000 7ff2827fc640 1 --2- 172.21.15.98:0/1131204097 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff27006b4d0 0x7ff27006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:29.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.149+0000 7ff2827fc640 1 --2- 172.21.15.98:0/1131204097 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff27006b4d0 0x7ff27006d990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff278006fd0 tx=0x7ff278006bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:29.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.153+0000 7ff26b7fe640 1 -- 172.21.15.98:0/1131204097 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff2841f8050 con 0x7ff2840cf950 2024-05-09T10:50:29.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.323+0000 7ff2895b6640 1 -- 172.21.15.98:0/1131204097 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff2840b72c0 con 0x7ff2840cf950 2024-05-09T10:50:29.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.323+0000 7ff26b7fe640 1 -- 172.21.15.98:0/1131204097 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff2840d1710 con 0x7ff2840cf950 2024-05-09T10:50:29.317 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2024-05-09T10:50:29.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.325+0000 7ff2895b6640 1 -- 172.21.15.98:0/1131204097 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff27006b4d0 msgr2=0x7ff27006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:29.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.325+0000 7ff2895b6640 1 --2- 172.21.15.98:0/1131204097 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff27006b4d0 0x7ff27006d990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff278006fd0 tx=0x7ff278006bc0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.325+0000 7ff2895b6640 1 -- 172.21.15.98:0/1131204097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2840cf950 msgr2=0x7ff2841f7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:29.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.325+0000 7ff2895b6640 1 --2- 172.21.15.98:0/1131204097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2840cf950 0x7ff2841f7b10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff274000c00 tx=0x7ff2740046c0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.325+0000 7ff2895b6640 1 -- 172.21.15.98:0/1131204097 shutdown_connections 2024-05-09T10:50:29.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.325+0000 7ff2895b6640 1 --2- 172.21.15.98:0/1131204097 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff27006b4d0 0x7ff27006d990 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.325+0000 7ff2895b6640 1 --2- 172.21.15.98:0/1131204097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2840d1fc0 0x7ff28414aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.325+0000 7ff2895b6640 1 --2- 172.21.15.98:0/1131204097 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2840d1be0 0x7ff2840afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.325+0000 7ff2895b6640 1 --2- 172.21.15.98:0/1131204097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2840cf950 0x7ff2841f7b10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.325+0000 7ff2895b6640 1 -- 172.21.15.98:0/1131204097 >> 172.21.15.98:0/1131204097 conn(0x7ff2840b6db0 msgr2=0x7ff28415d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:29.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.326+0000 7ff2895b6640 1 -- 172.21.15.98:0/1131204097 shutdown_connections 2024-05-09T10:50:29.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.326+0000 7ff2895b6640 1 -- 172.21.15.98:0/1131204097 wait complete. 2024-05-09T10:50:29.330 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T10:50:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.436+0000 7f6a60b7e640 1 Processor -- start 2024-05-09T10:50:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.437+0000 7f6a60b7e640 1 -- start start 2024-05-09T10:50:29.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.437+0000 7f6a60b7e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6a5c15de40 0x7f6a5c0d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:29.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.438+0000 7f6a60b7e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a5c15e220 con 0x7f6a5c0d0b80 2024-05-09T10:50:29.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.438+0000 7f6a60b7e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a5c0d19e0 con 0x7f6a5c0cf250 2024-05-09T10:50:29.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.438+0000 7f6a60b7e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5c0d1790 con 0x7f6a5c15de40 2024-05-09T10:50:29.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.438+0000 7f6a5ad9d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f6a5c0d0b80 0x7f6a5c0b6a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56070/0 (socket says 172.21.15.98:56070) 2024-05-09T10:50:29.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.438+0000 7f6a5a59c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6a5c15de40 0x7f6a5c0d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:29.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.438+0000 7f6a5ad9d640 1 -- 172.21.15.98:0/4286656657 learned_addr learned my addr 172.21.15.98:0/4286656657 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:29.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.438+0000 7f6a5a59c640 1 -- 172.21.15.98:0/4286656657 >> v1:172.21.15.105:6789/0 conn(0x7f6a5c0cf250 legacy=0x7f6a5c0d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:29.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.438+0000 7f6a5a59c640 1 -- 172.21.15.98:0/4286656657 >> v1:172.21.15.98:6789/0 conn(0x7f6a5c0d0b80 legacy=0x7f6a5c0b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:29.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.438+0000 7f6a5959a640 1 -- 172.21.15.98:0/4286656657 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 335857158 0 0) 0x7f6a5c15e220 con 0x7f6a5c0d0b80 2024-05-09T10:50:29.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.438+0000 7f6a5a59c640 1 -- 172.21.15.98:0/4286656657 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c0d1cc0 con 0x7f6a5c15de40 2024-05-09T10:50:29.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.439+0000 7f6a5a59c640 1 --2- 172.21.15.98:0/4286656657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a5c15de40 0x7f6a5c0d27a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009d90 tx=0x7f6a5402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f02a2fd502c3dd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:29.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.439+0000 7f6a5959a640 1 -- 172.21.15.98:0/4286656657 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54007740 con 0x7f6a5c15de40 2024-05-09T10:50:29.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.439+0000 7f6a5959a640 1 -- 172.21.15.98:0/4286656657 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a540078e0 con 0x7f6a5c15de40 2024-05-09T10:50:29.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.439+0000 7f6a5959a640 1 -- 172.21.15.98:0/4286656657 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54007cb0 con 0x7f6a5c15de40 2024-05-09T10:50:29.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.439+0000 7f6a60b7e640 1 -- 172.21.15.98:0/4286656657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a5c15de40 msgr2=0x7f6a5c0d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:29.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.439+0000 7f6a60b7e640 1 --2- 172.21.15.98:0/4286656657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a5c15de40 0x7f6a5c0d27a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009d90 tx=0x7f6a5402a6a0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.439+0000 7f6a60b7e640 1 -- 172.21.15.98:0/4286656657 shutdown_connections 2024-05-09T10:50:29.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.439+0000 7f6a60b7e640 1 --2- 172.21.15.98:0/4286656657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a5c15de40 0x7f6a5c0d27a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.439+0000 7f6a60b7e640 1 -- 172.21.15.98:0/4286656657 >> 172.21.15.98:0/4286656657 conn(0x7f6a5c09ce90 msgr2=0x7f6a5c0d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:29.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.439+0000 7f6a60b7e640 1 -- 172.21.15.98:0/4286656657 shutdown_connections 2024-05-09T10:50:29.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.439+0000 7f6a60b7e640 1 -- 172.21.15.98:0/4286656657 wait complete. 2024-05-09T10:50:29.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.441+0000 7f6a60b7e640 1 Processor -- start 2024-05-09T10:50:29.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.441+0000 7f6a60b7e640 1 -- start start 2024-05-09T10:50:29.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.442+0000 7f6a60b7e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a5c0cf250 0x7f6a5c09f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:29.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.442+0000 7f6a60b7e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a5c0d0b80 0x7f6a5c06c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:29.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.442+0000 7f6a60b7e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a5c15de40 0x7f6a5c152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:29.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.442+0000 7f6a60b7e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c0d4680 con 0x7f6a5c0d0b80 2024-05-09T10:50:29.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.442+0000 7f6a60b7e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c0d2280 con 0x7f6a5c0cf250 2024-05-09T10:50:29.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.442+0000 7f6a60b7e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c0d4800 con 0x7f6a5c15de40 2024-05-09T10:50:29.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.442+0000 7f6a59d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a5c0d0b80 0x7f6a5c06c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:29.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.442+0000 7f6a59d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a5c0d0b80 0x7f6a5c06c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42370/0 (socket says 172.21.15.98:42370) 2024-05-09T10:50:29.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.442+0000 7f6a59d9b640 1 -- 172.21.15.98:0/253165149 learned_addr learned my addr 172.21.15.98:0/253165149 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:29.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.442+0000 7f6a5a59c640 1 --2- 172.21.15.98:0/253165149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a5c0cf250 0x7f6a5c09f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:29.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.442+0000 7f6a5ad9d640 1 --2- 172.21.15.98:0/253165149 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a5c15de40 0x7f6a5c152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:29.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.442+0000 7f6a59d9b640 1 -- 172.21.15.98:0/253165149 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a5c15de40 msgr2=0x7f6a5c152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:29.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.442+0000 7f6a59d9b640 1 --2- 172.21.15.98:0/253165149 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a5c15de40 0x7f6a5c152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.442+0000 7f6a59d9b640 1 -- 172.21.15.98:0/253165149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a5c0cf250 msgr2=0x7f6a5c09f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:29.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.442+0000 7f6a59d9b640 1 --2- 172.21.15.98:0/253165149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a5c0cf250 0x7f6a5c09f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.442+0000 7f6a59d9b640 1 -- 172.21.15.98:0/253165149 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c09e860 con 0x7f6a5c0d0b80 2024-05-09T10:50:29.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.442+0000 7f6a59d9b640 1 --2- 172.21.15.98:0/253165149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a5c0d0b80 0x7f6a5c06c8a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6a4400c460 tx=0x7f6a4400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:29.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.442+0000 7f6a52ffd640 1 -- 172.21.15.98:0/253165149 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a44016020 con 0x7f6a5c0d0b80 2024-05-09T10:50:29.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.442+0000 7f6a52ffd640 1 -- 172.21.15.98:0/253165149 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a44012870 con 0x7f6a5c0d0b80 2024-05-09T10:50:29.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.442+0000 7f6a60b7e640 1 -- 172.21.15.98:0/253165149 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5c09eb40 con 0x7f6a5c0d0b80 2024-05-09T10:50:29.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.442+0000 7f6a52ffd640 1 -- 172.21.15.98:0/253165149 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a44012c60 con 0x7f6a5c0d0b80 2024-05-09T10:50:29.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.442+0000 7f6a60b7e640 1 -- 172.21.15.98:0/253165149 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5c09f100 con 0x7f6a5c0d0b80 2024-05-09T10:50:29.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.443+0000 7f6a60b7e640 1 -- 172.21.15.98:0/253165149 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a5c069cb0 con 0x7f6a5c0d0b80 2024-05-09T10:50:29.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.444+0000 7f6a52ffd640 1 -- 172.21.15.98:0/253165149 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a44019020 con 0x7f6a5c0d0b80 2024-05-09T10:50:29.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.444+0000 7f6a52ffd640 1 --2- 172.21.15.98:0/253165149 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6a3c06b4d0 0x7f6a3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:29.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.444+0000 7f6a52ffd640 1 -- 172.21.15.98:0/253165149 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a44089d40 con 0x7f6a5c0d0b80 2024-05-09T10:50:29.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.444+0000 7f6a5a59c640 1 --2- 172.21.15.98:0/253165149 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6a3c06b4d0 0x7f6a3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:29.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.445+0000 7f6a5a59c640 1 --2- 172.21.15.98:0/253165149 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6a3c06b4d0 0x7f6a3c06d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6a54034000 tx=0x7f6a5402f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:29.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.449+0000 7f6a52ffd640 1 -- 172.21.15.98:0/253165149 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a44020030 con 0x7f6a5c0d0b80 2024-05-09T10:50:29.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.657+0000 7f6a60b7e640 1 -- 172.21.15.98:0/253165149 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a5c0b9480 con 0x7f6a5c0d0b80 2024-05-09T10:50:29.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.657+0000 7f6a52ffd640 1 -- 172.21.15.98:0/253165149 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f6a44058600 con 0x7f6a5c0d0b80 2024-05-09T10:50:29.651 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:29.651 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2903091926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2903091926},{"type":"v1","addr":"172.21.15.98:6837","nonce":2903091926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/968936316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":968936316},{"type":"v1","addr":"172.21.15.105:6834","nonce":968936316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/937700297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":937700297},{"type":"v1","addr":"172.21.15.98:6835","nonce":937700297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T10:50:19.544432+0000","modified":"2024-05-09T10:50:21.512942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6835/1780062557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1780062557},{"type":"v1","addr":"172.21.15.105:6835","nonce":1780062557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T10:50:29.651 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 6 2024-05-09T10:50:29.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.659+0000 7f6a60b7e640 1 -- 172.21.15.98:0/253165149 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6a3c06b4d0 msgr2=0x7f6a3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:29.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.659+0000 7f6a60b7e640 1 --2- 172.21.15.98:0/253165149 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6a3c06b4d0 0x7f6a3c06d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6a54034000 tx=0x7f6a5402f000 comp rx=0 tx=0).stop 2024-05-09T10:50:29.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.659+0000 7f6a60b7e640 1 -- 172.21.15.98:0/253165149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a5c0d0b80 msgr2=0x7f6a5c06c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:29.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.659+0000 7f6a60b7e640 1 --2- 172.21.15.98:0/253165149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a5c0d0b80 0x7f6a5c06c8a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6a4400c460 tx=0x7f6a4400c920 comp rx=0 tx=0).stop 2024-05-09T10:50:29.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.659+0000 7f6a60b7e640 1 -- 172.21.15.98:0/253165149 shutdown_connections 2024-05-09T10:50:29.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.659+0000 7f6a60b7e640 1 --2- 172.21.15.98:0/253165149 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6a3c06b4d0 0x7f6a3c06d990 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.659+0000 7f6a60b7e640 1 --2- 172.21.15.98:0/253165149 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a5c15de40 0x7f6a5c152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.659+0000 7f6a60b7e640 1 --2- 172.21.15.98:0/253165149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a5c0d0b80 0x7f6a5c06c8a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.659+0000 7f6a60b7e640 1 --2- 172.21.15.98:0/253165149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a5c0cf250 0x7f6a5c09f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.659+0000 7f6a60b7e640 1 -- 172.21.15.98:0/253165149 >> 172.21.15.98:0/253165149 conn(0x7f6a5c09ce90 msgr2=0x7f6a5c0b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:29.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.659+0000 7f6a60b7e640 1 -- 172.21.15.98:0/253165149 shutdown_connections 2024-05-09T10:50:29.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.659+0000 7f6a60b7e640 1 -- 172.21.15.98:0/253165149 wait complete. 2024-05-09T10:50:29.664 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:29.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.769+0000 7fe970c5a640 1 Processor -- start 2024-05-09T10:50:29.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.770+0000 7fe970c5a640 1 -- start start 2024-05-09T10:50:29.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.770+0000 7fe970c5a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe96c15be50 0x7fe96c0cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:29.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.770+0000 7fe970c5a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe96c0b3610 con 0x7fe96c0b43d0 2024-05-09T10:50:29.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.770+0000 7fe970c5a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe96c0b38f0 con 0x7fe96c15ba70 2024-05-09T10:50:29.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.770+0000 7fe970c5a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe96c0b3bd0 con 0x7fe96c15be50 2024-05-09T10:50:29.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.771+0000 7fe963fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe96c15be50 0x7fe96c0cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:29.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.771+0000 7fe96ad9d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe96c0b43d0 0x7fe96c06cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56086/0 (socket says 172.21.15.98:56086) 2024-05-09T10:50:29.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.771+0000 7fe96ad9d640 1 -- 172.21.15.98:0/2753508598 learned_addr learned my addr 172.21.15.98:0/2753508598 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:29.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.771+0000 7fe963fff640 1 -- 172.21.15.98:0/2753508598 >> v1:172.21.15.105:6789/0 conn(0x7fe96c15ba70 legacy=0x7fe96c0cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:29.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.771+0000 7fe963fff640 1 -- 172.21.15.98:0/2753508598 >> v1:172.21.15.98:6789/0 conn(0x7fe96c0b43d0 legacy=0x7fe96c06cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:29.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.771+0000 7fe969d9b640 1 -- 172.21.15.98:0/2753508598 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2333105371 0 0) 0x7fe96c0b3610 con 0x7fe96c0b43d0 2024-05-09T10:50:29.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.771+0000 7fe963fff640 1 -- 172.21.15.98:0/2753508598 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96c0b3230 con 0x7fe96c15be50 2024-05-09T10:50:29.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.771+0000 7fe963fff640 1 --2- 172.21.15.98:0/2753508598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe96c15be50 0x7fe96c0cf440 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe9540097a0 tx=0x7fe95402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5840e08035a407c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:29.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.772+0000 7fe969d9b640 1 -- 172.21.15.98:0/2753508598 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954037070 con 0x7fe96c15be50 2024-05-09T10:50:29.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.772+0000 7fe969d9b640 1 -- 172.21.15.98:0/2753508598 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe954032440 con 0x7fe96c15be50 2024-05-09T10:50:29.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.772+0000 7fe969d9b640 1 -- 172.21.15.98:0/2753508598 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954032810 con 0x7fe96c15be50 2024-05-09T10:50:29.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.772+0000 7fe970c5a640 1 -- 172.21.15.98:0/2753508598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe96c15be50 msgr2=0x7fe96c0cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:29.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.772+0000 7fe970c5a640 1 --2- 172.21.15.98:0/2753508598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe96c15be50 0x7fe96c0cf440 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe9540097a0 tx=0x7fe95402a230 comp rx=0 tx=0).stop 2024-05-09T10:50:29.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.772+0000 7fe970c5a640 1 -- 172.21.15.98:0/2753508598 shutdown_connections 2024-05-09T10:50:29.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.772+0000 7fe970c5a640 1 --2- 172.21.15.98:0/2753508598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe96c15be50 0x7fe96c0cf440 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.772+0000 7fe970c5a640 1 -- 172.21.15.98:0/2753508598 >> 172.21.15.98:0/2753508598 conn(0x7fe96c06abe0 msgr2=0x7fe96c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:29.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.772+0000 7fe970c5a640 1 -- 172.21.15.98:0/2753508598 shutdown_connections 2024-05-09T10:50:29.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.772+0000 7fe970c5a640 1 -- 172.21.15.98:0/2753508598 wait complete. 2024-05-09T10:50:29.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.774+0000 7fe970c5a640 1 Processor -- start 2024-05-09T10:50:29.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.774+0000 7fe970c5a640 1 -- start start 2024-05-09T10:50:29.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.774+0000 7fe970c5a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe96c0b43d0 0x7fe96c156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:29.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.774+0000 7fe970c5a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe96c15ba70 0x7fe96c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:29.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.774+0000 7fe970c5a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe96c15be50 0x7fe96c152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:29.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.774+0000 7fe970c5a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c0cf980 con 0x7fe96c0b43d0 2024-05-09T10:50:29.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.774+0000 7fe970c5a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c0af0f0 con 0x7fe96c15ba70 2024-05-09T10:50:29.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.774+0000 7fe970c5a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c0cfb00 con 0x7fe96c15be50 2024-05-09T10:50:29.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.774+0000 7fe96a59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe96c0b43d0 0x7fe96c156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:29.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.774+0000 7fe96a59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe96c0b43d0 0x7fe96c156c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42390/0 (socket says 172.21.15.98:42390) 2024-05-09T10:50:29.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.774+0000 7fe96a59c640 1 -- 172.21.15.98:0/2724940008 learned_addr learned my addr 172.21.15.98:0/2724940008 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:29.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.775+0000 7fe96a59c640 1 -- 172.21.15.98:0/2724940008 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe96c15be50 msgr2=0x7fe96c152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:29.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.775+0000 7fe96a59c640 1 --2- 172.21.15.98:0/2724940008 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe96c15be50 0x7fe96c152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.775+0000 7fe96a59c640 1 -- 172.21.15.98:0/2724940008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe96c15ba70 msgr2=0x7fe96c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:29.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.775+0000 7fe96a59c640 1 --2- 172.21.15.98:0/2724940008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe96c15ba70 0x7fe96c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.775+0000 7fe96a59c640 1 -- 172.21.15.98:0/2724940008 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96c1f43c0 con 0x7fe96c0b43d0 2024-05-09T10:50:29.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.775+0000 7fe96a59c640 1 --2- 172.21.15.98:0/2724940008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe96c0b43d0 0x7fe96c156c40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00c450 tx=0x7fe95c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:29.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.775+0000 7fe962ffd640 1 -- 172.21.15.98:0/2724940008 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c017070 con 0x7fe96c0b43d0 2024-05-09T10:50:29.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.775+0000 7fe962ffd640 1 -- 172.21.15.98:0/2724940008 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95c0052d0 con 0x7fe96c0b43d0 2024-05-09T10:50:29.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.775+0000 7fe970c5a640 1 -- 172.21.15.98:0/2724940008 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe96c1f46a0 con 0x7fe96c0b43d0 2024-05-09T10:50:29.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.775+0000 7fe962ffd640 1 -- 172.21.15.98:0/2724940008 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c003830 con 0x7fe96c0b43d0 2024-05-09T10:50:29.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.775+0000 7fe970c5a640 1 -- 172.21.15.98:0/2724940008 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96c1f3c50 con 0x7fe96c0b43d0 2024-05-09T10:50:29.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.776+0000 7fe970c5a640 1 -- 172.21.15.98:0/2724940008 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe96c069cb0 con 0x7fe96c0b43d0 2024-05-09T10:50:29.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.776+0000 7fe962ffd640 1 -- 172.21.15.98:0/2724940008 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe95c01a070 con 0x7fe96c0b43d0 2024-05-09T10:50:29.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.777+0000 7fe962ffd640 1 --2- 172.21.15.98:0/2724940008 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe94c06b4d0 0x7fe94c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:29.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.777+0000 7fe962ffd640 1 -- 172.21.15.98:0/2724940008 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe95c089590 con 0x7fe96c0b43d0 2024-05-09T10:50:29.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.777+0000 7fe963fff640 1 --2- 172.21.15.98:0/2724940008 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe94c06b4d0 0x7fe94c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:29.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.777+0000 7fe963fff640 1 --2- 172.21.15.98:0/2724940008 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe94c06b4d0 0x7fe94c06d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe954034000 tx=0x7fe95402c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:29.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.781+0000 7fe962ffd640 1 -- 172.21.15.98:0/2724940008 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe95c058410 con 0x7fe96c0b43d0 2024-05-09T10:50:29.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.951+0000 7fe970c5a640 1 -- 172.21.15.98:0/2724940008 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe96c06c5a0 con 0x7fe96c0b43d0 2024-05-09T10:50:29.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.952+0000 7fe962ffd640 1 -- 172.21.15.98:0/2724940008 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7fe95c057d10 con 0x7fe96c0b43d0 2024-05-09T10:50:29.946 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:29.946 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":20,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:21.176419+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:50:17.323444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T10:50:18.482315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:29.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.954+0000 7fe970c5a640 1 -- 172.21.15.98:0/2724940008 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe94c06b4d0 msgr2=0x7fe94c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:29.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.954+0000 7fe970c5a640 1 --2- 172.21.15.98:0/2724940008 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe94c06b4d0 0x7fe94c06d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe954034000 tx=0x7fe95402c000 comp rx=0 tx=0).stop 2024-05-09T10:50:29.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.954+0000 7fe970c5a640 1 -- 172.21.15.98:0/2724940008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe96c0b43d0 msgr2=0x7fe96c156c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:29.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.954+0000 7fe970c5a640 1 --2- 172.21.15.98:0/2724940008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe96c0b43d0 0x7fe96c156c40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00c450 tx=0x7fe95c00c910 comp rx=0 tx=0).stop 2024-05-09T10:50:29.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.954+0000 7fe970c5a640 1 -- 172.21.15.98:0/2724940008 shutdown_connections 2024-05-09T10:50:29.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.954+0000 7fe970c5a640 1 --2- 172.21.15.98:0/2724940008 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe94c06b4d0 0x7fe94c06d990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.954+0000 7fe970c5a640 1 --2- 172.21.15.98:0/2724940008 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe96c15be50 0x7fe96c152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.954+0000 7fe970c5a640 1 --2- 172.21.15.98:0/2724940008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe96c15ba70 0x7fe96c0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.954+0000 7fe970c5a640 1 --2- 172.21.15.98:0/2724940008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe96c0b43d0 0x7fe96c156c40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:29.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.954+0000 7fe970c5a640 1 -- 172.21.15.98:0/2724940008 >> 172.21.15.98:0/2724940008 conn(0x7fe96c06abe0 msgr2=0x7fe96c15c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:29.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.954+0000 7fe970c5a640 1 -- 172.21.15.98:0/2724940008 shutdown_connections 2024-05-09T10:50:29.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:29.954+0000 7fe970c5a640 1 -- 172.21.15.98:0/2724940008 wait complete. 2024-05-09T10:50:29.958 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T10:50:07.396423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T10:50:17.323444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T10:50:18.482315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T10:50:29.959 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T10:50:30.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.068+0000 7f420b615640 1 Processor -- start 2024-05-09T10:50:30.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.068+0000 7f420b615640 1 -- start start 2024-05-09T10:50:30.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.069+0000 7f420b615640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f42040720d0 0x7f42040af720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.069+0000 7f420b615640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f420406abe0 0x7f420406afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.069+0000 7f420b615640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4204072690 con 0x7f420415db90 2024-05-09T10:50:30.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.069+0000 7f420b615640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42040afc60 con 0x7f42040720d0 2024-05-09T10:50:30.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.069+0000 7f420b615640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42040b5d70 con 0x7f420406abe0 2024-05-09T10:50:30.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.069+0000 7f4208bb0640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f42040720d0 0x7f42040af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.069+0000 7f4208bb0640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f42040720d0 0x7f42040af720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42394/0 (socket says 172.21.15.98:42394) 2024-05-09T10:50:30.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.069+0000 7f4208bb0640 1 -- 172.21.15.98:0/2212229645 learned_addr learned my addr 172.21.15.98:0/2212229645 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:30.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.069+0000 7f4209bb2640 1 --2- 172.21.15.98:0/2212229645 >> v2:172.21.15.105:3300/0 conn(0x7f420406abe0 0x7f420406afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.069+0000 7f4208bb0640 1 -- 172.21.15.98:0/2212229645 >> v2:172.21.15.105:3300/0 conn(0x7f420406abe0 msgr2=0x7f420406afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:30.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.069+0000 7f4208bb0640 1 --2- 172.21.15.98:0/2212229645 >> v2:172.21.15.105:3300/0 conn(0x7f420406abe0 0x7f420406afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.069+0000 7f4208bb0640 1 -- 172.21.15.98:0/2212229645 >> v1:172.21.15.105:6789/0 conn(0x7f420415db90 legacy=0x7f420406d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:30.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.069+0000 7f4208bb0640 1 -- 172.21.15.98:0/2212229645 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420406b520 con 0x7f42040720d0 2024-05-09T10:50:30.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.070+0000 7f4208bb0640 1 --2- 172.21.15.98:0/2212229645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42040720d0 0x7f42040af720 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f420406bbe0 tx=0x7f41f402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e629252f99834a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:30.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.070+0000 7f41fbfff640 1 -- 172.21.15.98:0/2212229645 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f4037070 con 0x7f42040720d0 2024-05-09T10:50:30.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.070+0000 7f41fbfff640 1 -- 172.21.15.98:0/2212229645 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f402ae20 con 0x7f42040720d0 2024-05-09T10:50:30.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.070+0000 7f41fbfff640 1 -- 172.21.15.98:0/2212229645 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f4032660 con 0x7f42040720d0 2024-05-09T10:50:30.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.070+0000 7f420b615640 1 -- 172.21.15.98:0/2212229645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42040720d0 msgr2=0x7f42040af720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:30.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.070+0000 7f420b615640 1 --2- 172.21.15.98:0/2212229645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42040720d0 0x7f42040af720 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f420406bbe0 tx=0x7f41f402a110 comp rx=0 tx=0).stop 2024-05-09T10:50:30.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.071+0000 7f420b615640 1 -- 172.21.15.98:0/2212229645 shutdown_connections 2024-05-09T10:50:30.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.071+0000 7f420b615640 1 --2- 172.21.15.98:0/2212229645 >> v2:172.21.15.105:3300/0 conn(0x7f420406abe0 0x7f420406afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.071+0000 7f420b615640 1 --2- 172.21.15.98:0/2212229645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42040720d0 0x7f42040af720 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.071+0000 7f420b615640 1 -- 172.21.15.98:0/2212229645 >> 172.21.15.98:0/2212229645 conn(0x7f4204066fd0 msgr2=0x7f420406bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:30.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.071+0000 7f420b615640 1 -- 172.21.15.98:0/2212229645 shutdown_connections 2024-05-09T10:50:30.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.071+0000 7f420b615640 1 -- 172.21.15.98:0/2212229645 wait complete. 2024-05-09T10:50:30.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.072+0000 7f420b615640 1 Processor -- start 2024-05-09T10:50:30.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.072+0000 7f420b615640 1 -- start start 2024-05-09T10:50:30.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.073+0000 7f420b615640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f420406abe0 0x7f4204156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.073+0000 7f420b615640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42040720d0 0x7f4204148420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.073+0000 7f420b615640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f420415db90 0x7f42041617b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.073+0000 7f420b615640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42040d0670 con 0x7f420415db90 2024-05-09T10:50:30.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.073+0000 7f420b615640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42040b5a90 con 0x7f42040720d0 2024-05-09T10:50:30.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.073+0000 7f420b615640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420415df70 con 0x7f420406abe0 2024-05-09T10:50:30.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.073+0000 7f4209bb2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f420415db90 0x7f42041617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.073+0000 7f4209bb2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f420415db90 0x7f42041617b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42408/0 (socket says 172.21.15.98:42408) 2024-05-09T10:50:30.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.073+0000 7f4209bb2640 1 -- 172.21.15.98:0/2364711212 learned_addr learned my addr 172.21.15.98:0/2364711212 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:30.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.073+0000 7f42093b1640 1 --2- 172.21.15.98:0/2364711212 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f420406abe0 0x7f4204156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.073+0000 7f4208bb0640 1 --2- 172.21.15.98:0/2364711212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42040720d0 0x7f4204148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.073+0000 7f4209bb2640 1 -- 172.21.15.98:0/2364711212 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f420406abe0 msgr2=0x7f4204156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:30.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.073+0000 7f4209bb2640 1 --2- 172.21.15.98:0/2364711212 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f420406abe0 0x7f4204156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.073+0000 7f4209bb2640 1 -- 172.21.15.98:0/2364711212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42040720d0 msgr2=0x7f4204148420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:30.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.073+0000 7f4209bb2640 1 --2- 172.21.15.98:0/2364711212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42040720d0 0x7f4204148420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.073+0000 7f4209bb2640 1 -- 172.21.15.98:0/2364711212 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4204155a70 con 0x7f420415db90 2024-05-09T10:50:30.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.073+0000 7f4209bb2640 1 --2- 172.21.15.98:0/2364711212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f420415db90 0x7f42041617b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f41fc007e70 tx=0x7f41fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:30.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.073+0000 7f41f9ffb640 1 -- 172.21.15.98:0/2364711212 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc014620 con 0x7f420415db90 2024-05-09T10:50:30.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.073+0000 7f41f9ffb640 1 -- 172.21.15.98:0/2364711212 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41fc019070 con 0x7f420415db90 2024-05-09T10:50:30.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.073+0000 7f420b615640 1 -- 172.21.15.98:0/2364711212 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4204156170 con 0x7f420415db90 2024-05-09T10:50:30.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.073+0000 7f41f9ffb640 1 -- 172.21.15.98:0/2364711212 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc014a20 con 0x7f420415db90 2024-05-09T10:50:30.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.073+0000 7f420b615640 1 -- 172.21.15.98:0/2364711212 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4204156730 con 0x7f420415db90 2024-05-09T10:50:30.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.075+0000 7f420b615640 1 -- 172.21.15.98:0/2364711212 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4204072dd0 con 0x7f420415db90 2024-05-09T10:50:30.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.075+0000 7f41f9ffb640 1 -- 172.21.15.98:0/2364711212 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f41fc014bc0 con 0x7f420415db90 2024-05-09T10:50:30.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.075+0000 7f41f9ffb640 1 --2- 172.21.15.98:0/2364711212 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f41e806b4d0 0x7f41e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.075+0000 7f41f9ffb640 1 -- 172.21.15.98:0/2364711212 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41fc08a930 con 0x7f420415db90 2024-05-09T10:50:30.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.075+0000 7f42093b1640 1 --2- 172.21.15.98:0/2364711212 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f41e806b4d0 0x7f41e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.076+0000 7f42093b1640 1 --2- 172.21.15.98:0/2364711212 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f41e806b4d0 0x7f41e806d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4204071bf0 tx=0x7f4200009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:30.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.079+0000 7f41f9ffb640 1 -- 172.21.15.98:0/2364711212 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f41fc0591f0 con 0x7f420415db90 2024-05-09T10:50:30.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.250+0000 7f420b615640 1 -- 172.21.15.98:0/2364711212 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f420404f000 con 0x7f420415db90 2024-05-09T10:50:30.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.251+0000 7f41f9ffb640 1 -- 172.21.15.98:0/2364711212 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f41fc05f070 con 0x7f420415db90 2024-05-09T10:50:30.244 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-05-09T10:50:30.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.253+0000 7f420b615640 1 -- 172.21.15.98:0/2364711212 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f41e806b4d0 msgr2=0x7f41e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:30.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.253+0000 7f420b615640 1 --2- 172.21.15.98:0/2364711212 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f41e806b4d0 0x7f41e806d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4204071bf0 tx=0x7f4200009060 comp rx=0 tx=0).stop 2024-05-09T10:50:30.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.253+0000 7f420b615640 1 -- 172.21.15.98:0/2364711212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f420415db90 msgr2=0x7f42041617b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:30.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.253+0000 7f420b615640 1 --2- 172.21.15.98:0/2364711212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f420415db90 0x7f42041617b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f41fc007e70 tx=0x7f41fc00e4b0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.253+0000 7f420b615640 1 -- 172.21.15.98:0/2364711212 shutdown_connections 2024-05-09T10:50:30.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.253+0000 7f420b615640 1 --2- 172.21.15.98:0/2364711212 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f41e806b4d0 0x7f41e806d990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.253+0000 7f420b615640 1 --2- 172.21.15.98:0/2364711212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f420415db90 0x7f42041617b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.253+0000 7f420b615640 1 --2- 172.21.15.98:0/2364711212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42040720d0 0x7f4204148420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.253+0000 7f420b615640 1 --2- 172.21.15.98:0/2364711212 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f420406abe0 0x7f4204156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.253+0000 7f420b615640 1 -- 172.21.15.98:0/2364711212 >> 172.21.15.98:0/2364711212 conn(0x7f4204066fd0 msgr2=0x7f420415c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:30.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.253+0000 7f420b615640 1 -- 172.21.15.98:0/2364711212 shutdown_connections 2024-05-09T10:50:30.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.253+0000 7f420b615640 1 -- 172.21.15.98:0/2364711212 wait complete. 2024-05-09T10:50:30.257 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T10:50:30.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.365+0000 7f2da2fdb640 1 Processor -- start 2024-05-09T10:50:30.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.365+0000 7f2da2fdb640 1 -- start start 2024-05-09T10:50:30.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.366+0000 7f2da2fdb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2d9c0cc610 0x7f2d9c0cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.366+0000 7f2da2fdb640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2d9c06ac50 0x7f2d9c06b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.366+0000 7f2da2fdb640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d9c15c460 con 0x7f2d9c0d07e0 2024-05-09T10:50:30.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.366+0000 7f2da2fdb640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d9c159d40 con 0x7f2d9c06ac50 2024-05-09T10:50:30.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.366+0000 7f2da2fdb640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d9c15d030 con 0x7f2d9c0cc610 2024-05-09T10:50:30.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.366+0000 7f2d9bfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2d9c06ac50 0x7f2d9c06b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.366+0000 7f2da1578640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2d9c0d07e0 0x7f2d9c0d0c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56100/0 (socket says 172.21.15.98:56100) 2024-05-09T10:50:30.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.366+0000 7f2da1578640 1 -- 172.21.15.98:0/1006368963 learned_addr learned my addr 172.21.15.98:0/1006368963 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:30.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.367+0000 7f2da0d77640 1 --2- 172.21.15.98:0/1006368963 >> v2:172.21.15.105:3300/0 conn(0x7f2d9c0cc610 0x7f2d9c0cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.367+0000 7f2d9bfff640 1 -- 172.21.15.98:0/1006368963 >> v2:172.21.15.105:3300/0 conn(0x7f2d9c0cc610 msgr2=0x7f2d9c0cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:30.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.367+0000 7f2d9bfff640 1 --2- 172.21.15.98:0/1006368963 >> v2:172.21.15.105:3300/0 conn(0x7f2d9c0cc610 0x7f2d9c0cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.367+0000 7f2d9bfff640 1 -- 172.21.15.98:0/1006368963 >> v1:172.21.15.98:6789/0 conn(0x7f2d9c0d07e0 legacy=0x7f2d9c0d0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:30.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.367+0000 7f2d9bfff640 1 -- 172.21.15.98:0/1006368963 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9c0ccf30 con 0x7f2d9c06ac50 2024-05-09T10:50:30.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.367+0000 7f2d9b7fe640 1 -- 172.21.15.98:0/1006368963 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2444382200 0 0) 0x7f2d9c15c460 con 0x7f2d9c0d07e0 2024-05-09T10:50:30.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.367+0000 7f2d9bfff640 1 --2- 172.21.15.98:0/1006368963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d9c06ac50 0x7f2d9c06b100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0685a0 tx=0x7f2d9002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9bbb3c5bd441b89 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:30.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.368+0000 7f2d9b7fe640 1 -- 172.21.15.98:0/1006368963 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90037070 con 0x7f2d9c06ac50 2024-05-09T10:50:30.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.368+0000 7f2d9b7fe640 1 -- 172.21.15.98:0/1006368963 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d90032440 con 0x7f2d9c06ac50 2024-05-09T10:50:30.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.368+0000 7f2d9b7fe640 1 -- 172.21.15.98:0/1006368963 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90032810 con 0x7f2d9c06ac50 2024-05-09T10:50:30.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.368+0000 7f2da2fdb640 1 -- 172.21.15.98:0/1006368963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d9c06ac50 msgr2=0x7f2d9c06b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:30.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.368+0000 7f2da2fdb640 1 --2- 172.21.15.98:0/1006368963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d9c06ac50 0x7f2d9c06b100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0685a0 tx=0x7f2d9002a1a0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.368+0000 7f2da2fdb640 1 -- 172.21.15.98:0/1006368963 shutdown_connections 2024-05-09T10:50:30.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.368+0000 7f2da2fdb640 1 --2- 172.21.15.98:0/1006368963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d9c06ac50 0x7f2d9c06b100 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.368+0000 7f2da2fdb640 1 --2- 172.21.15.98:0/1006368963 >> v2:172.21.15.105:3300/0 conn(0x7f2d9c0cc610 0x7f2d9c0cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.368+0000 7f2da2fdb640 1 -- 172.21.15.98:0/1006368963 >> 172.21.15.98:0/1006368963 conn(0x7f2d9c0b25a0 msgr2=0x7f2d9c0b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:30.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.368+0000 7f2da2fdb640 1 -- 172.21.15.98:0/1006368963 shutdown_connections 2024-05-09T10:50:30.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.368+0000 7f2da2fdb640 1 -- 172.21.15.98:0/1006368963 wait complete. 2024-05-09T10:50:30.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.369+0000 7f2da2fdb640 1 Processor -- start 2024-05-09T10:50:30.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.369+0000 7f2da2fdb640 1 -- start start 2024-05-09T10:50:30.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.370+0000 7f2da2fdb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d9c06ac50 0x7f2d9c14c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.370+0000 7f2da2fdb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d9c0cc610 0x7f2d9c14b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.370+0000 7f2da2fdb640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d9c0d07e0 0x7f2d9c149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.370+0000 7f2da2fdb640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c15ccc0 con 0x7f2d9c06ac50 2024-05-09T10:50:30.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.370+0000 7f2da2fdb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c15c9e0 con 0x7f2d9c0cc610 2024-05-09T10:50:30.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.370+0000 7f2da2fdb640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c06b640 con 0x7f2d9c0d07e0 2024-05-09T10:50:30.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.370+0000 7f2da0d77640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d9c06ac50 0x7f2d9c14c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.370+0000 7f2da0d77640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d9c06ac50 0x7f2d9c14c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42422/0 (socket says 172.21.15.98:42422) 2024-05-09T10:50:30.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.370+0000 7f2da0d77640 1 -- 172.21.15.98:0/2674603313 learned_addr learned my addr 172.21.15.98:0/2674603313 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:30.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.371+0000 7f2d9bfff640 1 --2- 172.21.15.98:0/2674603313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d9c0cc610 0x7f2d9c14b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.371+0000 7f2da1578640 1 --2- 172.21.15.98:0/2674603313 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d9c0d07e0 0x7f2d9c149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.371+0000 7f2da0d77640 1 -- 172.21.15.98:0/2674603313 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d9c0d07e0 msgr2=0x7f2d9c149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:30.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.371+0000 7f2da0d77640 1 --2- 172.21.15.98:0/2674603313 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d9c0d07e0 0x7f2d9c149920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.371+0000 7f2da0d77640 1 -- 172.21.15.98:0/2674603313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d9c0cc610 msgr2=0x7f2d9c14b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:30.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.371+0000 7f2da0d77640 1 --2- 172.21.15.98:0/2674603313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d9c0cc610 0x7f2d9c14b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.371+0000 7f2da0d77640 1 -- 172.21.15.98:0/2674603313 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9c14b6f0 con 0x7f2d9c06ac50 2024-05-09T10:50:30.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.371+0000 7f2d9bfff640 1 --2- 172.21.15.98:0/2674603313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d9c0cc610 0x7f2d9c14b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:30.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.371+0000 7f2da0d77640 1 --2- 172.21.15.98:0/2674603313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d9c06ac50 0x7f2d9c14c580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c007f20 tx=0x7f2d8c00e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:30.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.371+0000 7f2d997fa640 1 -- 172.21.15.98:0/2674603313 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c014620 con 0x7f2d9c06ac50 2024-05-09T10:50:30.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.371+0000 7f2d997fa640 1 -- 172.21.15.98:0/2674603313 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8c019070 con 0x7f2d9c06ac50 2024-05-09T10:50:30.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.371+0000 7f2d997fa640 1 -- 172.21.15.98:0/2674603313 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c014a00 con 0x7f2d9c06ac50 2024-05-09T10:50:30.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.371+0000 7f2da2fdb640 1 -- 172.21.15.98:0/2674603313 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d9c149e60 con 0x7f2d9c06ac50 2024-05-09T10:50:30.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.371+0000 7f2da2fdb640 1 -- 172.21.15.98:0/2674603313 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d9c14a3f0 con 0x7f2d9c06ac50 2024-05-09T10:50:30.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.372+0000 7f2da2fdb640 1 -- 172.21.15.98:0/2674603313 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d9c04f000 con 0x7f2d9c06ac50 2024-05-09T10:50:30.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.372+0000 7f2d997fa640 1 -- 172.21.15.98:0/2674603313 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d8c014ba0 con 0x7f2d9c06ac50 2024-05-09T10:50:30.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.373+0000 7f2d997fa640 1 --2- 172.21.15.98:0/2674603313 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2d8806b4d0 0x7f2d8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.373+0000 7f2d9bfff640 1 --2- 172.21.15.98:0/2674603313 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2d8806b4d0 0x7f2d8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.373+0000 7f2d997fa640 1 -- 172.21.15.98:0/2674603313 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d8c08ab20 con 0x7f2d9c06ac50 2024-05-09T10:50:30.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.373+0000 7f2d9bfff640 1 --2- 172.21.15.98:0/2674603313 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2d8806b4d0 0x7f2d8806d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2d90034000 tx=0x7f2d9002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:30.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.377+0000 7f2d997fa640 1 -- 172.21.15.98:0/2674603313 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d8c0599a0 con 0x7f2d9c06ac50 2024-05-09T10:50:30.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.547+0000 7f2da2fdb640 1 -- 172.21.15.98:0/2674603313 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2d9c074480 con 0x7f2d9c06ac50 2024-05-09T10:50:30.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.547+0000 7f2d997fa640 1 -- 172.21.15.98:0/2674603313 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2d8c0592a0 con 0x7f2d9c06ac50 2024-05-09T10:50:30.541 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2024-05-09T10:50:30.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.549+0000 7f2da2fdb640 1 -- 172.21.15.98:0/2674603313 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2d8806b4d0 msgr2=0x7f2d8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:30.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.549+0000 7f2da2fdb640 1 --2- 172.21.15.98:0/2674603313 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2d8806b4d0 0x7f2d8806d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2d90034000 tx=0x7f2d9002e000 comp rx=0 tx=0).stop 2024-05-09T10:50:30.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.549+0000 7f2da2fdb640 1 -- 172.21.15.98:0/2674603313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d9c06ac50 msgr2=0x7f2d9c14c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:30.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.549+0000 7f2da2fdb640 1 --2- 172.21.15.98:0/2674603313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d9c06ac50 0x7f2d9c14c580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c007f20 tx=0x7f2d8c00e570 comp rx=0 tx=0).stop 2024-05-09T10:50:30.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.549+0000 7f2da2fdb640 1 -- 172.21.15.98:0/2674603313 shutdown_connections 2024-05-09T10:50:30.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.549+0000 7f2da2fdb640 1 --2- 172.21.15.98:0/2674603313 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2d8806b4d0 0x7f2d8806d990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.549+0000 7f2da2fdb640 1 --2- 172.21.15.98:0/2674603313 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d9c0d07e0 0x7f2d9c149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.549+0000 7f2da2fdb640 1 --2- 172.21.15.98:0/2674603313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d9c0cc610 0x7f2d9c14b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.549+0000 7f2da2fdb640 1 --2- 172.21.15.98:0/2674603313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d9c06ac50 0x7f2d9c14c580 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.549+0000 7f2da2fdb640 1 -- 172.21.15.98:0/2674603313 >> 172.21.15.98:0/2674603313 conn(0x7f2d9c0b25a0 msgr2=0x7f2d9c0b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:30.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.549+0000 7f2da2fdb640 1 -- 172.21.15.98:0/2674603313 shutdown_connections 2024-05-09T10:50:30.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.549+0000 7f2da2fdb640 1 -- 172.21.15.98:0/2674603313 wait complete. 2024-05-09T10:50:30.554 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T10:50:30.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.661+0000 7f2cb0cab640 1 Processor -- start 2024-05-09T10:50:30.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.662+0000 7f2cb0cab640 1 -- start start 2024-05-09T10:50:30.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.662+0000 7f2cb0cab640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2cac1584e0 0x7f2cac0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.663+0000 7f2cb0cab640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cac0af6e0 0x7f2cac0afb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.663+0000 7f2cb0cab640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2cac0713f0 0x7f2cac071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.663+0000 7f2cb0cab640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cac0b01a0 con 0x7f2cac1584e0 2024-05-09T10:50:30.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.663+0000 7f2cb0cab640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cac0b42a0 con 0x7f2cac0713f0 2024-05-09T10:50:30.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.663+0000 7f2cb0cab640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac0b4580 con 0x7f2cac0af6e0 2024-05-09T10:50:30.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.663+0000 7f2caa59c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2cac1584e0 0x7f2cac0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.663+0000 7f2caa59c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2cac1584e0 0x7f2cac0b4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42436/0 (socket says 172.21.15.98:42436) 2024-05-09T10:50:30.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.663+0000 7f2caa59c640 1 -- 172.21.15.98:0/1690893744 learned_addr learned my addr 172.21.15.98:0/1690893744 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:30.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.663+0000 7f2ca9d9b640 1 --2- 172.21.15.98:0/1690893744 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cac0af6e0 0x7f2cac0afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.663+0000 7f2caa59c640 1 -- 172.21.15.98:0/1690893744 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cac0af6e0 msgr2=0x7f2cac0afb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:30.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.663+0000 7f2caa59c640 1 --2- 172.21.15.98:0/1690893744 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cac0af6e0 0x7f2cac0afb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.663+0000 7f2caa59c640 1 -- 172.21.15.98:0/1690893744 >> v2:172.21.15.105:3300/0 conn(0x7f2cac0713f0 msgr2=0x7f2cac071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:30.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.663+0000 7f2caad9d640 1 --2- 172.21.15.98:0/1690893744 >> v2:172.21.15.105:3300/0 conn(0x7f2cac0713f0 0x7f2cac071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.663+0000 7f2caa59c640 1 --2- 172.21.15.98:0/1690893744 >> v2:172.21.15.105:3300/0 conn(0x7f2cac0713f0 0x7f2cac071880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.663+0000 7f2caa59c640 1 -- 172.21.15.98:0/1690893744 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cac0b4860 con 0x7f2cac1584e0 2024-05-09T10:50:30.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.663+0000 7f2ca9d9b640 1 --2- 172.21.15.98:0/1690893744 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cac0af6e0 0x7f2cac0afb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:30.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.663+0000 7f2caad9d640 1 --2- 172.21.15.98:0/1690893744 >> v2:172.21.15.105:3300/0 conn(0x7f2cac0713f0 0x7f2cac071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:30.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.664+0000 7f2caa59c640 1 --2- 172.21.15.98:0/1690893744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cac1584e0 0x7f2cac0b4d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2cac0b5d30 tx=0x7f2ca402c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339c897a0599e381 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:30.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.664+0000 7f2ca959a640 1 -- 172.21.15.98:0/1690893744 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4039070 con 0x7f2cac1584e0 2024-05-09T10:50:30.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.664+0000 7f2ca959a640 1 -- 172.21.15.98:0/1690893744 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca4009600 con 0x7f2cac1584e0 2024-05-09T10:50:30.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.664+0000 7f2ca959a640 1 -- 172.21.15.98:0/1690893744 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca40099f0 con 0x7f2cac1584e0 2024-05-09T10:50:30.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.664+0000 7f2cb0cab640 1 -- 172.21.15.98:0/1690893744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cac1584e0 msgr2=0x7f2cac0b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:30.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.664+0000 7f2cb0cab640 1 --2- 172.21.15.98:0/1690893744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cac1584e0 0x7f2cac0b4d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2cac0b5d30 tx=0x7f2ca402c810 comp rx=0 tx=0).stop 2024-05-09T10:50:30.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.664+0000 7f2cb0cab640 1 -- 172.21.15.98:0/1690893744 shutdown_connections 2024-05-09T10:50:30.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.664+0000 7f2cb0cab640 1 --2- 172.21.15.98:0/1690893744 >> v2:172.21.15.105:3300/0 conn(0x7f2cac0713f0 0x7f2cac071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.664+0000 7f2cb0cab640 1 --2- 172.21.15.98:0/1690893744 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cac0af6e0 0x7f2cac0afb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.664+0000 7f2cb0cab640 1 --2- 172.21.15.98:0/1690893744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cac1584e0 0x7f2cac0b4d80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.665+0000 7f2cb0cab640 1 -- 172.21.15.98:0/1690893744 >> 172.21.15.98:0/1690893744 conn(0x7f2cac0d0860 msgr2=0x7f2cac0d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:30.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.665+0000 7f2cb0cab640 1 -- 172.21.15.98:0/1690893744 shutdown_connections 2024-05-09T10:50:30.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.665+0000 7f2cb0cab640 1 -- 172.21.15.98:0/1690893744 wait complete. 2024-05-09T10:50:30.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.666+0000 7f2cb0cab640 1 Processor -- start 2024-05-09T10:50:30.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.666+0000 7f2cb0cab640 1 -- start start 2024-05-09T10:50:30.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.667+0000 7f2cb0cab640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cac0713f0 0x7f2cac1443c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.667+0000 7f2cb0cab640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cac0af6e0 0x7f2cac1e0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.667+0000 7f2cb0cab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cac1584e0 0x7f2cac144cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.667+0000 7f2cb0cab640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac06d390 con 0x7f2cac0af6e0 2024-05-09T10:50:30.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.667+0000 7f2cb0cab640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac15bde0 con 0x7f2cac1584e0 2024-05-09T10:50:30.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.667+0000 7f2cb0cab640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac06d510 con 0x7f2cac0713f0 2024-05-09T10:50:30.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.667+0000 7f2ca9d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cac0af6e0 0x7f2cac1e0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.667+0000 7f2ca9d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cac0af6e0 0x7f2cac1e0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42448/0 (socket says 172.21.15.98:42448) 2024-05-09T10:50:30.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.667+0000 7f2ca9d9b640 1 -- 172.21.15.98:0/3600541553 learned_addr learned my addr 172.21.15.98:0/3600541553 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:30.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.667+0000 7f2caa59c640 1 --2- 172.21.15.98:0/3600541553 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cac0713f0 0x7f2cac1443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.667+0000 7f2caad9d640 1 --2- 172.21.15.98:0/3600541553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cac1584e0 0x7f2cac144cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.667+0000 7f2ca9d9b640 1 -- 172.21.15.98:0/3600541553 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cac0713f0 msgr2=0x7f2cac1443c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:30.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.667+0000 7f2ca9d9b640 1 --2- 172.21.15.98:0/3600541553 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cac0713f0 0x7f2cac1443c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.667+0000 7f2ca9d9b640 1 -- 172.21.15.98:0/3600541553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cac1584e0 msgr2=0x7f2cac144cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:30.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.667+0000 7f2ca9d9b640 1 --2- 172.21.15.98:0/3600541553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cac1584e0 0x7f2cac144cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.667+0000 7f2ca9d9b640 1 -- 172.21.15.98:0/3600541553 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cac1e1680 con 0x7f2cac0af6e0 2024-05-09T10:50:30.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.667+0000 7f2caa59c640 1 --2- 172.21.15.98:0/3600541553 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cac0713f0 0x7f2cac1443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:30.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.667+0000 7f2ca9d9b640 1 --2- 172.21.15.98:0/3600541553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cac0af6e0 0x7f2cac1e0c70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2c9400c820 tx=0x7f2c9400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:30.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.667+0000 7f2ca2ffd640 1 -- 172.21.15.98:0/3600541553 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c94015070 con 0x7f2cac0af6e0 2024-05-09T10:50:30.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.667+0000 7f2ca2ffd640 1 -- 172.21.15.98:0/3600541553 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c94007a30 con 0x7f2cac0af6e0 2024-05-09T10:50:30.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.667+0000 7f2ca2ffd640 1 -- 172.21.15.98:0/3600541553 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c94007e00 con 0x7f2cac0af6e0 2024-05-09T10:50:30.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.667+0000 7f2cb0cab640 1 -- 172.21.15.98:0/3600541553 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cac1e1960 con 0x7f2cac0af6e0 2024-05-09T10:50:30.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.667+0000 7f2cb0cab640 1 -- 172.21.15.98:0/3600541553 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cac1e1f20 con 0x7f2cac0af6e0 2024-05-09T10:50:30.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.668+0000 7f2cb0cab640 1 -- 172.21.15.98:0/3600541553 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cac072f30 con 0x7f2cac0af6e0 2024-05-09T10:50:30.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.669+0000 7f2ca2ffd640 1 -- 172.21.15.98:0/3600541553 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2c94018070 con 0x7f2cac0af6e0 2024-05-09T10:50:30.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.669+0000 7f2ca2ffd640 1 --2- 172.21.15.98:0/3600541553 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2c8c06b4d0 0x7f2c8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.669+0000 7f2ca2ffd640 1 -- 172.21.15.98:0/3600541553 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c9408a130 con 0x7f2cac0af6e0 2024-05-09T10:50:30.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.669+0000 7f2caa59c640 1 --2- 172.21.15.98:0/3600541553 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2c8c06b4d0 0x7f2c8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.670+0000 7f2caa59c640 1 --2- 172.21.15.98:0/3600541553 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2c8c06b4d0 0x7f2c8c06d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4036000 tx=0x7f2ca400b000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:30.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.673+0000 7f2ca2ffd640 1 -- 172.21.15.98:0/3600541553 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2c94058960 con 0x7f2cac0af6e0 2024-05-09T10:50:30.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.843+0000 7f2cb0cab640 1 -- 172.21.15.98:0/3600541553 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2cac0d1700 con 0x7f2cac0af6e0 2024-05-09T10:50:30.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.844+0000 7f2ca2ffd640 1 -- 172.21.15.98:0/3600541553 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2c9405ca00 con 0x7f2cac0af6e0 2024-05-09T10:50:30.837 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2024-05-09T10:50:30.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.846+0000 7f2cb0cab640 1 -- 172.21.15.98:0/3600541553 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2c8c06b4d0 msgr2=0x7f2c8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:30.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.846+0000 7f2cb0cab640 1 --2- 172.21.15.98:0/3600541553 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2c8c06b4d0 0x7f2c8c06d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4036000 tx=0x7f2ca400b000 comp rx=0 tx=0).stop 2024-05-09T10:50:30.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.846+0000 7f2cb0cab640 1 -- 172.21.15.98:0/3600541553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cac0af6e0 msgr2=0x7f2cac1e0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:30.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.846+0000 7f2cb0cab640 1 --2- 172.21.15.98:0/3600541553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cac0af6e0 0x7f2cac1e0c70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2c9400c820 tx=0x7f2c9400cc20 comp rx=0 tx=0).stop 2024-05-09T10:50:30.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.846+0000 7f2cb0cab640 1 -- 172.21.15.98:0/3600541553 shutdown_connections 2024-05-09T10:50:30.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.846+0000 7f2cb0cab640 1 --2- 172.21.15.98:0/3600541553 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2c8c06b4d0 0x7f2c8c06d990 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.846+0000 7f2cb0cab640 1 --2- 172.21.15.98:0/3600541553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cac1584e0 0x7f2cac144cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.846+0000 7f2cb0cab640 1 --2- 172.21.15.98:0/3600541553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cac0af6e0 0x7f2cac1e0c70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.846+0000 7f2cb0cab640 1 --2- 172.21.15.98:0/3600541553 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2cac0713f0 0x7f2cac1443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.846+0000 7f2cb0cab640 1 -- 172.21.15.98:0/3600541553 >> 172.21.15.98:0/3600541553 conn(0x7f2cac0d0860 msgr2=0x7f2cac157480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:30.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.846+0000 7f2cb0cab640 1 -- 172.21.15.98:0/3600541553 shutdown_connections 2024-05-09T10:50:30.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.846+0000 7f2cb0cab640 1 -- 172.21.15.98:0/3600541553 wait complete. 2024-05-09T10:50:30.850 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T10:50:30.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.956+0000 7f0ee608f640 1 Processor -- start 2024-05-09T10:50:30.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.956+0000 7f0ee608f640 1 -- start start 2024-05-09T10:50:30.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.957+0000 7f0ee608f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0ee015df70 0x7f0ee00713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.957+0000 7f0ee608f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ee00cdb30 con 0x7f0ee0071910 2024-05-09T10:50:30.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.957+0000 7f0ee608f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ee0071f40 con 0x7f0ee015db90 2024-05-09T10:50:30.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.957+0000 7f0ee608f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ee0072220 con 0x7f0ee015df70 2024-05-09T10:50:30.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.957+0000 7f0edffff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f0ee0071910 0x7f0ee006abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56104/0 (socket says 172.21.15.98:56104) 2024-05-09T10:50:30.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.957+0000 7f0edffff640 1 -- 172.21.15.98:0/1692521028 learned_addr learned my addr 172.21.15.98:0/1692521028 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:30.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.957+0000 7f0ede7fc640 1 -- 172.21.15.98:0/1692521028 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769055129 0 0) 0x7f0ed8001110 con 0x7f0ee0071910 2024-05-09T10:50:30.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.957+0000 7f0edeffd640 1 --2- 172.21.15.98:0/1692521028 >> v2:172.21.15.105:3300/0 conn(0x7f0ee015df70 0x7f0ee00713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.957+0000 7f0ede7fc640 1 -- 172.21.15.98:0/1692521028 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ec8003520 con 0x7f0ee0071910 2024-05-09T10:50:30.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.957+0000 7f0ede7fc640 1 -- 172.21.15.98:0/1692521028 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2114826586 0 0) 0x7f0ec8003520 con 0x7f0ee0071910 2024-05-09T10:50:30.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.957+0000 7f0ede7fc640 1 -- 172.21.15.98:0/1692521028 >> v2:172.21.15.105:3300/0 conn(0x7f0ee015df70 msgr2=0x7f0ee00713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:30.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.957+0000 7f0ede7fc640 1 --2- 172.21.15.98:0/1692521028 >> v2:172.21.15.105:3300/0 conn(0x7f0ee015df70 0x7f0ee00713d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.957+0000 7f0ede7fc640 1 -- 172.21.15.98:0/1692521028 >> v1:172.21.15.105:6789/0 conn(0x7f0ee015db90 legacy=0x7f0ee006d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:30.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.958+0000 7f0ede7fc640 1 -- 172.21.15.98:0/1692521028 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee0072500 con 0x7f0ee0071910 2024-05-09T10:50:30.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.958+0000 7f0ede7fc640 1 -- 172.21.15.98:0/1692521028 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f0ed8001840 con 0x7f0ee0071910 2024-05-09T10:50:30.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.958+0000 7f0ede7fc640 1 -- 172.21.15.98:0/1692521028 >> v1:172.21.15.98:6789/0 conn(0x7f0ee0071910 legacy=0x7f0ee006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:30.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.958+0000 7f0ede7fc640 1 --2- 172.21.15.98:0/1692521028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ec80039b0 0x7f0ec8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.958+0000 7f0ede7fc640 1 --2- 172.21.15.98:0/1692521028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ec8004800 0x7f0ec8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.958+0000 7f0ede7fc640 1 --2- 172.21.15.98:0/1692521028 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ec80051c0 0x7f0ec800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.958+0000 7f0ede7fc640 1 -- 172.21.15.98:0/1692521028 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee0072220 con 0x7f0ec8004800 2024-05-09T10:50:30.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.958+0000 7f0ede7fc640 1 -- 172.21.15.98:0/1692521028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec800c160 con 0x7f0ec80039b0 2024-05-09T10:50:30.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.958+0000 7f0ede7fc640 1 -- 172.21.15.98:0/1692521028 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec800c360 con 0x7f0ec80051c0 2024-05-09T10:50:30.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.958+0000 7f0edeffd640 1 --2- 172.21.15.98:0/1692521028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ec8004800 0x7f0ec8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.958+0000 7f0edf7fe640 1 --2- 172.21.15.98:0/1692521028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ec80039b0 0x7f0ec8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.958+0000 7f0edffff640 1 --2- 172.21.15.98:0/1692521028 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ec80051c0 0x7f0ec800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.958+0000 7f0edeffd640 1 -- 172.21.15.98:0/1692521028 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ec80051c0 msgr2=0x7f0ec800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:30.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.958+0000 7f0edeffd640 1 --2- 172.21.15.98:0/1692521028 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ec80051c0 0x7f0ec800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.958+0000 7f0edeffd640 1 -- 172.21.15.98:0/1692521028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ec80039b0 msgr2=0x7f0ec8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:30.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.958+0000 7f0edeffd640 1 --2- 172.21.15.98:0/1692521028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ec80039b0 0x7f0ec8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.958+0000 7f0edeffd640 1 -- 172.21.15.98:0/1692521028 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec8003520 con 0x7f0ec8004800 2024-05-09T10:50:30.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.958+0000 7f0edffff640 1 -- 172.21.15.98:0/1692521028 reap_dead start 2024-05-09T10:50:30.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.959+0000 7f0edeffd640 1 --2- 172.21.15.98:0/1692521028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ec8004800 0x7f0ec8004c80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4009a90 tx=0x7f0ec8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b92d65e1c3abe9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:30.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.959+0000 7f0ede7fc640 1 -- 172.21.15.98:0/1692521028 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4036070 con 0x7f0ec8004800 2024-05-09T10:50:30.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.959+0000 7f0ede7fc640 1 -- 172.21.15.98:0/1692521028 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed4031400 con 0x7f0ec8004800 2024-05-09T10:50:30.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.959+0000 7f0ede7fc640 1 -- 172.21.15.98:0/1692521028 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4009690 con 0x7f0ec8004800 2024-05-09T10:50:30.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.959+0000 7f0ee608f640 1 -- 172.21.15.98:0/1692521028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ec8004800 msgr2=0x7f0ec8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:30.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.959+0000 7f0ee608f640 1 --2- 172.21.15.98:0/1692521028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ec8004800 0x7f0ec8004c80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4009a90 tx=0x7f0ec8005830 comp rx=0 tx=0).stop 2024-05-09T10:50:30.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.959+0000 7f0ee608f640 1 -- 172.21.15.98:0/1692521028 shutdown_connections 2024-05-09T10:50:30.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.959+0000 7f0ee608f640 1 --2- 172.21.15.98:0/1692521028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ec8004800 0x7f0ec8004c80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.959+0000 7f0ee608f640 1 -- 172.21.15.98:0/1692521028 >> 172.21.15.98:0/1692521028 conn(0x7f0ee0066f20 msgr2=0x7f0ee006bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:30.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.959+0000 7f0ee608f640 1 -- 172.21.15.98:0/1692521028 shutdown_connections 2024-05-09T10:50:30.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.959+0000 7f0ee608f640 1 -- 172.21.15.98:0/1692521028 wait complete. 2024-05-09T10:50:30.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.960+0000 7f0ee608f640 1 Processor -- start 2024-05-09T10:50:30.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.961+0000 7f0ee608f640 1 -- start start 2024-05-09T10:50:30.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.961+0000 7f0ee608f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ec8004800 0x7f0ee0146d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.961+0000 7f0ee608f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ee01543d0 0x7f0ee00b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.961+0000 7f0ee608f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ee0155fc0 0x7f0ee0156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.961+0000 7f0ee608f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee00713d0 con 0x7f0ec8004800 2024-05-09T10:50:30.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.961+0000 7f0ee608f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee00cefd0 con 0x7f0ee01543d0 2024-05-09T10:50:30.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.961+0000 7f0ee608f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee0071550 con 0x7f0ee0155fc0 2024-05-09T10:50:30.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.961+0000 7f0edf7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ec8004800 0x7f0ee0146d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.961+0000 7f0edf7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ec8004800 0x7f0ee0146d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42452/0 (socket says 172.21.15.98:42452) 2024-05-09T10:50:30.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.961+0000 7f0edf7fe640 1 -- 172.21.15.98:0/1050388151 learned_addr learned my addr 172.21.15.98:0/1050388151 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:30.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.962+0000 7f0edf7fe640 1 -- 172.21.15.98:0/1050388151 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ee0155fc0 msgr2=0x7f0ee0156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:30.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.962+0000 7f0edffff640 1 --2- 172.21.15.98:0/1050388151 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ee0155fc0 0x7f0ee0156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.962+0000 7f0edf7fe640 1 --2- 172.21.15.98:0/1050388151 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ee0155fc0 0x7f0ee0156420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.962+0000 7f0edf7fe640 1 -- 172.21.15.98:0/1050388151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ee01543d0 msgr2=0x7f0ee00b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:30.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.962+0000 7f0edf7fe640 1 --2- 172.21.15.98:0/1050388151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ee01543d0 0x7f0ee00b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:30.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.962+0000 7f0edf7fe640 1 -- 172.21.15.98:0/1050388151 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee0154940 con 0x7f0ec8004800 2024-05-09T10:50:30.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.962+0000 7f0edf7fe640 1 --2- 172.21.15.98:0/1050388151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ec8004800 0x7f0ee0146d00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000fb90 tx=0x7f0ed000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:30.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.962+0000 7f0ec7fff640 1 -- 172.21.15.98:0/1050388151 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed000fe90 con 0x7f0ec8004800 2024-05-09T10:50:30.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.962+0000 7f0ec7fff640 1 -- 172.21.15.98:0/1050388151 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed0018070 con 0x7f0ec8004800 2024-05-09T10:50:30.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.962+0000 7f0ec7fff640 1 -- 172.21.15.98:0/1050388151 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed000de50 con 0x7f0ec8004800 2024-05-09T10:50:30.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.962+0000 7f0ee608f640 1 -- 172.21.15.98:0/1050388151 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee015f600 con 0x7f0ec8004800 2024-05-09T10:50:30.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.962+0000 7f0ee608f640 1 -- 172.21.15.98:0/1050388151 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee015fbc0 con 0x7f0ec8004800 2024-05-09T10:50:30.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.963+0000 7f0ee608f640 1 -- 172.21.15.98:0/1050388151 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee00b0a40 con 0x7f0ec8004800 2024-05-09T10:50:30.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.963+0000 7f0ec7fff640 1 -- 172.21.15.98:0/1050388151 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ed0003970 con 0x7f0ec8004800 2024-05-09T10:50:30.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.964+0000 7f0ec7fff640 1 --2- 172.21.15.98:0/1050388151 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0ecc06b670 0x7f0ecc06db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:30.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.964+0000 7f0edeffd640 1 --2- 172.21.15.98:0/1050388151 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0ecc06b670 0x7f0ecc06db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:30.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.964+0000 7f0ec7fff640 1 -- 172.21.15.98:0/1050388151 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ed008a3e0 con 0x7f0ec8004800 2024-05-09T10:50:30.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.964+0000 7f0edeffd640 1 --2- 172.21.15.98:0/1050388151 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0ecc06b670 0x7f0ecc06db30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4033000 tx=0x7f0ed402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:30.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:30.968+0000 7f0ec7fff640 1 -- 172.21.15.98:0/1050388151 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ee00b0a40 con 0x7f0ec8004800 2024-05-09T10:50:31.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.174+0000 7f0ee608f640 1 -- 172.21.15.98:0/1050388151 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ee0069cb0 con 0x7f0ec8004800 2024-05-09T10:50:31.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.175+0000 7f0ec7fff640 1 -- 172.21.15.98:0/1050388151 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f0ed001a310 con 0x7f0ec8004800 2024-05-09T10:50:31.169 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:31.169 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2903091926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2903091926},{"type":"v1","addr":"172.21.15.98:6837","nonce":2903091926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/968936316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":968936316},{"type":"v1","addr":"172.21.15.105:6834","nonce":968936316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/937700297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":937700297},{"type":"v1","addr":"172.21.15.98:6835","nonce":937700297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T10:50:19.544432+0000","modified":"2024-05-09T10:50:21.512942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6835/1780062557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1780062557},{"type":"v1","addr":"172.21.15.105:6835","nonce":1780062557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T10:50:31.169 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 6 2024-05-09T10:50:31.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.177+0000 7f0ee608f640 1 -- 172.21.15.98:0/1050388151 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0ecc06b670 msgr2=0x7f0ecc06db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:31.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.177+0000 7f0ee608f640 1 --2- 172.21.15.98:0/1050388151 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0ecc06b670 0x7f0ecc06db30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4033000 tx=0x7f0ed402e000 comp rx=0 tx=0).stop 2024-05-09T10:50:31.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.177+0000 7f0ee608f640 1 -- 172.21.15.98:0/1050388151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ec8004800 msgr2=0x7f0ee0146d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:31.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.177+0000 7f0ee608f640 1 --2- 172.21.15.98:0/1050388151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ec8004800 0x7f0ee0146d00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000fb90 tx=0x7f0ed000d520 comp rx=0 tx=0).stop 2024-05-09T10:50:31.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.177+0000 7f0ee608f640 1 -- 172.21.15.98:0/1050388151 shutdown_connections 2024-05-09T10:50:31.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.177+0000 7f0ee608f640 1 --2- 172.21.15.98:0/1050388151 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0ecc06b670 0x7f0ecc06db30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:31.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.177+0000 7f0ee608f640 1 --2- 172.21.15.98:0/1050388151 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ee0155fc0 0x7f0ee0156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:31.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.177+0000 7f0ee608f640 1 --2- 172.21.15.98:0/1050388151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ee01543d0 0x7f0ee00b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:31.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.177+0000 7f0ee608f640 1 --2- 172.21.15.98:0/1050388151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ec8004800 0x7f0ee0146d00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:31.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.177+0000 7f0ee608f640 1 -- 172.21.15.98:0/1050388151 >> 172.21.15.98:0/1050388151 conn(0x7f0ee0066f20 msgr2=0x7f0ee00b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:31.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.177+0000 7f0ee608f640 1 -- 172.21.15.98:0/1050388151 shutdown_connections 2024-05-09T10:50:31.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.177+0000 7f0ee608f640 1 -- 172.21.15.98:0/1050388151 wait complete. 2024-05-09T10:50:31.182 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:31.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.287+0000 7f78910bd640 1 Processor -- start 2024-05-09T10:50:31.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.288+0000 7f78910bd640 1 -- start start 2024-05-09T10:50:31.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.288+0000 7f78910bd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f788c06bbf0 0x7f788c06c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:31.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.288+0000 7f78910bd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f788c152150 0x7f788c152530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:31.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.288+0000 7f78910bd640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f788c15c190 con 0x7f788c15f0e0 2024-05-09T10:50:31.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.288+0000 7f78910bd640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788c15cda0 con 0x7f788c152150 2024-05-09T10:50:31.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.288+0000 7f78910bd640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788c15c470 con 0x7f788c06bbf0 2024-05-09T10:50:31.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.289+0000 7f788b59e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f788c152150 0x7f788c152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:31.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.289+0000 7f788ad9d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f788c15f0e0 0x7f788c0cf5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56112/0 (socket says 172.21.15.98:56112) 2024-05-09T10:50:31.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.289+0000 7f788ad9d640 1 -- 172.21.15.98:0/3940552798 learned_addr learned my addr 172.21.15.98:0/3940552798 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:31.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.289+0000 7f788a59c640 1 --2- 172.21.15.98:0/3940552798 >> v2:172.21.15.105:3300/0 conn(0x7f788c06bbf0 0x7f788c06c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:31.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.289+0000 7f788b59e640 1 -- 172.21.15.98:0/3940552798 >> v2:172.21.15.105:3300/0 conn(0x7f788c06bbf0 msgr2=0x7f788c06c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:31.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.289+0000 7f788b59e640 1 --2- 172.21.15.98:0/3940552798 >> v2:172.21.15.105:3300/0 conn(0x7f788c06bbf0 0x7f788c06c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:31.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.289+0000 7f788b59e640 1 -- 172.21.15.98:0/3940552798 >> v1:172.21.15.98:6789/0 conn(0x7f788c15f0e0 legacy=0x7f788c0cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:31.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.289+0000 7f788b59e640 1 -- 172.21.15.98:0/3940552798 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c15c750 con 0x7f788c152150 2024-05-09T10:50:31.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.289+0000 7f7889d9b640 1 -- 172.21.15.98:0/3940552798 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1148655051 0 0) 0x7f788c15c190 con 0x7f788c15f0e0 2024-05-09T10:50:31.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.289+0000 7f788b59e640 1 --2- 172.21.15.98:0/3940552798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f788c152150 0x7f788c152530 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f788c06dcc0 tx=0x7f788402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646579e5f2b79db6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:31.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.290+0000 7f7889d9b640 1 -- 172.21.15.98:0/3940552798 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7884007740 con 0x7f788c152150 2024-05-09T10:50:31.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.290+0000 7f7889d9b640 1 -- 172.21.15.98:0/3940552798 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78840078e0 con 0x7f788c152150 2024-05-09T10:50:31.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.290+0000 7f7889d9b640 1 -- 172.21.15.98:0/3940552798 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7884007cb0 con 0x7f788c152150 2024-05-09T10:50:31.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.290+0000 7f78910bd640 1 -- 172.21.15.98:0/3940552798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f788c152150 msgr2=0x7f788c152530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:31.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.290+0000 7f78910bd640 1 --2- 172.21.15.98:0/3940552798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f788c152150 0x7f788c152530 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f788c06dcc0 tx=0x7f788402a680 comp rx=0 tx=0).stop 2024-05-09T10:50:31.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.290+0000 7f78910bd640 1 -- 172.21.15.98:0/3940552798 shutdown_connections 2024-05-09T10:50:31.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.290+0000 7f78910bd640 1 --2- 172.21.15.98:0/3940552798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f788c152150 0x7f788c152530 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:31.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.290+0000 7f78910bd640 1 --2- 172.21.15.98:0/3940552798 >> v2:172.21.15.105:3300/0 conn(0x7f788c06bbf0 0x7f788c06c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:31.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.290+0000 7f78910bd640 1 -- 172.21.15.98:0/3940552798 >> 172.21.15.98:0/3940552798 conn(0x7f788c06cae0 msgr2=0x7f788c06cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:31.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.290+0000 7f78910bd640 1 -- 172.21.15.98:0/3940552798 shutdown_connections 2024-05-09T10:50:31.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.290+0000 7f78910bd640 1 -- 172.21.15.98:0/3940552798 wait complete. 2024-05-09T10:50:31.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.292+0000 7f78910bd640 1 Processor -- start 2024-05-09T10:50:31.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.292+0000 7f78910bd640 1 -- start start 2024-05-09T10:50:31.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.292+0000 7f78910bd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f788c06bbf0 0x7f788c152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:31.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.292+0000 7f78910bd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f788c152150 0x7f788c14aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:31.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.292+0000 7f78910bd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f788c15f0e0 0x7f788c13e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:31.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.292+0000 7f78910bd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c0afa70 con 0x7f788c15f0e0 2024-05-09T10:50:31.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.292+0000 7f78910bd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c0af8f0 con 0x7f788c06bbf0 2024-05-09T10:50:31.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.292+0000 7f78910bd640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788c0cfcb0 con 0x7f788c152150 2024-05-09T10:50:31.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.292+0000 7f788b59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f788c15f0e0 0x7f788c13e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:31.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.292+0000 7f788b59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f788c15f0e0 0x7f788c13e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42468/0 (socket says 172.21.15.98:42468) 2024-05-09T10:50:31.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.292+0000 7f788b59e640 1 -- 172.21.15.98:0/628168437 learned_addr learned my addr 172.21.15.98:0/628168437 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:31.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.293+0000 7f788ad9d640 1 --2- 172.21.15.98:0/628168437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f788c06bbf0 0x7f788c152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:31.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.293+0000 7f788a59c640 1 --2- 172.21.15.98:0/628168437 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f788c152150 0x7f788c14aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:31.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.293+0000 7f788b59e640 1 -- 172.21.15.98:0/628168437 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f788c152150 msgr2=0x7f788c14aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:31.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.293+0000 7f788b59e640 1 --2- 172.21.15.98:0/628168437 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f788c152150 0x7f788c14aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:31.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.293+0000 7f788b59e640 1 -- 172.21.15.98:0/628168437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f788c06bbf0 msgr2=0x7f788c152ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:31.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.293+0000 7f788b59e640 1 --2- 172.21.15.98:0/628168437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f788c06bbf0 0x7f788c152ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:31.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.293+0000 7f788b59e640 1 -- 172.21.15.98:0/628168437 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c153430 con 0x7f788c15f0e0 2024-05-09T10:50:31.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.293+0000 7f788ad9d640 1 --2- 172.21.15.98:0/628168437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f788c06bbf0 0x7f788c152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:31.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.293+0000 7f788a59c640 1 --2- 172.21.15.98:0/628168437 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f788c152150 0x7f788c14aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:31.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.293+0000 7f788b59e640 1 --2- 172.21.15.98:0/628168437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f788c15f0e0 0x7f788c13e560 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f788402ac70 tx=0x7f7884004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:31.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.293+0000 7f78737fe640 1 -- 172.21.15.98:0/628168437 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7884040070 con 0x7f788c15f0e0 2024-05-09T10:50:31.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.293+0000 7f78737fe640 1 -- 172.21.15.98:0/628168437 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7884031bc0 con 0x7f788c15f0e0 2024-05-09T10:50:31.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.293+0000 7f78910bd640 1 -- 172.21.15.98:0/628168437 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f788c1f18a0 con 0x7f788c15f0e0 2024-05-09T10:50:31.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.293+0000 7f78737fe640 1 -- 172.21.15.98:0/628168437 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7884009450 con 0x7f788c15f0e0 2024-05-09T10:50:31.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.293+0000 7f78910bd640 1 -- 172.21.15.98:0/628168437 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f788c1f1e60 con 0x7f788c15f0e0 2024-05-09T10:50:31.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.294+0000 7f78910bd640 1 -- 172.21.15.98:0/628168437 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f788c069cb0 con 0x7f788c15f0e0 2024-05-09T10:50:31.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.295+0000 7f78737fe640 1 -- 172.21.15.98:0/628168437 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7884039040 con 0x7f788c15f0e0 2024-05-09T10:50:31.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.295+0000 7f78737fe640 1 --2- 172.21.15.98:0/628168437 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f787806b4d0 0x7f787806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:31.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.295+0000 7f788ad9d640 1 --2- 172.21.15.98:0/628168437 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f787806b4d0 0x7f787806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:31.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.295+0000 7f78737fe640 1 -- 172.21.15.98:0/628168437 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78840aba50 con 0x7f788c15f0e0 2024-05-09T10:50:31.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.295+0000 7f788ad9d640 1 --2- 172.21.15.98:0/628168437 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f787806b4d0 0x7f787806d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7880000f60 tx=0x7f7880009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:31.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.299+0000 7f78737fe640 1 -- 172.21.15.98:0/628168437 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f788407a280 con 0x7f788c15f0e0 2024-05-09T10:50:31.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.473+0000 7f78910bd640 1 -- 172.21.15.98:0/628168437 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f788c0b1840 con 0x7f788c15f0e0 2024-05-09T10:50:31.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.474+0000 7f78737fe640 1 -- 172.21.15.98:0/628168437 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f788407e320 con 0x7f788c15f0e0 2024-05-09T10:50:31.469 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:31.469 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":20,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:21.176419+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:50:17.323444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T10:50:18.482315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:31.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.476+0000 7f78910bd640 1 -- 172.21.15.98:0/628168437 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f787806b4d0 msgr2=0x7f787806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:31.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.476+0000 7f78910bd640 1 --2- 172.21.15.98:0/628168437 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f787806b4d0 0x7f787806d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7880000f60 tx=0x7f7880009040 comp rx=0 tx=0).stop 2024-05-09T10:50:31.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.476+0000 7f78910bd640 1 -- 172.21.15.98:0/628168437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f788c15f0e0 msgr2=0x7f788c13e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:31.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.476+0000 7f78910bd640 1 --2- 172.21.15.98:0/628168437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f788c15f0e0 0x7f788c13e560 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f788402ac70 tx=0x7f7884004a40 comp rx=0 tx=0).stop 2024-05-09T10:50:31.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.476+0000 7f78910bd640 1 -- 172.21.15.98:0/628168437 shutdown_connections 2024-05-09T10:50:31.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.476+0000 7f78910bd640 1 --2- 172.21.15.98:0/628168437 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f787806b4d0 0x7f787806d990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:31.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.476+0000 7f78910bd640 1 --2- 172.21.15.98:0/628168437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f788c15f0e0 0x7f788c13e560 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:31.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.476+0000 7f78910bd640 1 --2- 172.21.15.98:0/628168437 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f788c152150 0x7f788c14aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:31.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.476+0000 7f78910bd640 1 --2- 172.21.15.98:0/628168437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f788c06bbf0 0x7f788c152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:31.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.476+0000 7f78910bd640 1 -- 172.21.15.98:0/628168437 >> 172.21.15.98:0/628168437 conn(0x7f788c06cae0 msgr2=0x7f788c0b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:31.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.477+0000 7f78910bd640 1 -- 172.21.15.98:0/628168437 shutdown_connections 2024-05-09T10:50:31.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.477+0000 7f78910bd640 1 -- 172.21.15.98:0/628168437 wait complete. 2024-05-09T10:50:31.481 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T10:50:31.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.587+0000 7f71c0476640 1 Processor -- start 2024-05-09T10:50:31.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.587+0000 7f71c0476640 1 -- start start 2024-05-09T10:50:31.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.588+0000 7f71c0476640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71b815de40 0x7f71b80d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:31.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.588+0000 7f71c0476640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f71b80cf250 0x7f71b80cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:31.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.588+0000 7f71c0476640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b815e220 con 0x7f71b80d0470 2024-05-09T10:50:31.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.588+0000 7f71c0476640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b80d1790 con 0x7f71b80cf250 2024-05-09T10:50:31.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.588+0000 7f71c0476640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b80d19e0 con 0x7f71b815de40 2024-05-09T10:50:31.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.588+0000 7f71bda11640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f71b80cf250 0x7f71b80cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:31.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.588+0000 7f71bda11640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f71b80cf250 0x7f71b80cf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42484/0 (socket says 172.21.15.98:42484) 2024-05-09T10:50:31.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.588+0000 7f71bda11640 1 -- 172.21.15.98:0/526567118 learned_addr learned my addr 172.21.15.98:0/526567118 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:31.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.588+0000 7f71bda11640 1 -- 172.21.15.98:0/526567118 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71b815de40 msgr2=0x7f71b80d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:31.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.588+0000 7f71bda11640 1 --2- 172.21.15.98:0/526567118 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71b815de40 0x7f71b80d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:31.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.588+0000 7f71bda11640 1 -- 172.21.15.98:0/526567118 >> v1:172.21.15.105:6789/0 conn(0x7f71b80d0470 legacy=0x7f71b80b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:31.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.588+0000 7f71bda11640 1 -- 172.21.15.98:0/526567118 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b80d1cc0 con 0x7f71b80cf250 2024-05-09T10:50:31.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.589+0000 7f71bda11640 1 --2- 172.21.15.98:0/526567118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b80cf250 0x7f71b80cf6b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f71a8009870 tx=0x7f71a802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae0281e595487b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:31.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.589+0000 7f71bd210640 1 -- 172.21.15.98:0/526567118 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8037070 con 0x7f71b80cf250 2024-05-09T10:50:31.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.589+0000 7f71bd210640 1 -- 172.21.15.98:0/526567118 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a8032440 con 0x7f71b80cf250 2024-05-09T10:50:31.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.589+0000 7f71bd210640 1 -- 172.21.15.98:0/526567118 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8032810 con 0x7f71b80cf250 2024-05-09T10:50:31.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.589+0000 7f71c0476640 1 -- 172.21.15.98:0/526567118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b80cf250 msgr2=0x7f71b80cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:31.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.589+0000 7f71c0476640 1 --2- 172.21.15.98:0/526567118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b80cf250 0x7f71b80cf6b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f71a8009870 tx=0x7f71a802a210 comp rx=0 tx=0).stop 2024-05-09T10:50:31.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.590+0000 7f71c0476640 1 -- 172.21.15.98:0/526567118 shutdown_connections 2024-05-09T10:50:31.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.590+0000 7f71c0476640 1 --2- 172.21.15.98:0/526567118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b80cf250 0x7f71b80cf6b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:31.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.590+0000 7f71c0476640 1 --2- 172.21.15.98:0/526567118 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71b815de40 0x7f71b80d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:31.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.590+0000 7f71c0476640 1 -- 172.21.15.98:0/526567118 >> 172.21.15.98:0/526567118 conn(0x7f71b809ce90 msgr2=0x7f71b80d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:31.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.590+0000 7f71c0476640 1 -- 172.21.15.98:0/526567118 shutdown_connections 2024-05-09T10:50:31.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.590+0000 7f71c0476640 1 -- 172.21.15.98:0/526567118 wait complete. 2024-05-09T10:50:31.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.591+0000 7f71c0476640 1 Processor -- start 2024-05-09T10:50:31.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.591+0000 7f71c0476640 1 -- start start 2024-05-09T10:50:31.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.592+0000 7f71c0476640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71b80cf250 0x7f71b81f3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:31.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.592+0000 7f71c0476640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71b80d0470 0x7f71b80c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:31.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.592+0000 7f71c0476640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b815de40 0x7f71b80c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:31.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.592+0000 7f71c0476640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b80d4680 con 0x7f71b815de40 2024-05-09T10:50:31.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.592+0000 7f71c0476640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b80d2280 con 0x7f71b80d0470 2024-05-09T10:50:31.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.592+0000 7f71c0476640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b80d4800 con 0x7f71b80cf250 2024-05-09T10:50:31.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.592+0000 7f71bea13640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b815de40 0x7f71b80c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:31.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.592+0000 7f71bea13640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b815de40 0x7f71b80c7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42500/0 (socket says 172.21.15.98:42500) 2024-05-09T10:50:31.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.592+0000 7f71bda11640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71b80d0470 0x7f71b80c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:31.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.592+0000 7f71bea13640 1 -- 172.21.15.98:0/687341126 learned_addr learned my addr 172.21.15.98:0/687341126 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:31.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.592+0000 7f71be212640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71b80cf250 0x7f71b81f3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:31.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.592+0000 7f71bea13640 1 -- 172.21.15.98:0/687341126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71b80cf250 msgr2=0x7f71b81f3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:31.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.592+0000 7f71bea13640 1 --2- 172.21.15.98:0/687341126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71b80cf250 0x7f71b81f3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:31.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.592+0000 7f71bea13640 1 -- 172.21.15.98:0/687341126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71b80d0470 msgr2=0x7f71b80c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:31.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.592+0000 7f71bea13640 1 --2- 172.21.15.98:0/687341126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71b80d0470 0x7f71b80c78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:31.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.592+0000 7f71bea13640 1 -- 172.21.15.98:0/687341126 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b80cd600 con 0x7f71b815de40 2024-05-09T10:50:31.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.592+0000 7f71bda11640 1 --2- 172.21.15.98:0/687341126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71b80d0470 0x7f71b80c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:50:31.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.592+0000 7f71bea13640 1 --2- 172.21.15.98:0/687341126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b815de40 0x7f71b80c7e20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f71b000c780 tx=0x7f71b000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:31.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.592+0000 7f71aeffd640 1 -- 172.21.15.98:0/687341126 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b0004b60 con 0x7f71b815de40 2024-05-09T10:50:31.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.592+0000 7f71aeffd640 1 -- 172.21.15.98:0/687341126 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b0004d00 con 0x7f71b815de40 2024-05-09T10:50:31.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.592+0000 7f71c0476640 1 -- 172.21.15.98:0/687341126 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b80cd8e0 con 0x7f71b815de40 2024-05-09T10:50:31.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.592+0000 7f71aeffd640 1 -- 172.21.15.98:0/687341126 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b000aa90 con 0x7f71b815de40 2024-05-09T10:50:31.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.592+0000 7f71c0476640 1 -- 172.21.15.98:0/687341126 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b81f4620 con 0x7f71b815de40 2024-05-09T10:50:31.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.593+0000 7f71c0476640 1 -- 172.21.15.98:0/687341126 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b80cfa20 con 0x7f71b815de40 2024-05-09T10:50:31.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.594+0000 7f71aeffd640 1 -- 172.21.15.98:0/687341126 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f71b000ac30 con 0x7f71b815de40 2024-05-09T10:50:31.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.594+0000 7f71aeffd640 1 --2- 172.21.15.98:0/687341126 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f719c06b4d0 0x7f719c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:31.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.594+0000 7f71aeffd640 1 -- 172.21.15.98:0/687341126 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71b008a2f0 con 0x7f71b815de40 2024-05-09T10:50:31.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.594+0000 7f71be212640 1 --2- 172.21.15.98:0/687341126 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f719c06b4d0 0x7f719c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:31.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.595+0000 7f71be212640 1 --2- 172.21.15.98:0/687341126 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f719c06b4d0 0x7f719c06d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f71b40098e0 tx=0x7f71b4009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:31.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.598+0000 7f71aeffd640 1 -- 172.21.15.98:0/687341126 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f71b80cfa20 con 0x7f71b815de40 2024-05-09T10:50:31.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.802+0000 7f71c0476640 1 -- 172.21.15.98:0/687341126 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71b80d42d0 con 0x7f71b815de40 2024-05-09T10:50:31.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.803+0000 7f71aeffd640 1 -- 172.21.15.98:0/687341126 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f71b0020070 con 0x7f71b815de40 2024-05-09T10:50:31.796 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:31.797 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2903091926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2903091926},{"type":"v1","addr":"172.21.15.98:6837","nonce":2903091926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/968936316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":968936316},{"type":"v1","addr":"172.21.15.105:6834","nonce":968936316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/937700297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":937700297},{"type":"v1","addr":"172.21.15.98:6835","nonce":937700297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T10:50:19.544432+0000","modified":"2024-05-09T10:50:21.512942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6835/1780062557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1780062557},{"type":"v1","addr":"172.21.15.105:6835","nonce":1780062557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T10:50:31.797 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 6 2024-05-09T10:50:31.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.805+0000 7f71c0476640 1 -- 172.21.15.98:0/687341126 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f719c06b4d0 msgr2=0x7f719c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:31.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.805+0000 7f71c0476640 1 --2- 172.21.15.98:0/687341126 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f719c06b4d0 0x7f719c06d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f71b40098e0 tx=0x7f71b4009060 comp rx=0 tx=0).stop 2024-05-09T10:50:31.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.805+0000 7f71c0476640 1 -- 172.21.15.98:0/687341126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b815de40 msgr2=0x7f71b80c7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:31.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.805+0000 7f71c0476640 1 --2- 172.21.15.98:0/687341126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b815de40 0x7f71b80c7e20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f71b000c780 tx=0x7f71b000cc40 comp rx=0 tx=0).stop 2024-05-09T10:50:31.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.805+0000 7f71c0476640 1 -- 172.21.15.98:0/687341126 shutdown_connections 2024-05-09T10:50:31.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.805+0000 7f71c0476640 1 --2- 172.21.15.98:0/687341126 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f719c06b4d0 0x7f719c06d990 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:31.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.805+0000 7f71c0476640 1 --2- 172.21.15.98:0/687341126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71b815de40 0x7f71b80c7e20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:31.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.805+0000 7f71c0476640 1 --2- 172.21.15.98:0/687341126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71b80d0470 0x7f71b80c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:31.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.805+0000 7f71c0476640 1 --2- 172.21.15.98:0/687341126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71b80cf250 0x7f71b81f3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:31.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.805+0000 7f71c0476640 1 -- 172.21.15.98:0/687341126 >> 172.21.15.98:0/687341126 conn(0x7f71b809ce90 msgr2=0x7f71b80b7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:31.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.805+0000 7f71c0476640 1 -- 172.21.15.98:0/687341126 shutdown_connections 2024-05-09T10:50:31.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:31.805+0000 7f71c0476640 1 -- 172.21.15.98:0/687341126 wait complete. 2024-05-09T10:50:32.213 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T10:50:19.544432+0000', 'modified': '2024-05-09T10:50:21.512942+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6835/1780062557', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6833', 'nonce': 1780062557}, {'type': 'v1', 'addr': '172.21.15.105:6835', 'nonce': 1780062557}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-09T10:50:32.213 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T10:50:32.213 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-09T10:50:32.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.322+0000 7f6da434f640 1 Processor -- start 2024-05-09T10:50:32.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.322+0000 7f6da434f640 1 -- start start 2024-05-09T10:50:32.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da434f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d9c06ed40 0x7f6d9c0af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da434f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d9c1553e0 con 0x7f6d9c155000 2024-05-09T10:50:32.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da434f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d9c0b3860 con 0x7f6d9c071b00 2024-05-09T10:50:32.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da434f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c0b4430 con 0x7f6d9c06ed40 2024-05-09T10:50:32.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da20eb640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f6d9c155000 0x7f6d9c0713f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56118/0 (socket says 172.21.15.98:56118) 2024-05-09T10:50:32.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da20eb640 1 -- 172.21.15.98:0/1058614120 learned_addr learned my addr 172.21.15.98:0/1058614120 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:32.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da28ec640 1 --2- 172.21.15.98:0/1058614120 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d9c06ed40 0x7f6d9c0af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:32.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da10e9640 1 -- 172.21.15.98:0/1058614120 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360858247 0 0) 0x7f6d9c1553e0 con 0x7f6d9c155000 2024-05-09T10:50:32.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da10e9640 1 -- 172.21.15.98:0/1058614120 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d88003520 con 0x7f6d9c155000 2024-05-09T10:50:32.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da10e9640 1 -- 172.21.15.98:0/1058614120 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4016025408 0 0) 0x7f6d9c0b3860 con 0x7f6d9c071b00 2024-05-09T10:50:32.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da10e9640 1 -- 172.21.15.98:0/1058614120 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d9c1553e0 con 0x7f6d9c071b00 2024-05-09T10:50:32.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da10e9640 1 -- 172.21.15.98:0/1058614120 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2386581397 0 0) 0x7f6d88003520 con 0x7f6d9c155000 2024-05-09T10:50:32.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da10e9640 1 -- 172.21.15.98:0/1058614120 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d9c06ed40 msgr2=0x7f6d9c0af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:32.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da10e9640 1 --2- 172.21.15.98:0/1058614120 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d9c06ed40 0x7f6d9c0af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da10e9640 1 -- 172.21.15.98:0/1058614120 >> v1:172.21.15.105:6789/0 conn(0x7f6d9c071b00 legacy=0x7f6d9c06e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:32.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da10e9640 1 -- 172.21.15.98:0/1058614120 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9c0b3b00 con 0x7f6d9c155000 2024-05-09T10:50:32.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da10e9640 1 -- 172.21.15.98:0/1058614120 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f6d98001670 con 0x7f6d9c155000 2024-05-09T10:50:32.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da10e9640 1 -- 172.21.15.98:0/1058614120 >> v1:172.21.15.98:6789/0 conn(0x7f6d9c155000 legacy=0x7f6d9c0713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:32.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da10e9640 1 --2- 172.21.15.98:0/1058614120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d88003b70 0x7f6d88003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da10e9640 1 --2- 172.21.15.98:0/1058614120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d88004800 0x7f6d88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da10e9640 1 --2- 172.21.15.98:0/1058614120 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d9c06ed40 0x7f6d880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da10e9640 1 -- 172.21.15.98:0/1058614120 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c0b4430 con 0x7f6d88003b70 2024-05-09T10:50:32.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da10e9640 1 -- 172.21.15.98:0/1058614120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8800dc80 con 0x7f6d88004800 2024-05-09T10:50:32.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.323+0000 7f6da10e9640 1 -- 172.21.15.98:0/1058614120 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8800de50 con 0x7f6d9c06ed40 2024-05-09T10:50:32.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.324+0000 7f6da20eb640 1 --2- 172.21.15.98:0/1058614120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d88003b70 0x7f6d88003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:32.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.324+0000 7f6da18ea640 1 --2- 172.21.15.98:0/1058614120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d88004800 0x7f6d88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:32.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.324+0000 7f6da28ec640 1 --2- 172.21.15.98:0/1058614120 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d9c06ed40 0x7f6d880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:32.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.324+0000 7f6da20eb640 1 -- 172.21.15.98:0/1058614120 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d9c06ed40 msgr2=0x7f6d880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:32.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.324+0000 7f6da20eb640 1 --2- 172.21.15.98:0/1058614120 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d9c06ed40 0x7f6d880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.324+0000 7f6da20eb640 1 -- 172.21.15.98:0/1058614120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d88004800 msgr2=0x7f6d88004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:32.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.324+0000 7f6da20eb640 1 --2- 172.21.15.98:0/1058614120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d88004800 0x7f6d88004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.324+0000 7f6da20eb640 1 -- 172.21.15.98:0/1058614120 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d88003520 con 0x7f6d88003b70 2024-05-09T10:50:32.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.324+0000 7f6da28ec640 1 --2- 172.21.15.98:0/1058614120 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d9c06ed40 0x7f6d880051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:32.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.324+0000 7f6da20eb640 1 --2- 172.21.15.98:0/1058614120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d88003b70 0x7f6d88003f70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800c190 tx=0x7f6d88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4599ef1ae3dafb26 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:32.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.325+0000 7f6da10e9640 1 -- 172.21.15.98:0/1058614120 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98007cc0 con 0x7f6d88003b70 2024-05-09T10:50:32.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.325+0000 7f6da10e9640 1 -- 172.21.15.98:0/1058614120 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d98005af0 con 0x7f6d88003b70 2024-05-09T10:50:32.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.325+0000 7f6da10e9640 1 -- 172.21.15.98:0/1058614120 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98007070 con 0x7f6d88003b70 2024-05-09T10:50:32.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.325+0000 7f6da434f640 1 -- 172.21.15.98:0/1058614120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d88003b70 msgr2=0x7f6d88003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:32.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.325+0000 7f6da434f640 1 --2- 172.21.15.98:0/1058614120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d88003b70 0x7f6d88003f70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800c190 tx=0x7f6d88005830 comp rx=0 tx=0).stop 2024-05-09T10:50:32.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.325+0000 7f6da28ec640 1 -- 172.21.15.98:0/1058614120 reap_dead start 2024-05-09T10:50:32.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.325+0000 7f6da434f640 1 -- 172.21.15.98:0/1058614120 shutdown_connections 2024-05-09T10:50:32.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.325+0000 7f6da434f640 1 -- 172.21.15.98:0/1058614120 >> 172.21.15.98:0/1058614120 conn(0x7f6d9c0b24c0 msgr2=0x7f6d9c0b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:32.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.325+0000 7f6da434f640 1 -- 172.21.15.98:0/1058614120 shutdown_connections 2024-05-09T10:50:32.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.325+0000 7f6da434f640 1 -- 172.21.15.98:0/1058614120 wait complete. 2024-05-09T10:50:32.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.326+0000 7f6da434f640 1 Processor -- start 2024-05-09T10:50:32.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.326+0000 7f6da434f640 1 -- start start 2024-05-09T10:50:32.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.327+0000 7f6da434f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d88003b70 0x7f6d9c149200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.327+0000 7f6da434f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d9c148510 0x7f6d9c148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.327+0000 7f6da434f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d9c156ba0 0x7f6d9c157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.327+0000 7f6da434f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c15bdd0 con 0x7f6d88003b70 2024-05-09T10:50:32.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.327+0000 7f6da434f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c0b40c0 con 0x7f6d9c148510 2024-05-09T10:50:32.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.327+0000 7f6da434f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c158b60 con 0x7f6d9c156ba0 2024-05-09T10:50:32.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.327+0000 7f6da20eb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d88003b70 0x7f6d9c149200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:32.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.327+0000 7f6da20eb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d88003b70 0x7f6d9c149200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42508/0 (socket says 172.21.15.98:42508) 2024-05-09T10:50:32.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.327+0000 7f6da20eb640 1 -- 172.21.15.98:0/1480002007 learned_addr learned my addr 172.21.15.98:0/1480002007 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:32.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.327+0000 7f6da28ec640 1 --2- 172.21.15.98:0/1480002007 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d9c156ba0 0x7f6d9c157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:32.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.327+0000 7f6da20eb640 1 -- 172.21.15.98:0/1480002007 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d9c156ba0 msgr2=0x7f6d9c157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:32.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.327+0000 7f6da18ea640 1 --2- 172.21.15.98:0/1480002007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d9c148510 0x7f6d9c148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:32.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.327+0000 7f6da20eb640 1 --2- 172.21.15.98:0/1480002007 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d9c156ba0 0x7f6d9c157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.327+0000 7f6da20eb640 1 -- 172.21.15.98:0/1480002007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d9c148510 msgr2=0x7f6d9c148990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:32.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.327+0000 7f6da20eb640 1 --2- 172.21.15.98:0/1480002007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d9c148510 0x7f6d9c148990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.327+0000 7f6da20eb640 1 -- 172.21.15.98:0/1480002007 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9c1e30b0 con 0x7f6d88003b70 2024-05-09T10:50:32.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.328+0000 7f6da20eb640 1 --2- 172.21.15.98:0/1480002007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d88003b70 0x7f6d9c149200 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800c190 tx=0x7f6d9c0b3cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:32.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.328+0000 7f6d92ffd640 1 -- 172.21.15.98:0/1480002007 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98041020 con 0x7f6d88003b70 2024-05-09T10:50:32.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.328+0000 7f6d92ffd640 1 -- 172.21.15.98:0/1480002007 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d98001470 con 0x7f6d88003b70 2024-05-09T10:50:32.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.328+0000 7f6da434f640 1 -- 172.21.15.98:0/1480002007 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d9c1e3350 con 0x7f6d88003b70 2024-05-09T10:50:32.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.328+0000 7f6d92ffd640 1 -- 172.21.15.98:0/1480002007 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98007270 con 0x7f6d88003b70 2024-05-09T10:50:32.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.328+0000 7f6da434f640 1 -- 172.21.15.98:0/1480002007 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d9c1e3910 con 0x7f6d88003b70 2024-05-09T10:50:32.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.329+0000 7f6d92ffd640 1 -- 172.21.15.98:0/1480002007 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6d98044020 con 0x7f6d88003b70 2024-05-09T10:50:32.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.330+0000 7f6d92ffd640 1 --2- 172.21.15.98:0/1480002007 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6d8006b4d0 0x7f6d8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.330+0000 7f6d92ffd640 1 -- 172.21.15.98:0/1480002007 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d980aa230 con 0x7f6d88003b70 2024-05-09T10:50:32.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.330+0000 7f6da18ea640 1 --2- 172.21.15.98:0/1480002007 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6d8006b4d0 0x7f6d8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:32.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.330+0000 7f6da18ea640 1 --2- 172.21.15.98:0/1480002007 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6d8006b4d0 0x7f6d8006d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c158320 tx=0x7f6d8c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:32.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.331+0000 7f6da704c740 1 Processor -- start 2024-05-09T10:50:32.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.331+0000 7f6da704c740 1 -- start start 2024-05-09T10:50:32.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.331+0000 7f6da704c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb50ae1230 0x55eb50ae1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.331+0000 7f6da704c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50afead0 0x55eb50aff020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.331+0000 7f6da704c740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55eb50ab6f10 0x55eb50afd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.331+0000 7f6da704c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb50a19a10 con 0x55eb50afead0 2024-05-09T10:50:32.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.331+0000 7f6da704c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb509c2b30 con 0x55eb50ae1230 2024-05-09T10:50:32.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.331+0000 7f6da704c740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb50a15c50 con 0x55eb50ab6f10 2024-05-09T10:50:32.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.332+0000 7f6da18ea640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50afead0 0x55eb50aff020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:32.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.332+0000 7f6da18ea640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50afead0 0x55eb50aff020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42518/0 (socket says 172.21.15.98:42518) 2024-05-09T10:50:32.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.332+0000 7f6da18ea640 1 -- 172.21.15.98:0/2114577332 learned_addr learned my addr 172.21.15.98:0/2114577332 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:32.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.332+0000 7f6da28ec640 1 --2- 172.21.15.98:0/2114577332 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55eb50ab6f10 0x55eb50afd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:32.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.332+0000 7f6da20eb640 1 --2- 172.21.15.98:0/2114577332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb50ae1230 0x55eb50ae1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:32.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.332+0000 7f6da18ea640 1 -- 172.21.15.98:0/2114577332 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55eb50ab6f10 msgr2=0x55eb50afd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:32.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.332+0000 7f6da18ea640 1 --2- 172.21.15.98:0/2114577332 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55eb50ab6f10 0x55eb50afd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.332+0000 7f6da18ea640 1 -- 172.21.15.98:0/2114577332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb50ae1230 msgr2=0x55eb50ae1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:32.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.332+0000 7f6da18ea640 1 --2- 172.21.15.98:0/2114577332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb50ae1230 0x55eb50ae1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.332+0000 7f6da18ea640 1 -- 172.21.15.98:0/2114577332 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb509d7e10 con 0x55eb50afead0 2024-05-09T10:50:32.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.332+0000 7f6da20eb640 1 --2- 172.21.15.98:0/2114577332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb50ae1230 0x55eb50ae1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:32.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.332+0000 7f6da18ea640 1 --2- 172.21.15.98:0/2114577332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50afead0 0x55eb50aff020 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c007830 tx=0x7f6d8c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff4d76086ff54a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:32.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.332+0000 7f6d7ffff640 1 -- 172.21.15.98:0/2114577332 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c027070 con 0x55eb50afead0 2024-05-09T10:50:32.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.332+0000 7f6d7ffff640 1 -- 172.21.15.98:0/2114577332 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8c0147c0 con 0x55eb50afead0 2024-05-09T10:50:32.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.332+0000 7f6d7ffff640 1 -- 172.21.15.98:0/2114577332 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c014b70 con 0x55eb50afead0 2024-05-09T10:50:32.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.333+0000 7f6da704c740 1 -- 172.21.15.98:0/2114577332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50afead0 msgr2=0x55eb50aff020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:32.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.333+0000 7f6da704c740 1 --2- 172.21.15.98:0/2114577332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50afead0 0x55eb50aff020 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c007830 tx=0x7f6d8c013000 comp rx=0 tx=0).stop 2024-05-09T10:50:32.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.333+0000 7f6da704c740 1 -- 172.21.15.98:0/2114577332 shutdown_connections 2024-05-09T10:50:32.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.333+0000 7f6da704c740 1 --2- 172.21.15.98:0/2114577332 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55eb50ab6f10 0x55eb50afd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.333+0000 7f6da704c740 1 --2- 172.21.15.98:0/2114577332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50afead0 0x55eb50aff020 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.333+0000 7f6da704c740 1 --2- 172.21.15.98:0/2114577332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb50ae1230 0x55eb50ae1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.333+0000 7f6da704c740 1 -- 172.21.15.98:0/2114577332 >> 172.21.15.98:0/2114577332 conn(0x55eb50ab3840 msgr2=0x55eb50afc040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:32.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.333+0000 7f6da704c740 1 -- 172.21.15.98:0/2114577332 shutdown_connections 2024-05-09T10:50:32.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.333+0000 7f6da704c740 1 -- 172.21.15.98:0/2114577332 wait complete. 2024-05-09T10:50:32.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.333+0000 7f6da704c740 1 -- start start 2024-05-09T10:50:32.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.334+0000 7f6da704c740 1 Processor -- start 2024-05-09T10:50:32.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.334+0000 7f6da704c740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55eb50ae1230 0x55eb50adf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.334+0000 7f6da704c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50afead0 0x55eb50af32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.334+0000 7f6da704c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb50af37f0 0x55eb50acc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.334+0000 7f6da704c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb50ab7cb0 con 0x55eb50afead0 2024-05-09T10:50:32.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.334+0000 7f6da704c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb509d60c0 con 0x55eb50af37f0 2024-05-09T10:50:32.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.334+0000 7f6da704c740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb50ab1770 con 0x55eb50ae1230 2024-05-09T10:50:32.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.335+0000 7f6da704c740 10 client.0 _ll_register_callbacks cb 0x55eb509c9ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T10:50:32.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.335+0000 7f6da18ea640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50afead0 0x55eb50af32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:32.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.335+0000 7f6da18ea640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50afead0 0x55eb50af32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42528/0 (socket says 172.21.15.98:42528) 2024-05-09T10:50:32.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.335+0000 7f6da18ea640 1 -- 172.21.15.98:0/4119075321 learned_addr learned my addr 172.21.15.98:0/4119075321 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:32.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.335+0000 7f6da28ec640 1 --2- 172.21.15.98:0/4119075321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb50af37f0 0x55eb50acc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:32.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.335+0000 7f6da20eb640 1 --2- 172.21.15.98:0/4119075321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55eb50ae1230 0x55eb50adf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:32.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.335+0000 7f6da18ea640 1 -- 172.21.15.98:0/4119075321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55eb50ae1230 msgr2=0x55eb50adf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:32.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.335+0000 7f6da18ea640 1 --2- 172.21.15.98:0/4119075321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55eb50ae1230 0x55eb50adf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.335+0000 7f6da18ea640 1 -- 172.21.15.98:0/4119075321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb50af37f0 msgr2=0x55eb50acc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:32.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.335+0000 7f6da18ea640 1 --2- 172.21.15.98:0/4119075321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb50af37f0 0x55eb50acc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.335+0000 7f6da18ea640 1 -- 172.21.15.98:0/4119075321 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb5072e2d0 con 0x55eb50afead0 2024-05-09T10:50:32.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.335+0000 7f6da18ea640 1 --2- 172.21.15.98:0/4119075321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50afead0 0x55eb50af32b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c001de0 tx=0x7f6d8c0012f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acfd777035659502 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:32.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.335+0000 7f6d7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-05-09T10:50:32.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.335+0000 7f6d7d7fa640 1 -- 172.21.15.98:0/4119075321 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c027040 con 0x55eb50afead0 2024-05-09T10:50:32.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.335+0000 7f6d7d7fa640 1 -- 172.21.15.98:0/4119075321 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d8c021620 con 0x55eb50afead0 2024-05-09T10:50:32.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.335+0000 7f6da704c740 1 -- 172.21.15.98:0/4119075321 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d8c0147c0 con 0x55eb50afead0 2024-05-09T10:50:32.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.335+0000 7f6d7d7fa640 1 -- 172.21.15.98:0/4119075321 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d8c039050 con 0x55eb50afead0 2024-05-09T10:50:32.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.335+0000 7f6da704c740 10 client.4683 fetch_fsmap learned FSMap version 6 2024-05-09T10:50:32.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.335+0000 7f6da704c740 1 -- 172.21.15.98:0/4119075321 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55eb507f86b0 con 0x55eb50afead0 2024-05-09T10:50:32.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.335+0000 7f6d7d7fa640 1 -- 172.21.15.98:0/4119075321 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8c0147c0 con 0x55eb50afead0 2024-05-09T10:50:32.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.336+0000 7f6d7d7fa640 1 -- 172.21.15.98:0/4119075321 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6d8c03c020 con 0x55eb50afead0 2024-05-09T10:50:32.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.336+0000 7f6da704c740 10 client.4683 fetch_fsmap finished waiting for FSMap version 6 2024-05-09T10:50:32.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.336+0000 7f6da704c740 10 client.4683 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-09T10:50:32.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.336+0000 7f6da704c740 20 client.4683 populate_metadata read hostname 'smithi098' 2024-05-09T10:50:32.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.336+0000 7f6da704c740 1 --2- 172.21.15.98:0/4119075321 >> [v2:172.21.15.105:6833/1780062557,v1:172.21.15.105:6835/1780062557] conn(0x55eb50b0c6d0 0x55eb50b0cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.336+0000 7f6da704c740 4 client.4683 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T10:50:32.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.336+0000 7f6da704c740 1 -- 172.21.15.98:0/4119075321 --> [v2:172.21.15.105:6833/1780062557,v1:172.21.15.105:6835/1780062557] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb50a78920 con 0x55eb50b0c6d0 2024-05-09T10:50:32.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.336+0000 7f6da20eb640 1 --2- 172.21.15.98:0/4119075321 >> [v2:172.21.15.105:6833/1780062557,v1:172.21.15.105:6835/1780062557] conn(0x55eb50b0c6d0 0x55eb50b0cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:32.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.336+0000 7f6da20eb640 1 --2- 172.21.15.98:0/4119075321 >> [v2:172.21.15.105:6833/1780062557,v1:172.21.15.105:6835/1780062557] conn(0x55eb50b0c6d0 0x55eb50b0cab0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b484f886399f70bd server_cookie=c4ebb5bf0a26a0b1 in_seq=0 out_seq=0 2024-05-09T10:50:32.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.336+0000 7f6d7d7fa640 10 client.4683 ms_handle_connect on v2:172.21.15.105:6833/1780062557 2024-05-09T10:50:32.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.337+0000 7f6d7d7fa640 1 -- 172.21.15.98:0/4119075321 <== mds.0 v2:172.21.15.105:6833/1780062557 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eb50a78920 con 0x55eb50b0c6d0 2024-05-09T10:50:32.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.337+0000 7f6d7d7fa640 10 client.4683 handle_command_reply: tid=0 2024-05-09T10:50:32.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.337+0000 7f6da704c740 1 client.4683 shutdown 2024-05-09T10:50:32.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.337+0000 7f6da704c740 1 -- 172.21.15.98:0/4119075321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50afead0 msgr2=0x55eb50af32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:32.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.337+0000 7f6da704c740 1 --2- 172.21.15.98:0/4119075321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50afead0 0x55eb50af32b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c001de0 tx=0x7f6d8c0012f0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.337+0000 7f6da704c740 1 -- 172.21.15.98:0/4119075321 shutdown_connections 2024-05-09T10:50:32.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.337+0000 7f6da704c740 1 --2- 172.21.15.98:0/4119075321 >> [v2:172.21.15.105:6833/1780062557,v1:172.21.15.105:6835/1780062557] conn(0x55eb50b0c6d0 0x55eb50b0cab0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.337+0000 7f6da704c740 1 --2- 172.21.15.98:0/4119075321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb50af37f0 0x55eb50acc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.337+0000 7f6d7d7fa640 0 client.4683 ms_handle_reset on v2:172.21.15.105:6833/1780062557 2024-05-09T10:50:32.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.337+0000 7f6da704c740 1 --2- 172.21.15.98:0/4119075321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50afead0 0x55eb50af32b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.337+0000 7f6da704c740 1 --2- 172.21.15.98:0/4119075321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55eb50ae1230 0x55eb50adf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.337+0000 7f6da704c740 1 -- 172.21.15.98:0/4119075321 >> 172.21.15.98:0/4119075321 conn(0x55eb50ab6f10 msgr2=0x55eb50aff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:32.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.337+0000 7f6da704c740 1 -- 172.21.15.98:0/4119075321 shutdown_connections 2024-05-09T10:50:32.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.337+0000 7f6da704c740 1 -- 172.21.15.98:0/4119075321 wait complete. 2024-05-09T10:50:32.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.337+0000 7f6da704c740 20 client.4683 trim_cache size 0 max 16384 2024-05-09T10:50:32.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.353+0000 7f6da704c740 1 Processor -- start 2024-05-09T10:50:32.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.353+0000 7f6da704c740 1 -- start start 2024-05-09T10:50:32.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.353+0000 7f6da704c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb50ae1230 0x55eb50ab74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.353+0000 7f6da704c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50af37f0 0x55eb50b61c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.353+0000 7f6da704c740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55eb50afead0 0x55eb50b621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.353+0000 7f6da704c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c1ffa00 con 0x55eb50af37f0 2024-05-09T10:50:32.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.353+0000 7f6da704c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c1fee00 con 0x55eb50ae1230 2024-05-09T10:50:32.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.353+0000 7f6da704c740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c1ff400 con 0x55eb50afead0 2024-05-09T10:50:32.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.354+0000 7f6da18ea640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50af37f0 0x55eb50b61c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:32.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.354+0000 7f6da18ea640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50af37f0 0x55eb50b61c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42530/0 (socket says 172.21.15.98:42530) 2024-05-09T10:50:32.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.354+0000 7f6da18ea640 1 -- 172.21.15.98:0/1628128418 learned_addr learned my addr 172.21.15.98:0/1628128418 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:32.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.354+0000 7f6da18ea640 1 -- 172.21.15.98:0/1628128418 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55eb50afead0 msgr2=0x55eb50b621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:32.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.354+0000 7f6da18ea640 1 --2- 172.21.15.98:0/1628128418 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55eb50afead0 0x55eb50b621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.354+0000 7f6da18ea640 1 -- 172.21.15.98:0/1628128418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb50ae1230 msgr2=0x55eb50ab74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:32.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.354+0000 7f6da18ea640 1 --2- 172.21.15.98:0/1628128418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb50ae1230 0x55eb50ab74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.354+0000 7f6da18ea640 1 -- 172.21.15.98:0/1628128418 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb50b1f760 con 0x55eb50af37f0 2024-05-09T10:50:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.354+0000 7f6da18ea640 1 --2- 172.21.15.98:0/1628128418 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50af37f0 0x55eb50b61c70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c014920 tx=0x7f6d8c02b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e3704eec39481a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.354+0000 7f6d7f7fe640 1 -- 172.21.15.98:0/1628128418 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c027040 con 0x55eb50af37f0 2024-05-09T10:50:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.354+0000 7f6d7f7fe640 1 -- 172.21.15.98:0/1628128418 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8c031070 con 0x55eb50af37f0 2024-05-09T10:50:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.354+0000 7f6d7f7fe640 1 -- 172.21.15.98:0/1628128418 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c021b10 con 0x55eb50af37f0 2024-05-09T10:50:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.355+0000 7f6da704c740 1 -- 172.21.15.98:0/1628128418 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50af37f0 msgr2=0x55eb50b61c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.355+0000 7f6da704c740 1 --2- 172.21.15.98:0/1628128418 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50af37f0 0x55eb50b61c70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c014920 tx=0x7f6d8c02b000 comp rx=0 tx=0).stop 2024-05-09T10:50:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.355+0000 7f6da704c740 1 -- 172.21.15.98:0/1628128418 shutdown_connections 2024-05-09T10:50:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.355+0000 7f6da704c740 1 --2- 172.21.15.98:0/1628128418 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55eb50afead0 0x55eb50b621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.355+0000 7f6da704c740 1 --2- 172.21.15.98:0/1628128418 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50af37f0 0x55eb50b61c70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.355+0000 7f6da704c740 1 --2- 172.21.15.98:0/1628128418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb50ae1230 0x55eb50ab74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.355+0000 7f6da704c740 1 -- 172.21.15.98:0/1628128418 >> 172.21.15.98:0/1628128418 conn(0x55eb50ab6f10 msgr2=0x55eb50af2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.355+0000 7f6da704c740 1 -- 172.21.15.98:0/1628128418 shutdown_connections 2024-05-09T10:50:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.355+0000 7f6da704c740 1 -- 172.21.15.98:0/1628128418 wait complete. 2024-05-09T10:50:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.355+0000 7f6da704c740 1 -- start start 2024-05-09T10:50:32.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.355+0000 7f6da704c740 1 Processor -- start 2024-05-09T10:50:32.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.356+0000 7f6da704c740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50ae1230 0x55eb50b0eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.356+0000 7f6da704c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb50af37f0 0x55eb50b0f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.356+0000 7f6da704c740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55eb50afead0 0x55eb50ac4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.356+0000 7f6da704c740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb50ae3120 con 0x55eb50ae1230 2024-05-09T10:50:32.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.356+0000 7f6da704c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb50acd920 con 0x55eb50af37f0 2024-05-09T10:50:32.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.356+0000 7f6da704c740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb50ae27b0 con 0x55eb50afead0 2024-05-09T10:50:32.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.356+0000 7f6da704c740 10 client.0 _ll_register_callbacks cb 0x55eb50ab1c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T10:50:32.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.356+0000 7f6da20eb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50ae1230 0x55eb50b0eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:32.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.356+0000 7f6da20eb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50ae1230 0x55eb50b0eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42544/0 (socket says 172.21.15.98:42544) 2024-05-09T10:50:32.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.356+0000 7f6da20eb640 1 -- 172.21.15.98:0/647952757 learned_addr learned my addr 172.21.15.98:0/647952757 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:32.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.356+0000 7f6da20eb640 1 -- 172.21.15.98:0/647952757 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55eb50afead0 msgr2=0x55eb50ac4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:32.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.356+0000 7f6da20eb640 1 --2- 172.21.15.98:0/647952757 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55eb50afead0 0x55eb50ac4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.357+0000 7f6da20eb640 1 -- 172.21.15.98:0/647952757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb50af37f0 msgr2=0x55eb50b0f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:32.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.357+0000 7f6da20eb640 1 --2- 172.21.15.98:0/647952757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb50af37f0 0x55eb50b0f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.357+0000 7f6da20eb640 1 -- 172.21.15.98:0/647952757 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb50afdc90 con 0x55eb50ae1230 2024-05-09T10:50:32.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.357+0000 7f6da20eb640 1 --2- 172.21.15.98:0/647952757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50ae1230 0x55eb50b0eb30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6d98078950 tx=0x7f6d980785b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d5db2680f7ba41 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:32.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.357+0000 7f6d7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-05-09T10:50:32.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.357+0000 7f6d7d7fa640 1 -- 172.21.15.98:0/647952757 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98095060 con 0x55eb50ae1230 2024-05-09T10:50:32.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.357+0000 7f6d7d7fa640 1 -- 172.21.15.98:0/647952757 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d98005a20 con 0x55eb50ae1230 2024-05-09T10:50:32.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.357+0000 7f6da704c740 1 -- 172.21.15.98:0/647952757 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d8c031070 con 0x55eb50ae1230 2024-05-09T10:50:32.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.357+0000 7f6d7d7fa640 1 -- 172.21.15.98:0/647952757 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d980a8050 con 0x55eb50ae1230 2024-05-09T10:50:32.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.357+0000 7f6da704c740 10 client.4689 fetch_fsmap learned FSMap version 6 2024-05-09T10:50:32.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.357+0000 7f6da704c740 1 -- 172.21.15.98:0/647952757 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55eb50ac5bb0 con 0x55eb50ae1230 2024-05-09T10:50:32.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.357+0000 7f6d7d7fa640 1 -- 172.21.15.98:0/647952757 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8c031070 con 0x55eb50ae1230 2024-05-09T10:50:32.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.357+0000 7f6d7d7fa640 1 -- 172.21.15.98:0/647952757 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6d98075030 con 0x55eb50ae1230 2024-05-09T10:50:32.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.357+0000 7f6da704c740 10 client.4689 fetch_fsmap finished waiting for FSMap version 6 2024-05-09T10:50:32.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.357+0000 7f6da704c740 10 client.4689 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-09T10:50:32.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.357+0000 7f6da704c740 20 client.4689 populate_metadata read hostname 'smithi098' 2024-05-09T10:50:32.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.357+0000 7f6da704c740 1 --2- 172.21.15.98:0/647952757 >> [v2:172.21.15.105:6833/1780062557,v1:172.21.15.105:6835/1780062557] conn(0x55eb50ab7550 0x55eb50ac5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.357+0000 7f6da704c740 4 client.4689 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T10:50:32.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.357+0000 7f6da704c740 1 -- 172.21.15.98:0/647952757 --> [v2:172.21.15.105:6833/1780062557,v1:172.21.15.105:6835/1780062557] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55eb50a26f50 con 0x55eb50ab7550 2024-05-09T10:50:32.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.357+0000 7f6da18ea640 1 --2- 172.21.15.98:0/647952757 >> [v2:172.21.15.105:6833/1780062557,v1:172.21.15.105:6835/1780062557] conn(0x55eb50ab7550 0x55eb50ac5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:32.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.358+0000 7f6d7d7fa640 10 client.4689 ms_handle_connect on v2:172.21.15.105:6833/1780062557 2024-05-09T10:50:32.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.358+0000 7f6da18ea640 1 --2- 172.21.15.98:0/647952757 >> [v2:172.21.15.105:6833/1780062557,v1:172.21.15.105:6835/1780062557] conn(0x55eb50ab7550 0x55eb50ac5360 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fbb5e62ad776ad3 server_cookie=4dd41e5e0adc379a in_seq=0 out_seq=0 2024-05-09T10:50:32.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.358+0000 7f6d7d7fa640 1 -- 172.21.15.98:0/647952757 <== mds.0 v2:172.21.15.105:6833/1780062557 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55eb50a26f50 con 0x55eb50ab7550 2024-05-09T10:50:32.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.358+0000 7f6d7d7fa640 10 client.4689 handle_command_reply: tid=0 2024-05-09T10:50:32.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.358+0000 7f6da704c740 1 client.4689 shutdown 2024-05-09T10:50:32.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.358+0000 7f6da704c740 1 -- 172.21.15.98:0/647952757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50ae1230 msgr2=0x55eb50b0eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:32.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.358+0000 7f6da704c740 1 --2- 172.21.15.98:0/647952757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50ae1230 0x55eb50b0eb30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6d98078950 tx=0x7f6d980785b0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.358+0000 7f6da704c740 1 -- 172.21.15.98:0/647952757 shutdown_connections 2024-05-09T10:50:32.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.358+0000 7f6da704c740 1 --2- 172.21.15.98:0/647952757 >> [v2:172.21.15.105:6833/1780062557,v1:172.21.15.105:6835/1780062557] conn(0x55eb50ab7550 0x55eb50ac5360 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.358+0000 7f6da704c740 1 --2- 172.21.15.98:0/647952757 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55eb50afead0 0x55eb50ac4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.358+0000 7f6d7d7fa640 0 client.4689 ms_handle_reset on v2:172.21.15.105:6833/1780062557 2024-05-09T10:50:32.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.358+0000 7f6da704c740 1 --2- 172.21.15.98:0/647952757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb50af37f0 0x55eb50b0f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.358+0000 7f6da704c740 1 --2- 172.21.15.98:0/647952757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb50ae1230 0x55eb50b0eb30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.358+0000 7f6da704c740 1 -- 172.21.15.98:0/647952757 >> 172.21.15.98:0/647952757 conn(0x55eb50ab6f10 msgr2=0x55eb50afc530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:32.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.358+0000 7f6da704c740 1 -- 172.21.15.98:0/647952757 shutdown_connections 2024-05-09T10:50:32.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.358+0000 7f6da704c740 1 -- 172.21.15.98:0/647952757 wait complete. 2024-05-09T10:50:32.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.358+0000 7f6da704c740 20 client.4689 trim_cache size 0 max 16384 2024-05-09T10:50:32.359 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-05-09T10:50:32.359 INFO:teuthology.orchestra.run.smithi098.stdout: "cluster_fsid": "3f705980-2043-4fe5-8639-5415c9d3c944", 2024-05-09T10:50:32.359 INFO:teuthology.orchestra.run.smithi098.stdout: "whoami": 0, 2024-05-09T10:50:32.359 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 4285, 2024-05-09T10:50:32.359 INFO:teuthology.orchestra.run.smithi098.stdout: "want_state": "up:active", 2024-05-09T10:50:32.359 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "up:active", 2024-05-09T10:50:32.359 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs", 2024-05-09T10:50:32.359 INFO:teuthology.orchestra.run.smithi098.stdout: "rank_uptime": 12.193373104999999, 2024-05-09T10:50:32.359 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsmap_epoch": 6, 2024-05-09T10:50:32.360 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch": 20, 2024-05-09T10:50:32.360 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch_barrier": 20, 2024-05-09T10:50:32.360 INFO:teuthology.orchestra.run.smithi098.stdout: "uptime": 19.420846115 2024-05-09T10:50:32.360 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-05-09T10:50:32.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.359+0000 7f6da434f640 1 -- 172.21.15.98:0/1480002007 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6d8006b4d0 msgr2=0x7f6d8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:32.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.359+0000 7f6da434f640 1 --2- 172.21.15.98:0/1480002007 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6d8006b4d0 0x7f6d8006d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c158320 tx=0x7f6d8c009040 comp rx=0 tx=0).stop 2024-05-09T10:50:32.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.359+0000 7f6da434f640 1 -- 172.21.15.98:0/1480002007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d88003b70 msgr2=0x7f6d9c149200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:32.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.359+0000 7f6da434f640 1 --2- 172.21.15.98:0/1480002007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d88003b70 0x7f6d9c149200 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800c190 tx=0x7f6d9c0b3cd0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.359+0000 7f6da434f640 1 -- 172.21.15.98:0/1480002007 shutdown_connections 2024-05-09T10:50:32.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.359+0000 7f6da434f640 1 --2- 172.21.15.98:0/1480002007 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6d8006b4d0 0x7f6d8006d990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.359+0000 7f6da434f640 1 --2- 172.21.15.98:0/1480002007 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6d9c156ba0 0x7f6d9c157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.359+0000 7f6da434f640 1 --2- 172.21.15.98:0/1480002007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d9c148510 0x7f6d9c148990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.359+0000 7f6da434f640 1 --2- 172.21.15.98:0/1480002007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d88003b70 0x7f6d9c149200 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.359+0000 7f6da434f640 1 -- 172.21.15.98:0/1480002007 >> 172.21.15.98:0/1480002007 conn(0x7f6d9c0b24c0 msgr2=0x7f6d9c15d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:32.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.359+0000 7f6da434f640 1 -- 172.21.15.98:0/1480002007 shutdown_connections 2024-05-09T10:50:32.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.359+0000 7f6da434f640 1 -- 172.21.15.98:0/1480002007 wait complete. 2024-05-09T10:50:32.363 INFO:teuthology.run_tasks:Running task kclient... 2024-05-09T10:50:32.391 INFO:tasks.kclient:Mounting kernel clients... 2024-05-09T10:50:32.391 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-09T10:50:32.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:32.392 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-05-09T10:50:32.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:32.409 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link delete ceph-brx 2024-05-09T10:50:32.474 INFO:teuthology.orchestra.run.smithi098.stderr:Cannot find device "ceph-brx" 2024-05-09T10:50:32.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T10:50:32.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:32.476 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-05-09T10:50:32.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:32.531 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link delete ceph-brx 2024-05-09T10:50:32.599 INFO:teuthology.orchestra.run.smithi098.stderr:Cannot find device "ceph-brx" 2024-05-09T10:50:32.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T10:50:32.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:32.601 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-05-09T10:50:32.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:32.621 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2024-05-09T10:50:32.693 INFO:teuthology.orchestra.run.smithi105.stderr:Cannot find device "ceph-brx" 2024-05-09T10:50:32.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T10:50:32.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:32.695 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-05-09T10:50:32.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:32.753 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2024-05-09T10:50:32.823 INFO:teuthology.orchestra.run.smithi105.stderr:Cannot find device "ceph-brx" 2024-05-09T10:50:32.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T10:50:32.824 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-05-09T10:50:32.825 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T10:50:32.825 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T10:50:32.825 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T10:50:32.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.932+0000 7f22959a9640 1 Processor -- start 2024-05-09T10:50:32.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.932+0000 7f22959a9640 1 -- start start 2024-05-09T10:50:32.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.933+0000 7f22959a9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f22900ae7c0 0x7f22900b0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.933+0000 7f22959a9640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22900aeda0 con 0x7f229015f4e0 2024-05-09T10:50:32.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.933+0000 7f22959a9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22900af080 con 0x7f22900b0a90 2024-05-09T10:50:32.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.933+0000 7f22959a9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22900af360 con 0x7f22900ae7c0 2024-05-09T10:50:32.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.933+0000 7f228effd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f229015f4e0 0x7f22900afe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56132/0 (socket says 172.21.15.98:56132) 2024-05-09T10:50:32.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.933+0000 7f228effd640 1 -- 172.21.15.98:0/1592271671 learned_addr learned my addr 172.21.15.98:0/1592271671 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:32.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.933+0000 7f22867fc640 1 --2- 172.21.15.98:0/1592271671 >> v2:172.21.15.105:3300/0 conn(0x7f22900ae7c0 0x7f22900b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:32.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.934+0000 7f228e7fc640 1 -- 172.21.15.98:0/1592271671 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3378557691 0 0) 0x7f22900aeda0 con 0x7f229015f4e0 2024-05-09T10:50:32.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.934+0000 7f228e7fc640 1 -- 172.21.15.98:0/1592271671 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f227c003520 con 0x7f229015f4e0 2024-05-09T10:50:32.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.934+0000 7f228e7fc640 1 -- 172.21.15.98:0/1592271671 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4123548399 0 0) 0x7f227c003520 con 0x7f229015f4e0 2024-05-09T10:50:32.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.934+0000 7f22867fc640 1 -- 172.21.15.98:0/1592271671 >> v1:172.21.15.105:6789/0 conn(0x7f22900b0a90 legacy=0x7f2290165c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:32.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.934+0000 7f22867fc640 1 -- 172.21.15.98:0/1592271671 >> v1:172.21.15.98:6789/0 conn(0x7f229015f4e0 legacy=0x7f22900afe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:32.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.934+0000 7f22867fc640 1 -- 172.21.15.98:0/1592271671 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22900af640 con 0x7f22900ae7c0 2024-05-09T10:50:32.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.935+0000 7f22867fc640 1 --2- 172.21.15.98:0/1592271671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22900ae7c0 0x7f22900b0550 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f22780097a0 tx=0x7f227802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4eec74a5d04365a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:32.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.935+0000 7f228e7fc640 1 -- 172.21.15.98:0/1592271671 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2278037070 con 0x7f22900ae7c0 2024-05-09T10:50:32.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.935+0000 7f228e7fc640 1 -- 172.21.15.98:0/1592271671 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227802abc0 con 0x7f22900ae7c0 2024-05-09T10:50:32.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.935+0000 7f228e7fc640 1 -- 172.21.15.98:0/1592271671 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2280001670 con 0x7f22900ae7c0 2024-05-09T10:50:32.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.935+0000 7f22959a9640 1 -- 172.21.15.98:0/1592271671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22900ae7c0 msgr2=0x7f22900b0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:32.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.935+0000 7f22959a9640 1 --2- 172.21.15.98:0/1592271671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22900ae7c0 0x7f22900b0550 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f22780097a0 tx=0x7f227802a130 comp rx=0 tx=0).stop 2024-05-09T10:50:32.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.935+0000 7f22959a9640 1 -- 172.21.15.98:0/1592271671 shutdown_connections 2024-05-09T10:50:32.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.935+0000 7f22959a9640 1 --2- 172.21.15.98:0/1592271671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22900ae7c0 0x7f22900b0550 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.935+0000 7f22959a9640 1 -- 172.21.15.98:0/1592271671 >> 172.21.15.98:0/1592271671 conn(0x7f22900b34e0 msgr2=0x7f22900b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:32.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.935+0000 7f22959a9640 1 -- 172.21.15.98:0/1592271671 shutdown_connections 2024-05-09T10:50:32.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.935+0000 7f22959a9640 1 -- 172.21.15.98:0/1592271671 wait complete. 2024-05-09T10:50:32.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.936+0000 7f22959a9640 1 Processor -- start 2024-05-09T10:50:32.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.937+0000 7f22959a9640 1 -- start start 2024-05-09T10:50:32.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.937+0000 7f22959a9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22900ae7c0 0x7f2290159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.937+0000 7f22959a9640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22900b0a90 0x7f22900c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.937+0000 7f22959a9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f229015f4e0 0x7f22901579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.937+0000 7f22959a9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229015f8c0 con 0x7f22900ae7c0 2024-05-09T10:50:32.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.937+0000 7f22959a9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22900cbcf0 con 0x7f229015f4e0 2024-05-09T10:50:32.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.937+0000 7f22959a9640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22900b0ea0 con 0x7f22900b0a90 2024-05-09T10:50:32.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.937+0000 7f228effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22900ae7c0 0x7f2290159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:32.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.937+0000 7f228effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22900ae7c0 0x7f2290159a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42558/0 (socket says 172.21.15.98:42558) 2024-05-09T10:50:32.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.937+0000 7f228effd640 1 -- 172.21.15.98:0/1500536362 learned_addr learned my addr 172.21.15.98:0/1500536362 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:32.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.937+0000 7f228f7fe640 1 --2- 172.21.15.98:0/1500536362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f229015f4e0 0x7f22901579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:32.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.937+0000 7f22867fc640 1 --2- 172.21.15.98:0/1500536362 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22900b0a90 0x7f22900c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:32.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.938+0000 7f228effd640 1 -- 172.21.15.98:0/1500536362 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22900b0a90 msgr2=0x7f22900c75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:32.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.938+0000 7f228effd640 1 --2- 172.21.15.98:0/1500536362 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22900b0a90 0x7f22900c75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.938+0000 7f228effd640 1 -- 172.21.15.98:0/1500536362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f229015f4e0 msgr2=0x7f22901579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:32.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.938+0000 7f228effd640 1 --2- 172.21.15.98:0/1500536362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f229015f4e0 0x7f22901579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:32.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.938+0000 7f228effd640 1 -- 172.21.15.98:0/1500536362 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2290158140 con 0x7f22900ae7c0 2024-05-09T10:50:32.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.938+0000 7f228effd640 1 --2- 172.21.15.98:0/1500536362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22900ae7c0 0x7f2290159a00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f228000c5b0 tx=0x7f228000ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:32.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.938+0000 7f228cff9640 1 -- 172.21.15.98:0/1500536362 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2280017070 con 0x7f22900ae7c0 2024-05-09T10:50:32.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.938+0000 7f228cff9640 1 -- 172.21.15.98:0/1500536362 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2280005ce0 con 0x7f22900ae7c0 2024-05-09T10:50:32.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.938+0000 7f228cff9640 1 -- 172.21.15.98:0/1500536362 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2280007330 con 0x7f22900ae7c0 2024-05-09T10:50:32.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.938+0000 7f22959a9640 1 -- 172.21.15.98:0/1500536362 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2290158420 con 0x7f22900ae7c0 2024-05-09T10:50:32.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.938+0000 7f22959a9640 1 -- 172.21.15.98:0/1500536362 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22901589e0 con 0x7f22900ae7c0 2024-05-09T10:50:32.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.939+0000 7f22959a9640 1 -- 172.21.15.98:0/1500536362 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f229006ffd0 con 0x7f22900ae7c0 2024-05-09T10:50:32.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.939+0000 7f228cff9640 1 -- 172.21.15.98:0/1500536362 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f228001a070 con 0x7f22900ae7c0 2024-05-09T10:50:32.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.940+0000 7f228cff9640 1 --2- 172.21.15.98:0/1500536362 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f227006b500 0x7f227006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:32.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.940+0000 7f228cff9640 1 -- 172.21.15.98:0/1500536362 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22800895f0 con 0x7f22900ae7c0 2024-05-09T10:50:32.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.940+0000 7f22867fc640 1 --2- 172.21.15.98:0/1500536362 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f227006b500 0x7f227006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:32.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.940+0000 7f22867fc640 1 --2- 172.21.15.98:0/1500536362 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f227006b500 0x7f227006d9c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2278034000 tx=0x7f227802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:32.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:32.944+0000 7f228cff9640 1 -- 172.21.15.98:0/1500536362 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2280057eb0 con 0x7f22900ae7c0 2024-05-09T10:50:33.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:33.149+0000 7f22959a9640 1 -- 172.21.15.98:0/1500536362 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f22900d12a0 con 0x7f22900ae7c0 2024-05-09T10:50:33.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:33.150+0000 7f228cff9640 1 -- 172.21.15.98:0/1500536362 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f228005bf50 con 0x7f22900ae7c0 2024-05-09T10:50:33.143 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T10:50:33.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:33.151+0000 7f22959a9640 1 -- 172.21.15.98:0/1500536362 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f227006b500 msgr2=0x7f227006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:33.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:33.151+0000 7f22959a9640 1 --2- 172.21.15.98:0/1500536362 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f227006b500 0x7f227006d9c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2278034000 tx=0x7f227802e000 comp rx=0 tx=0).stop 2024-05-09T10:50:33.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:33.151+0000 7f22959a9640 1 -- 172.21.15.98:0/1500536362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22900ae7c0 msgr2=0x7f2290159a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:33.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:33.151+0000 7f22959a9640 1 --2- 172.21.15.98:0/1500536362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22900ae7c0 0x7f2290159a00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f228000c5b0 tx=0x7f228000ca70 comp rx=0 tx=0).stop 2024-05-09T10:50:33.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:33.151+0000 7f22959a9640 1 -- 172.21.15.98:0/1500536362 shutdown_connections 2024-05-09T10:50:33.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:33.151+0000 7f22959a9640 1 --2- 172.21.15.98:0/1500536362 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f227006b500 0x7f227006d9c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:33.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:33.151+0000 7f22959a9640 1 --2- 172.21.15.98:0/1500536362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f229015f4e0 0x7f22901579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:33.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:33.151+0000 7f22959a9640 1 --2- 172.21.15.98:0/1500536362 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22900b0a90 0x7f22900c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:33.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:33.151+0000 7f22959a9640 1 --2- 172.21.15.98:0/1500536362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22900ae7c0 0x7f2290159a00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:33.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:33.151+0000 7f22959a9640 1 -- 172.21.15.98:0/1500536362 >> 172.21.15.98:0/1500536362 conn(0x7f22900b34e0 msgr2=0x7f22900b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:33.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:33.152+0000 7f22959a9640 1 -- 172.21.15.98:0/1500536362 shutdown_connections 2024-05-09T10:50:33.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:33.152+0000 7f22959a9640 1 -- 172.21.15.98:0/1500536362 wait complete. 2024-05-09T10:50:33.157 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T10:50:33.157 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T10:50:33.157 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2024-05-09T10:50:33.157 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T10:50:33.157 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T10:50:33.158 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T10:50:33.158 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T10:50:33.158 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T10:50:33.158 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T10:50:33.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:33.158 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2024-05-09T10:50:33.174 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T10:50:33.174 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T10:50:33.174 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T10:50:33.174 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:33.174 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-05-09T10:50:33.174 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:33.174 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T10:50:33.174 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2024-05-09T10:50:33.174 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2024-05-09T10:50:33.175 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T10:50:33.175 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2024-05-09T10:50:33.175 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2024-05-09T10:50:33.175 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T10:50:33.175 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2024-05-09T10:50:33.175 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T10:50:33.175 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2024-05-09T10:50:33.175 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-09T10:50:33.175 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 42777sec preferred_lft 42777sec 2024-05-09T10:50:33.175 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link 2024-05-09T10:50:33.175 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:33.176 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T10:50:33.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:33.176 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2024-05-09T10:50:33.176 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-05-09T10:50:33.176 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link add name ceph-brx type bridge 2024-05-09T10:50:33.176 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip addr flush dev ceph-brx 2024-05-09T10:50:33.176 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set ceph-brx up 2024-05-09T10:50:33.176 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T10:50:33.176 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-05-09T10:50:33.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:33.337 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T10:50:33.361 INFO:teuthology.orchestra.run.smithi098.stdout:1 2024-05-09T10:50:33.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:33.363 DEBUG:teuthology.orchestra.run.smithi098:> route 2024-05-09T10:50:33.426 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2024-05-09T10:50:33.427 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T10:50:33.427 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T10:50:33.427 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T10:50:33.427 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T10:50:33.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:33.427 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2024-05-09T10:50:33.427 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-05-09T10:50:33.427 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T10:50:33.427 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T10:50:33.428 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T10:50:33.428 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-05-09T10:50:33.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:33.693 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-05-09T10:50:33.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:33.709 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list-id 2024-05-09T10:50:33.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:33.765 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2024-05-09T10:50:33.765 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-05-09T10:50:33.765 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T10:50:33.765 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-09T10:50:33.765 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-05-09T10:50:33.844 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-09T10:50:33.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:33.844 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2024-05-09T10:50:33.845 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-05-09T10:50:33.845 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-09T10:50:33.845 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T10:50:33.845 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-09T10:50:33.845 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-09T10:50:33.845 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-09T10:50:33.845 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-05-09T10:50:33.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:33.979 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2024-05-09T10:50:33.979 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-05-09T10:50:33.979 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.0 up 2024-05-09T10:50:33.979 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T10:50:33.979 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-05-09T10:50:34.015 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T10:50:34.015 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T10:50:34.071 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T10:50:34.071 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T10:50:34.071 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T10:50:34.126 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-09T10:50:34.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:34.127 DEBUG:teuthology.orchestra.run.smithi098:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-09T10:50:34.214 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-09T10:50:34.214 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-09T10:50:34.214 INFO:teuthology.orchestra.run.smithi098.stdout:invalid new device string format 2024-05-09T10:50:34.334 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-09T10:50:34.334 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-09T10:50:34.334 INFO:teuthology.orchestra.run.smithi098.stdout:invalid new device string format 2024-05-09T10:50:34.334 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.105:3300,172.21.15.105:3301" 2024-05-09T10:50:34.334 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-09T10:50:34.340 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-09T10:50:34.340 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-09T10:50:34.340 INFO:teuthology.orchestra.run.smithi098.stdout:invalid new device string format 2024-05-09T10:50:34.340 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.105:3300,172.21.15.105:3301" 2024-05-09T10:50:34.340 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-09T10:50:34.340 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:6789,172.21.15.105:6789,172.21.15.105:6790" 2024-05-09T10:50:34.341 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.98:6789,172.21.15.105:6789,172.21.15.105:6790:/ 2024-05-09T10:50:34.341 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=3f705980-2043-4fe5-8639-5415c9d3c944" will pass to kernel 2024-05-09T10:50:34.341 INFO:teuthology.orchestra.run.smithi098.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-09T10:50:34.341 INFO:teuthology.orchestra.run.smithi098.stdout: You just mounted a file system that supports labels which does not 2024-05-09T10:50:34.341 INFO:teuthology.orchestra.run.smithi098.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-09T10:50:34.341 INFO:teuthology.orchestra.run.smithi098.stdout: applications will generate AVC messages and not be allowed access to 2024-05-09T10:50:34.341 INFO:teuthology.orchestra.run.smithi098.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-09T10:50:34.343 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-09T10:50:34.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:34.343 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T10:50:34.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:34.369 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T10:50:34.431 INFO:teuthology.orchestra.run.smithi098.stdout:client4698 2024-05-09T10:50:34.431 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:50:34.431 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:50:34.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:34.487 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T10:50:34.548 INFO:teuthology.orchestra.run.smithi098.stdout:client4698 2024-05-09T10:50:34.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:34.549 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T10:50:34.609 INFO:teuthology.orchestra.run.smithi098.stdout:client4698 2024-05-09T10:50:34.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:34.609 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/3f705980-2043-4fe5-8639-5415c9d3c944.client4698/status') 2024-05-09T10:50:34.677 INFO:teuthology.orchestra.run.smithi098.stdout:instance: client.4698 (3)192.168.0.1:0/1285563508 2024-05-09T10:50:34.677 INFO:teuthology.orchestra.run.smithi098.stdout:blocklisted: false 2024-05-09T10:50:34.677 INFO:teuthology.orchestra.run.smithi098.stderr:0+1 records in 2024-05-09T10:50:34.678 INFO:teuthology.orchestra.run.smithi098.stderr:0+1 records out 2024-05-09T10:50:34.678 INFO:teuthology.orchestra.run.smithi098.stderr:69 bytes copied, 2.2426e-05 s, 3.1 MB/s 2024-05-09T10:50:34.679 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-05-09T10:50:34.679 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T10:50:34.679 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-09T10:50:34.679 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T10:50:34.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.825+0000 7f899361f640 1 Processor -- start 2024-05-09T10:50:34.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.825+0000 7f899361f640 1 -- start start 2024-05-09T10:50:34.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.826+0000 7f899361f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f898c143740 0x7f898c0720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:34.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.826+0000 7f899361f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f898c1443e0 0x7f898c15e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:34.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.826+0000 7f899361f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f898c143b20 con 0x7f898c143d70 2024-05-09T10:50:34.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.826+0000 7f899361f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f898c15f460 con 0x7f898c1443e0 2024-05-09T10:50:34.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.826+0000 7f899361f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f898c0b28c0 con 0x7f898c143740 2024-05-09T10:50:34.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.826+0000 7f8990bba640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f898c1443e0 0x7f898c15e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:34.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.826+0000 7f8991bbc640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f898c143d70 0x7f898c15ec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38054/0 (socket says 172.21.15.98:38054) 2024-05-09T10:50:34.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.826+0000 7f8990bba640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f898c1443e0 0x7f898c15e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48028/0 (socket says 172.21.15.98:48028) 2024-05-09T10:50:34.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.826+0000 7f8991bbc640 1 -- 172.21.15.98:0/1566591798 learned_addr learned my addr 172.21.15.98:0/1566591798 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:34.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.826+0000 7f8990bba640 1 -- 172.21.15.98:0/1566591798 >> v2:172.21.15.105:3300/0 conn(0x7f898c143740 msgr2=0x7f898c0720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:34.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.826+0000 7f89913bb640 1 --2- 172.21.15.98:0/1566591798 >> v2:172.21.15.105:3300/0 conn(0x7f898c143740 0x7f898c0720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:34.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.826+0000 7f8990bba640 1 --2- 172.21.15.98:0/1566591798 >> v2:172.21.15.105:3300/0 conn(0x7f898c143740 0x7f898c0720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:34.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.826+0000 7f8983fff640 1 -- 172.21.15.98:0/1566591798 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1189714163 0 0) 0x7f898c143b20 con 0x7f898c143d70 2024-05-09T10:50:34.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.826+0000 7f8990bba640 1 -- 172.21.15.98:0/1566591798 >> v1:172.21.15.98:6789/0 conn(0x7f898c143d70 legacy=0x7f898c15ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:34.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.826+0000 7f8990bba640 1 -- 172.21.15.98:0/1566591798 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c0b2ba0 con 0x7f898c1443e0 2024-05-09T10:50:34.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.826+0000 7f89913bb640 1 --2- 172.21.15.98:0/1566591798 >> v2:172.21.15.105:3300/0 conn(0x7f898c143740 0x7f898c0720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:34.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.827+0000 7f8990bba640 1 --2- 172.21.15.98:0/1566591798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f898c1443e0 0x7f898c15e6e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f898c06e470 tx=0x7f897c02c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0221ac2dd9733c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:34.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.827+0000 7f8983fff640 1 -- 172.21.15.98:0/1566591798 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c032770 con 0x7f898c1443e0 2024-05-09T10:50:34.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.827+0000 7f8983fff640 1 -- 172.21.15.98:0/1566591798 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897c038070 con 0x7f898c1443e0 2024-05-09T10:50:34.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.827+0000 7f8983fff640 1 -- 172.21.15.98:0/1566591798 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c032b30 con 0x7f898c1443e0 2024-05-09T10:50:34.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.827+0000 7f899361f640 1 -- 172.21.15.98:0/1566591798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f898c1443e0 msgr2=0x7f898c15e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:34.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.827+0000 7f899361f640 1 --2- 172.21.15.98:0/1566591798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f898c1443e0 0x7f898c15e6e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f898c06e470 tx=0x7f897c02c870 comp rx=0 tx=0).stop 2024-05-09T10:50:34.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.828+0000 7f899361f640 1 -- 172.21.15.98:0/1566591798 shutdown_connections 2024-05-09T10:50:34.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.828+0000 7f899361f640 1 --2- 172.21.15.98:0/1566591798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f898c1443e0 0x7f898c15e6e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:34.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.828+0000 7f899361f640 1 --2- 172.21.15.98:0/1566591798 >> v2:172.21.15.105:3300/0 conn(0x7f898c143740 0x7f898c0720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:34.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.828+0000 7f899361f640 1 -- 172.21.15.98:0/1566591798 >> 172.21.15.98:0/1566591798 conn(0x7f898c06abe0 msgr2=0x7f898c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:34.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.828+0000 7f899361f640 1 -- 172.21.15.98:0/1566591798 shutdown_connections 2024-05-09T10:50:34.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.828+0000 7f899361f640 1 -- 172.21.15.98:0/1566591798 wait complete. 2024-05-09T10:50:34.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.829+0000 7f899361f640 1 Processor -- start 2024-05-09T10:50:34.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.829+0000 7f899361f640 1 -- start start 2024-05-09T10:50:34.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.830+0000 7f899361f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f898c143740 0x7f898c13dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:34.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.830+0000 7f899361f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f898c143d70 0x7f898c1ef9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:34.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.830+0000 7f899361f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f898c1443e0 0x7f898c15b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:34.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.830+0000 7f899361f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c0cc570 con 0x7f898c143740 2024-05-09T10:50:34.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.830+0000 7f899361f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c0cc3f0 con 0x7f898c143d70 2024-05-09T10:50:34.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.830+0000 7f899361f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898c0cc6f0 con 0x7f898c1443e0 2024-05-09T10:50:34.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.830+0000 7f89913bb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f898c143740 0x7f898c13dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:34.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.830+0000 7f89913bb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f898c143740 0x7f898c13dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48030/0 (socket says 172.21.15.98:48030) 2024-05-09T10:50:34.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.830+0000 7f89913bb640 1 -- 172.21.15.98:0/3741463014 learned_addr learned my addr 172.21.15.98:0/3741463014 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:34.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.830+0000 7f8990bba640 1 --2- 172.21.15.98:0/3741463014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f898c143d70 0x7f898c1ef9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:34.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.830+0000 7f8991bbc640 1 --2- 172.21.15.98:0/3741463014 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f898c1443e0 0x7f898c15b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:34.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.830+0000 7f89913bb640 1 -- 172.21.15.98:0/3741463014 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f898c1443e0 msgr2=0x7f898c15b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:34.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.830+0000 7f89913bb640 1 --2- 172.21.15.98:0/3741463014 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f898c1443e0 0x7f898c15b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:34.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.830+0000 7f89913bb640 1 -- 172.21.15.98:0/3741463014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f898c143d70 msgr2=0x7f898c1ef9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:34.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.830+0000 7f89913bb640 1 --2- 172.21.15.98:0/3741463014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f898c143d70 0x7f898c1ef9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:34.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.830+0000 7f89913bb640 1 -- 172.21.15.98:0/3741463014 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c15c2b0 con 0x7f898c143740 2024-05-09T10:50:34.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.830+0000 7f8990bba640 1 --2- 172.21.15.98:0/3741463014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f898c143d70 0x7f898c1ef9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:34.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.830+0000 7f8991bbc640 1 --2- 172.21.15.98:0/3741463014 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f898c1443e0 0x7f898c15b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:34.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.830+0000 7f89913bb640 1 --2- 172.21.15.98:0/3741463014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f898c143740 0x7f898c13dab0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f898800c820 tx=0x7f898800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:34.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.830+0000 7f8981ffb640 1 -- 172.21.15.98:0/3741463014 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8988015070 con 0x7f898c143740 2024-05-09T10:50:34.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.830+0000 7f8981ffb640 1 -- 172.21.15.98:0/3741463014 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89880079d0 con 0x7f898c143740 2024-05-09T10:50:34.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.830+0000 7f8981ffb640 1 -- 172.21.15.98:0/3741463014 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8988007da0 con 0x7f898c143740 2024-05-09T10:50:34.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.830+0000 7f899361f640 1 -- 172.21.15.98:0/3741463014 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898c15c590 con 0x7f898c143740 2024-05-09T10:50:34.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.831+0000 7f899361f640 1 -- 172.21.15.98:0/3741463014 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898c15cb50 con 0x7f898c143740 2024-05-09T10:50:34.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.831+0000 7f899361f640 1 -- 172.21.15.98:0/3741463014 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898c069cb0 con 0x7f898c143740 2024-05-09T10:50:34.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.832+0000 7f8981ffb640 1 -- 172.21.15.98:0/3741463014 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8988018070 con 0x7f898c143740 2024-05-09T10:50:34.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.832+0000 7f8981ffb640 1 --2- 172.21.15.98:0/3741463014 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f897006b4d0 0x7f897006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:34.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.832+0000 7f8981ffb640 1 -- 172.21.15.98:0/3741463014 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f898808a170 con 0x7f898c143740 2024-05-09T10:50:34.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.832+0000 7f8990bba640 1 --2- 172.21.15.98:0/3741463014 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f897006b4d0 0x7f897006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:34.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.833+0000 7f8990bba640 1 --2- 172.21.15.98:0/3741463014 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f897006b4d0 0x7f897006d990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f897c037000 tx=0x7f897c032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:34.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:34.836+0000 7f8981ffb640 1 -- 172.21.15.98:0/3741463014 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8988058ff0 con 0x7f898c143740 2024-05-09T10:50:35.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:35.043+0000 7f899361f640 1 -- 172.21.15.98:0/3741463014 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f898c06d8b0 con 0x7f898c143740 2024-05-09T10:50:35.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:35.044+0000 7f8981ffb640 1 -- 172.21.15.98:0/3741463014 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f89880588f0 con 0x7f898c143740 2024-05-09T10:50:35.037 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T10:50:35.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:35.045+0000 7f899361f640 1 -- 172.21.15.98:0/3741463014 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f897006b4d0 msgr2=0x7f897006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:35.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:35.045+0000 7f899361f640 1 --2- 172.21.15.98:0/3741463014 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f897006b4d0 0x7f897006d990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f897c037000 tx=0x7f897c032000 comp rx=0 tx=0).stop 2024-05-09T10:50:35.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:35.045+0000 7f899361f640 1 -- 172.21.15.98:0/3741463014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f898c143740 msgr2=0x7f898c13dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:35.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:35.045+0000 7f899361f640 1 --2- 172.21.15.98:0/3741463014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f898c143740 0x7f898c13dab0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f898800c820 tx=0x7f898800cc20 comp rx=0 tx=0).stop 2024-05-09T10:50:35.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:35.046+0000 7f899361f640 1 -- 172.21.15.98:0/3741463014 shutdown_connections 2024-05-09T10:50:35.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:35.046+0000 7f899361f640 1 --2- 172.21.15.98:0/3741463014 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f897006b4d0 0x7f897006d990 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:35.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:35.046+0000 7f899361f640 1 --2- 172.21.15.98:0/3741463014 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f898c1443e0 0x7f898c15b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:35.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:35.046+0000 7f899361f640 1 --2- 172.21.15.98:0/3741463014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f898c143d70 0x7f898c1ef9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:35.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:35.046+0000 7f899361f640 1 --2- 172.21.15.98:0/3741463014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f898c143740 0x7f898c13dab0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:35.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:35.046+0000 7f899361f640 1 -- 172.21.15.98:0/3741463014 >> 172.21.15.98:0/3741463014 conn(0x7f898c06abe0 msgr2=0x7f898c0b38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:35.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:35.046+0000 7f899361f640 1 -- 172.21.15.98:0/3741463014 shutdown_connections 2024-05-09T10:50:35.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:35.046+0000 7f899361f640 1 -- 172.21.15.98:0/3741463014 wait complete. 2024-05-09T10:50:35.049 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T10:50:35.050 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T10:50:35.050 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2024-05-09T10:50:35.050 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-09T10:50:35.050 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-09T10:50:35.050 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T10:50:35.050 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T10:50:35.050 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T10:50:35.050 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-09T10:50:35.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:35.050 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2024-05-09T10:50:35.066 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T10:50:35.066 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T10:50:35.066 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T10:50:35.066 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:35.066 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-05-09T10:50:35.066 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:35.066 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T10:50:35.066 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2024-05-09T10:50:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2024-05-09T10:50:35.067 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T10:50:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2024-05-09T10:50:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2024-05-09T10:50:35.067 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T10:50:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2024-05-09T10:50:35.067 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T10:50:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2024-05-09T10:50:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-09T10:50:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 42776sec preferred_lft 42776sec 2024-05-09T10:50:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link 2024-05-09T10:50:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:35.067 INFO:teuthology.orchestra.run.smithi098.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T10:50:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether ba:0c:be:30:aa:9c brd ff:ff:ff:ff:ff:ff 2024-05-09T10:50:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T10:50:35.068 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:35.068 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::f0ac:f3ff:fe82:9c89/64 scope link tentative 2024-05-09T10:50:35.068 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:35.068 INFO:teuthology.orchestra.run.smithi098.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T10:50:35.068 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether ba:0c:be:30:aa:9c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T10:50:35.068 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::b80c:beff:fe30:aa9c/64 scope link tentative 2024-05-09T10:50:35.068 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:35.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:35.068 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T10:50:35.129 INFO:teuthology.orchestra.run.smithi098.stdout:1 2024-05-09T10:50:35.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:35.131 DEBUG:teuthology.orchestra.run.smithi098:> route 2024-05-09T10:50:35.188 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2024-05-09T10:50:35.188 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T10:50:35.188 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T10:50:35.189 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T10:50:35.189 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T10:50:35.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:35.189 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2024-05-09T10:50:35.189 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-05-09T10:50:35.189 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T10:50:35.189 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T10:50:35.189 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T10:50:35.189 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-05-09T10:50:35.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:35.276 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-05-09T10:50:35.291 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T10:50:35.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:35.292 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list-id 2024-05-09T10:50:35.347 INFO:teuthology.orchestra.run.smithi098.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-09T10:50:35.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:35.348 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2024-05-09T10:50:35.348 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-05-09T10:50:35.348 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T10:50:35.348 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-09T10:50:35.348 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-05-09T10:50:35.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:35.426 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-09T10:50:35.494 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T10:50:35.494 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T10:50:35.494 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T10:50:35.494 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-05-09T10:50:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:35.495 INFO:teuthology.orchestra.run.smithi098.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T10:50:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 9a:d1:8e:3d:02:f0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-09T10:50:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-09T10:50:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::98d1:8eff:fe3d:2f0/64 scope link 2024-05-09T10:50:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:35.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:35.496 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-09T10:50:35.561 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T10:50:35.561 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T10:50:35.561 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T10:50:35.561 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:35.561 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-05-09T10:50:35.561 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:35.561 INFO:teuthology.orchestra.run.smithi098.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T10:50:35.562 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 9a:d1:8e:3d:02:f0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-09T10:50:35.562 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-09T10:50:35.562 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:35.562 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::98d1:8eff:fe3d:2f0/64 scope link 2024-05-09T10:50:35.562 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:35.563 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-09T10:50:35.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:35.563 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2024-05-09T10:50:35.563 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-05-09T10:50:35.563 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-09T10:50:35.563 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-09T10:50:35.563 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-09T10:50:35.563 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-09T10:50:35.564 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-09T10:50:35.564 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-05-09T10:50:35.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:35.676 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2024-05-09T10:50:35.676 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-05-09T10:50:35.676 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.1 up 2024-05-09T10:50:35.676 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set dev brx.1 master ceph-brx 2024-05-09T10:50:35.676 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-05-09T10:50:35.713 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T10:50:35.713 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-09T10:50:35.768 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-09T10:50:35.769 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T10:50:35.769 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-09T10:50:35.825 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-09T10:50:35.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:35.826 DEBUG:teuthology.orchestra.run.smithi098:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-09T10:50:35.908 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-09T10:50:35.908 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-09T10:50:35.908 INFO:teuthology.orchestra.run.smithi098.stdout:invalid new device string format 2024-05-09T10:50:35.916 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-09T10:50:35.916 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-09T10:50:35.916 INFO:teuthology.orchestra.run.smithi098.stdout:invalid new device string format 2024-05-09T10:50:35.917 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.105:3300,172.21.15.105:3301" 2024-05-09T10:50:35.917 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-09T10:50:35.922 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-09T10:50:35.922 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-09T10:50:35.922 INFO:teuthology.orchestra.run.smithi098.stdout:invalid new device string format 2024-05-09T10:50:35.922 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.105:3300,172.21.15.105:3301" 2024-05-09T10:50:35.922 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-09T10:50:35.922 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:6789,172.21.15.105:6789,172.21.15.105:6790" 2024-05-09T10:50:35.922 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.98:6789,172.21.15.105:6789,172.21.15.105:6790:/ 2024-05-09T10:50:35.922 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=3f705980-2043-4fe5-8639-5415c9d3c944" will pass to kernel 2024-05-09T10:50:35.923 INFO:teuthology.orchestra.run.smithi098.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-05-09T10:50:35.923 INFO:teuthology.orchestra.run.smithi098.stdout: You just mounted a file system that supports labels which does not 2024-05-09T10:50:35.923 INFO:teuthology.orchestra.run.smithi098.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-09T10:50:35.923 INFO:teuthology.orchestra.run.smithi098.stdout: applications will generate AVC messages and not be allowed access to 2024-05-09T10:50:35.923 INFO:teuthology.orchestra.run.smithi098.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-09T10:50:35.924 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-09T10:50:35.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:35.925 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-09T10:50:35.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:35.987 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T10:50:36.044 INFO:teuthology.orchestra.run.smithi098.stdout:client4582 2024-05-09T10:50:36.044 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:50:36.044 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:50:36.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:36.101 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T10:50:36.158 INFO:teuthology.orchestra.run.smithi098.stdout:client4582 2024-05-09T10:50:36.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:36.158 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T10:50:36.215 INFO:teuthology.orchestra.run.smithi098.stdout:client4582 2024-05-09T10:50:36.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:36.215 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/3f705980-2043-4fe5-8639-5415c9d3c944.client4582/status') 2024-05-09T10:50:36.279 INFO:teuthology.orchestra.run.smithi098.stdout:instance: client.4582 (3)172.21.15.98:0/3519978457 2024-05-09T10:50:36.279 INFO:teuthology.orchestra.run.smithi098.stdout:blocklisted: false 2024-05-09T10:50:36.280 INFO:teuthology.orchestra.run.smithi098.stderr:0+1 records in 2024-05-09T10:50:36.280 INFO:teuthology.orchestra.run.smithi098.stderr:0+1 records out 2024-05-09T10:50:36.280 INFO:teuthology.orchestra.run.smithi098.stderr:70 bytes copied, 2.2486e-05 s, 3.1 MB/s 2024-05-09T10:50:36.281 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-05-09T10:50:36.281 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T10:50:36.281 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-09T10:50:36.281 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T10:50:36.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.427+0000 7ff2aaae1640 1 Processor -- start 2024-05-09T10:50:36.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.427+0000 7ff2aaae1640 1 -- start start 2024-05-09T10:50:36.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.427+0000 7ff2aaae1640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff2a406ed40 0x7ff2a40af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:36.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.427+0000 7ff2aaae1640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a41553e0 con 0x7ff2a4155000 2024-05-09T10:50:36.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.427+0000 7ff2aaae1640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a40b3860 con 0x7ff2a4071b00 2024-05-09T10:50:36.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.427+0000 7ff2aaae1640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a40b4430 con 0x7ff2a406ed40 2024-05-09T10:50:36.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.428+0000 7ff2a907e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff2a406ed40 0x7ff2a40af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:36.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.428+0000 7ff2a887d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff2a4155000 0x7ff2a40713f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38060/0 (socket says 172.21.15.98:38060) 2024-05-09T10:50:36.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.428+0000 7ff2a907e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff2a406ed40 0x7ff2a40af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48032/0 (socket says 172.21.15.98:48032) 2024-05-09T10:50:36.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.428+0000 7ff2a887d640 1 -- 172.21.15.98:0/1401268886 learned_addr learned my addr 172.21.15.98:0/1401268886 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:36.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.428+0000 7ff2a907e640 1 -- 172.21.15.98:0/1401268886 >> v1:172.21.15.105:6789/0 conn(0x7ff2a4071b00 legacy=0x7ff2a406e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:36.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.428+0000 7ff2a907e640 1 -- 172.21.15.98:0/1401268886 >> v1:172.21.15.98:6789/0 conn(0x7ff2a4155000 legacy=0x7ff2a40713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:36.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.428+0000 7ff2a907e640 1 -- 172.21.15.98:0/1401268886 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a40b3b00 con 0x7ff2a406ed40 2024-05-09T10:50:36.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.428+0000 7ff2a37fe640 1 -- 172.21.15.98:0/1401268886 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1685479055 0 0) 0x7ff2a41553e0 con 0x7ff2a4155000 2024-05-09T10:50:36.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.429+0000 7ff2a907e640 1 --2- 172.21.15.98:0/1401268886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2a406ed40 0x7ff2a40af4d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff2a406d190 tx=0x7ff29c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8783ec02ade9d6c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:36.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.429+0000 7ff2a37fe640 1 -- 172.21.15.98:0/1401268886 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c02ae60 con 0x7ff2a406ed40 2024-05-09T10:50:36.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.429+0000 7ff2a37fe640 1 -- 172.21.15.98:0/1401268886 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c007740 con 0x7ff2a406ed40 2024-05-09T10:50:36.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.429+0000 7ff2a37fe640 1 -- 172.21.15.98:0/1401268886 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c007b10 con 0x7ff2a406ed40 2024-05-09T10:50:36.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.429+0000 7ff2aaae1640 1 -- 172.21.15.98:0/1401268886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2a406ed40 msgr2=0x7ff2a40af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:36.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.429+0000 7ff2aaae1640 1 --2- 172.21.15.98:0/1401268886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2a406ed40 0x7ff2a40af4d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff2a406d190 tx=0x7ff29c02a680 comp rx=0 tx=0).stop 2024-05-09T10:50:36.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.429+0000 7ff2aaae1640 1 -- 172.21.15.98:0/1401268886 shutdown_connections 2024-05-09T10:50:36.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.429+0000 7ff2aaae1640 1 --2- 172.21.15.98:0/1401268886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2a406ed40 0x7ff2a40af4d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:36.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.429+0000 7ff2aaae1640 1 -- 172.21.15.98:0/1401268886 >> 172.21.15.98:0/1401268886 conn(0x7ff2a40b24c0 msgr2=0x7ff2a40b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:36.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.429+0000 7ff2aaae1640 1 -- 172.21.15.98:0/1401268886 shutdown_connections 2024-05-09T10:50:36.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.429+0000 7ff2aaae1640 1 -- 172.21.15.98:0/1401268886 wait complete. 2024-05-09T10:50:36.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.430+0000 7ff2aaae1640 1 Processor -- start 2024-05-09T10:50:36.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.431+0000 7ff2aaae1640 1 -- start start 2024-05-09T10:50:36.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.431+0000 7ff2aaae1640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2a406ed40 0x7ff2a41481c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:36.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.431+0000 7ff2aaae1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2a4071b00 0x7ff2a4157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:36.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.431+0000 7ff2aaae1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2a4155000 0x7ff2a40cf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:36.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.431+0000 7ff2aaae1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a415bdd0 con 0x7ff2a4155000 2024-05-09T10:50:36.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.431+0000 7ff2aaae1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a40b40c0 con 0x7ff2a4071b00 2024-05-09T10:50:36.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.431+0000 7ff2aaae1640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a4158b60 con 0x7ff2a406ed40 2024-05-09T10:50:36.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.431+0000 7ff2a907e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2a4155000 0x7ff2a40cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:36.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.432+0000 7ff2a907e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2a4155000 0x7ff2a40cf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48048/0 (socket says 172.21.15.98:48048) 2024-05-09T10:50:36.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.432+0000 7ff2a907e640 1 -- 172.21.15.98:0/3337776616 learned_addr learned my addr 172.21.15.98:0/3337776616 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:36.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.432+0000 7ff2a3fff640 1 --2- 172.21.15.98:0/3337776616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2a4071b00 0x7ff2a4157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:36.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.432+0000 7ff2a887d640 1 --2- 172.21.15.98:0/3337776616 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2a406ed40 0x7ff2a41481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:36.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.432+0000 7ff2a907e640 1 -- 172.21.15.98:0/3337776616 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2a406ed40 msgr2=0x7ff2a41481c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:36.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.432+0000 7ff2a907e640 1 --2- 172.21.15.98:0/3337776616 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2a406ed40 0x7ff2a41481c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:36.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.432+0000 7ff2a907e640 1 -- 172.21.15.98:0/3337776616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2a4071b00 msgr2=0x7ff2a4157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:36.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.432+0000 7ff2a907e640 1 --2- 172.21.15.98:0/3337776616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2a4071b00 0x7ff2a4157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:36.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.432+0000 7ff2a907e640 1 -- 172.21.15.98:0/3337776616 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a415b3e0 con 0x7ff2a4155000 2024-05-09T10:50:36.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.432+0000 7ff2a887d640 1 --2- 172.21.15.98:0/3337776616 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2a406ed40 0x7ff2a41481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:36.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.432+0000 7ff2a3fff640 1 --2- 172.21.15.98:0/3337776616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2a4071b00 0x7ff2a4157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:36.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.432+0000 7ff2a907e640 1 --2- 172.21.15.98:0/3337776616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2a4155000 0x7ff2a40cf530 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff29c02ac70 tx=0x7ff29c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:36.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.432+0000 7ff2a17fa640 1 -- 172.21.15.98:0/3337776616 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c040070 con 0x7ff2a4155000 2024-05-09T10:50:36.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.432+0000 7ff2a17fa640 1 -- 172.21.15.98:0/3337776616 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c02ae60 con 0x7ff2a4155000 2024-05-09T10:50:36.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.432+0000 7ff2aaae1640 1 -- 172.21.15.98:0/3337776616 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a415a340 con 0x7ff2a4155000 2024-05-09T10:50:36.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.432+0000 7ff2a17fa640 1 -- 172.21.15.98:0/3337776616 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c045460 con 0x7ff2a4155000 2024-05-09T10:50:36.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.432+0000 7ff2aaae1640 1 -- 172.21.15.98:0/3337776616 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a415abe0 con 0x7ff2a4155000 2024-05-09T10:50:36.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.433+0000 7ff2aaae1640 1 -- 172.21.15.98:0/3337776616 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a415a730 con 0x7ff2a4155000 2024-05-09T10:50:36.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.434+0000 7ff2a17fa640 1 -- 172.21.15.98:0/3337776616 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff29c039040 con 0x7ff2a4155000 2024-05-09T10:50:36.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.434+0000 7ff2a17fa640 1 --2- 172.21.15.98:0/3337776616 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff28806b4d0 0x7ff28806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:36.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.434+0000 7ff2a887d640 1 --2- 172.21.15.98:0/3337776616 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff28806b4d0 0x7ff28806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:36.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.434+0000 7ff2a17fa640 1 -- 172.21.15.98:0/3337776616 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff29c0ab9e0 con 0x7ff2a4155000 2024-05-09T10:50:36.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.435+0000 7ff2a887d640 1 --2- 172.21.15.98:0/3337776616 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff28806b4d0 0x7ff28806d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff298000f60 tx=0x7ff298009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:36.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.439+0000 7ff2a17fa640 1 -- 172.21.15.98:0/3337776616 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff29c07a2d0 con 0x7ff2a4155000 2024-05-09T10:50:36.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.644+0000 7ff2aaae1640 1 -- 172.21.15.98:0/3337776616 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff2a4069cb0 con 0x7ff2a4155000 2024-05-09T10:50:36.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.645+0000 7ff2a17fa640 1 -- 172.21.15.98:0/3337776616 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff29c07e370 con 0x7ff2a4155000 2024-05-09T10:50:36.638 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T10:50:36.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.646+0000 7ff2aaae1640 1 -- 172.21.15.98:0/3337776616 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff28806b4d0 msgr2=0x7ff28806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:36.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.646+0000 7ff2aaae1640 1 --2- 172.21.15.98:0/3337776616 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff28806b4d0 0x7ff28806d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff298000f60 tx=0x7ff298009040 comp rx=0 tx=0).stop 2024-05-09T10:50:36.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.647+0000 7ff2aaae1640 1 -- 172.21.15.98:0/3337776616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2a4155000 msgr2=0x7ff2a40cf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:36.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.647+0000 7ff2aaae1640 1 --2- 172.21.15.98:0/3337776616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2a4155000 0x7ff2a40cf530 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff29c02ac70 tx=0x7ff29c004a40 comp rx=0 tx=0).stop 2024-05-09T10:50:36.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.647+0000 7ff2aaae1640 1 -- 172.21.15.98:0/3337776616 shutdown_connections 2024-05-09T10:50:36.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.647+0000 7ff2aaae1640 1 --2- 172.21.15.98:0/3337776616 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff28806b4d0 0x7ff28806d990 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:36.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.647+0000 7ff2aaae1640 1 --2- 172.21.15.98:0/3337776616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2a4155000 0x7ff2a40cf530 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:36.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.647+0000 7ff2aaae1640 1 --2- 172.21.15.98:0/3337776616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2a4071b00 0x7ff2a4157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:36.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.647+0000 7ff2aaae1640 1 --2- 172.21.15.98:0/3337776616 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2a406ed40 0x7ff2a41481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:36.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.647+0000 7ff2aaae1640 1 -- 172.21.15.98:0/3337776616 >> 172.21.15.98:0/3337776616 conn(0x7ff2a40b24c0 msgr2=0x7ff2a415bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:36.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.647+0000 7ff2aaae1640 1 -- 172.21.15.98:0/3337776616 shutdown_connections 2024-05-09T10:50:36.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:36.647+0000 7ff2aaae1640 1 -- 172.21.15.98:0/3337776616 wait complete. 2024-05-09T10:50:36.651 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T10:50:36.651 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T10:50:36.651 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2024-05-09T10:50:36.651 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-09T10:50:36.651 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-09T10:50:36.651 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T10:50:36.651 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T10:50:36.651 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T10:50:36.651 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-09T10:50:36.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:36.651 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2024-05-09T10:50:36.671 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T10:50:36.671 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T10:50:36.671 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T10:50:36.671 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:36.671 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-05-09T10:50:36.671 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:36.672 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T10:50:36.672 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2024-05-09T10:50:36.672 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2024-05-09T10:50:36.672 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T10:50:36.672 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2024-05-09T10:50:36.672 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2024-05-09T10:50:36.672 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T10:50:36.672 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2024-05-09T10:50:36.672 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T10:50:36.672 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2024-05-09T10:50:36.672 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-09T10:50:36.672 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42769sec preferred_lft 42769sec 2024-05-09T10:50:36.672 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2024-05-09T10:50:36.672 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:36.673 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T10:50:36.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:36.673 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-05-09T10:50:36.673 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-05-09T10:50:36.673 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add name ceph-brx type bridge 2024-05-09T10:50:36.673 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr flush dev ceph-brx 2024-05-09T10:50:36.673 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set ceph-brx up 2024-05-09T10:50:36.673 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T10:50:36.673 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-05-09T10:50:36.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:36.829 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T10:50:36.854 INFO:teuthology.orchestra.run.smithi105.stdout:1 2024-05-09T10:50:36.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:36.858 DEBUG:teuthology.orchestra.run.smithi105:> route 2024-05-09T10:50:36.927 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2024-05-09T10:50:36.927 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T10:50:36.927 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T10:50:36.928 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T10:50:36.928 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T10:50:36.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:36.928 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-05-09T10:50:36.928 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-05-09T10:50:36.928 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T10:50:36.928 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T10:50:36.928 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T10:50:36.928 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-05-09T10:50:37.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:37.196 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-05-09T10:50:37.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:37.211 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list-id 2024-05-09T10:50:37.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:37.267 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-05-09T10:50:37.267 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-05-09T10:50:37.267 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T10:50:37.267 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-09T10:50:37.267 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-05-09T10:50:37.346 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-09T10:50:37.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:37.347 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-05-09T10:50:37.347 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-05-09T10:50:37.347 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-09T10:50:37.347 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T10:50:37.347 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-09T10:50:37.347 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-09T10:50:37.347 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-09T10:50:37.347 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-05-09T10:50:37.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:37.476 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-05-09T10:50:37.476 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-05-09T10:50:37.476 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2024-05-09T10:50:37.476 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T10:50:37.476 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-05-09T10:50:37.513 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T10:50:37.513 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-09T10:50:37.568 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-09T10:50:37.569 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T10:50:37.569 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-09T10:50:37.624 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-09T10:50:37.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:37.625 DEBUG:teuthology.orchestra.run.smithi105:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-09T10:50:37.727 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-09T10:50:37.727 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-09T10:50:37.727 INFO:teuthology.orchestra.run.smithi105.stdout:invalid new device string format 2024-05-09T10:50:37.845 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-09T10:50:37.846 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-09T10:50:37.846 INFO:teuthology.orchestra.run.smithi105.stdout:invalid new device string format 2024-05-09T10:50:37.846 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.105:3300,172.21.15.105:3301" 2024-05-09T10:50:37.846 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-09T10:50:37.851 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-09T10:50:37.851 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-09T10:50:37.851 INFO:teuthology.orchestra.run.smithi105.stdout:invalid new device string format 2024-05-09T10:50:37.851 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.105:3300,172.21.15.105:3301" 2024-05-09T10:50:37.851 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-09T10:50:37.851 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: resolved to: "172.21.15.98:6789,172.21.15.105:6789,172.21.15.105:6790" 2024-05-09T10:50:37.851 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.98:6789,172.21.15.105:6789,172.21.15.105:6790:/ 2024-05-09T10:50:37.851 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=3f705980-2043-4fe5-8639-5415c9d3c944" will pass to kernel 2024-05-09T10:50:37.852 INFO:teuthology.orchestra.run.smithi105.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-09T10:50:37.852 INFO:teuthology.orchestra.run.smithi105.stdout: You just mounted a file system that supports labels which does not 2024-05-09T10:50:37.852 INFO:teuthology.orchestra.run.smithi105.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-09T10:50:37.852 INFO:teuthology.orchestra.run.smithi105.stdout: applications will generate AVC messages and not be allowed access to 2024-05-09T10:50:37.852 INFO:teuthology.orchestra.run.smithi105.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-09T10:50:37.853 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-09T10:50:37.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:37.853 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-09T10:50:37.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:37.878 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T10:50:37.935 INFO:teuthology.orchestra.run.smithi105.stdout:client4475 2024-05-09T10:50:37.935 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:50:37.935 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:50:37.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:37.991 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T10:50:38.047 INFO:teuthology.orchestra.run.smithi105.stdout:client4475 2024-05-09T10:50:38.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:38.048 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T10:50:38.104 INFO:teuthology.orchestra.run.smithi105.stdout:client4475 2024-05-09T10:50:38.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:38.105 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/3f705980-2043-4fe5-8639-5415c9d3c944.client4475/status') 2024-05-09T10:50:38.168 INFO:teuthology.orchestra.run.smithi105.stdout:instance: client.4475 (3)192.168.0.1:0/3414939301 2024-05-09T10:50:38.168 INFO:teuthology.orchestra.run.smithi105.stdout:blocklisted: false 2024-05-09T10:50:38.169 INFO:teuthology.orchestra.run.smithi105.stderr:0+1 records in 2024-05-09T10:50:38.169 INFO:teuthology.orchestra.run.smithi105.stderr:0+1 records out 2024-05-09T10:50:38.169 INFO:teuthology.orchestra.run.smithi105.stderr:69 bytes copied, 2.092e-05 s, 3.3 MB/s 2024-05-09T10:50:38.170 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-05-09T10:50:38.170 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T10:50:38.170 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-09T10:50:38.170 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T10:50:38.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.289+0000 7f68dba76640 1 Processor -- start 2024-05-09T10:50:38.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.289+0000 7f68dba76640 1 -- start start 2024-05-09T10:50:38.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.290+0000 7f68dba76640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68d406b2f0 0x7f68d40d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:38.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.290+0000 7f68dba76640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f68d40af6e0 0x7f68d40cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:38.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.290+0000 7f68dba76640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d406f830 con 0x7f68d415b9e0 2024-05-09T10:50:38.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.290+0000 7f68dba76640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d406fdb0 con 0x7f68d40af6e0 2024-05-09T10:50:38.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.290+0000 7f68dba76640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d4160210 con 0x7f68d406b2f0 2024-05-09T10:50:38.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.290+0000 7f68da013640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f68d40af6e0 0x7f68d40cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:38.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.290+0000 7f68da013640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f68d40af6e0 0x7f68d40cef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48062/0 (socket says 172.21.15.98:48062) 2024-05-09T10:50:38.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.290+0000 7f68da013640 1 -- 172.21.15.98:0/2508466306 learned_addr learned my addr 172.21.15.98:0/2508466306 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:38.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.291+0000 7f68da013640 1 -- 172.21.15.98:0/2508466306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68d406b2f0 msgr2=0x7f68d40d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:38.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.291+0000 7f68d9011640 1 --2- 172.21.15.98:0/2508466306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68d406b2f0 0x7f68d40d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:38.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.291+0000 7f68da013640 1 --2- 172.21.15.98:0/2508466306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68d406b2f0 0x7f68d40d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:38.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.291+0000 7f68da013640 1 -- 172.21.15.98:0/2508466306 >> v1:172.21.15.105:6789/0 conn(0x7f68d415b9e0 legacy=0x7f68d406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:38.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.291+0000 7f68da013640 1 -- 172.21.15.98:0/2508466306 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d406fad0 con 0x7f68d40af6e0 2024-05-09T10:50:38.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.291+0000 7f68d9011640 1 --2- 172.21.15.98:0/2508466306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68d406b2f0 0x7f68d40d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:38.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.291+0000 7f68da013640 1 --2- 172.21.15.98:0/2508466306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68d40af6e0 0x7f68d40cef00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f68cc009e50 tx=0x7f68cc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ffb68060519d9c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:38.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.292+0000 7f68d8810640 1 -- 172.21.15.98:0/2508466306 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68cc036070 con 0x7f68d40af6e0 2024-05-09T10:50:38.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.292+0000 7f68d8810640 1 -- 172.21.15.98:0/2508466306 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68cc031600 con 0x7f68d40af6e0 2024-05-09T10:50:38.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.292+0000 7f68d8810640 1 -- 172.21.15.98:0/2508466306 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68cc0319d0 con 0x7f68d40af6e0 2024-05-09T10:50:38.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.292+0000 7f68dba76640 1 -- 172.21.15.98:0/2508466306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68d40af6e0 msgr2=0x7f68d40cef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:38.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.292+0000 7f68dba76640 1 --2- 172.21.15.98:0/2508466306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68d40af6e0 0x7f68d40cef00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f68cc009e50 tx=0x7f68cc02a710 comp rx=0 tx=0).stop 2024-05-09T10:50:38.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.292+0000 7f68dba76640 1 -- 172.21.15.98:0/2508466306 shutdown_connections 2024-05-09T10:50:38.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.292+0000 7f68dba76640 1 --2- 172.21.15.98:0/2508466306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68d406b2f0 0x7f68d40d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:38.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.292+0000 7f68dba76640 1 --2- 172.21.15.98:0/2508466306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68d40af6e0 0x7f68d40cef00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:38.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.292+0000 7f68dba76640 1 -- 172.21.15.98:0/2508466306 >> 172.21.15.98:0/2508466306 conn(0x7f68d40b24c0 msgr2=0x7f68d40b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:38.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.292+0000 7f68dba76640 1 -- 172.21.15.98:0/2508466306 shutdown_connections 2024-05-09T10:50:38.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.292+0000 7f68dba76640 1 -- 172.21.15.98:0/2508466306 wait complete. 2024-05-09T10:50:38.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.293+0000 7f68dba76640 1 Processor -- start 2024-05-09T10:50:38.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.293+0000 7f68dba76640 1 -- start start 2024-05-09T10:50:38.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.294+0000 7f68dba76640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68d406b2f0 0x7f68d4147470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:38.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.294+0000 7f68dba76640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68d40af6e0 0x7f68d415c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:38.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.294+0000 7f68dba76640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68d415b9e0 0x7f68d415c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:38.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.294+0000 7f68dba76640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d406d7c0 con 0x7f68d415b9e0 2024-05-09T10:50:38.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.294+0000 7f68dba76640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d406d640 con 0x7f68d40af6e0 2024-05-09T10:50:38.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.294+0000 7f68dba76640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d415bdc0 con 0x7f68d406b2f0 2024-05-09T10:50:38.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.294+0000 7f68da013640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68d415b9e0 0x7f68d415c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:38.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.294+0000 7f68da013640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68d415b9e0 0x7f68d415c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48068/0 (socket says 172.21.15.98:48068) 2024-05-09T10:50:38.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.294+0000 7f68da013640 1 -- 172.21.15.98:0/2729085907 learned_addr learned my addr 172.21.15.98:0/2729085907 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:38.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.294+0000 7f68d9812640 1 --2- 172.21.15.98:0/2729085907 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68d406b2f0 0x7f68d4147470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:38.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.294+0000 7f68d9011640 1 --2- 172.21.15.98:0/2729085907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68d40af6e0 0x7f68d415c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:38.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.295+0000 7f68da013640 1 -- 172.21.15.98:0/2729085907 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68d406b2f0 msgr2=0x7f68d4147470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:38.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.295+0000 7f68da013640 1 --2- 172.21.15.98:0/2729085907 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68d406b2f0 0x7f68d4147470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:38.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.295+0000 7f68da013640 1 -- 172.21.15.98:0/2729085907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68d40af6e0 msgr2=0x7f68d415c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:38.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.295+0000 7f68da013640 1 --2- 172.21.15.98:0/2729085907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68d40af6e0 0x7f68d415c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:38.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.295+0000 7f68da013640 1 -- 172.21.15.98:0/2729085907 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d415ccd0 con 0x7f68d415b9e0 2024-05-09T10:50:38.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.295+0000 7f68d9011640 1 --2- 172.21.15.98:0/2729085907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68d40af6e0 0x7f68d415c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:38.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.295+0000 7f68d9812640 1 --2- 172.21.15.98:0/2729085907 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68d406b2f0 0x7f68d4147470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:38.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.295+0000 7f68da013640 1 --2- 172.21.15.98:0/2729085907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68d415b9e0 0x7f68d415c6a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f68cc034040 tx=0x7f68cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:38.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.295+0000 7f68ca7fc640 1 -- 172.21.15.98:0/2729085907 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68cc036050 con 0x7f68d415b9e0 2024-05-09T10:50:38.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.295+0000 7f68ca7fc640 1 -- 172.21.15.98:0/2729085907 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68cc042070 con 0x7f68d415b9e0 2024-05-09T10:50:38.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.295+0000 7f68dba76640 1 -- 172.21.15.98:0/2729085907 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d415cfb0 con 0x7f68d415b9e0 2024-05-09T10:50:38.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.295+0000 7f68ca7fc640 1 -- 172.21.15.98:0/2729085907 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68cc009510 con 0x7f68d415b9e0 2024-05-09T10:50:38.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.295+0000 7f68dba76640 1 -- 172.21.15.98:0/2729085907 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d414f230 con 0x7f68d415b9e0 2024-05-09T10:50:38.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.296+0000 7f68dba76640 1 -- 172.21.15.98:0/2729085907 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68d4069cb0 con 0x7f68d415b9e0 2024-05-09T10:50:38.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.296+0000 7f68ca7fc640 1 -- 172.21.15.98:0/2729085907 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f68cc007bd0 con 0x7f68d415b9e0 2024-05-09T10:50:38.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.297+0000 7f68ca7fc640 1 --2- 172.21.15.98:0/2729085907 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f68b806b4d0 0x7f68b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:38.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.297+0000 7f68ca7fc640 1 -- 172.21.15.98:0/2729085907 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68cc0ab0c0 con 0x7f68d415b9e0 2024-05-09T10:50:38.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.297+0000 7f68d9812640 1 --2- 172.21.15.98:0/2729085907 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f68b806b4d0 0x7f68b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:38.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.297+0000 7f68d9812640 1 --2- 172.21.15.98:0/2729085907 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f68b806b4d0 0x7f68b806d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f68d0007fd0 tx=0x7f68d0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:38.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.301+0000 7f68ca7fc640 1 -- 172.21.15.98:0/2729085907 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f68cc045360 con 0x7f68d415b9e0 2024-05-09T10:50:38.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.510+0000 7f68dba76640 1 -- 172.21.15.98:0/2729085907 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f68d414e2e0 con 0x7f68d415b9e0 2024-05-09T10:50:38.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.510+0000 7f68ca7fc640 1 -- 172.21.15.98:0/2729085907 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f68cc079fd0 con 0x7f68d415b9e0 2024-05-09T10:50:38.504 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T10:50:38.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.512+0000 7f68dba76640 1 -- 172.21.15.98:0/2729085907 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f68b806b4d0 msgr2=0x7f68b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:38.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.512+0000 7f68dba76640 1 --2- 172.21.15.98:0/2729085907 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f68b806b4d0 0x7f68b806d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f68d0007fd0 tx=0x7f68d0009040 comp rx=0 tx=0).stop 2024-05-09T10:50:38.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.512+0000 7f68dba76640 1 -- 172.21.15.98:0/2729085907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68d415b9e0 msgr2=0x7f68d415c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:38.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.512+0000 7f68dba76640 1 --2- 172.21.15.98:0/2729085907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68d415b9e0 0x7f68d415c6a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f68cc034040 tx=0x7f68cc004a40 comp rx=0 tx=0).stop 2024-05-09T10:50:38.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.512+0000 7f68dba76640 1 -- 172.21.15.98:0/2729085907 shutdown_connections 2024-05-09T10:50:38.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.512+0000 7f68dba76640 1 --2- 172.21.15.98:0/2729085907 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f68b806b4d0 0x7f68b806d990 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:38.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.512+0000 7f68dba76640 1 --2- 172.21.15.98:0/2729085907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68d415b9e0 0x7f68d415c6a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:38.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.512+0000 7f68dba76640 1 --2- 172.21.15.98:0/2729085907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68d40af6e0 0x7f68d415c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:38.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.512+0000 7f68dba76640 1 --2- 172.21.15.98:0/2729085907 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f68d406b2f0 0x7f68d4147470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:38.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.512+0000 7f68dba76640 1 -- 172.21.15.98:0/2729085907 >> 172.21.15.98:0/2729085907 conn(0x7f68d40b24c0 msgr2=0x7f68d414dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:38.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.512+0000 7f68dba76640 1 -- 172.21.15.98:0/2729085907 shutdown_connections 2024-05-09T10:50:38.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:38.512+0000 7f68dba76640 1 -- 172.21.15.98:0/2729085907 wait complete. 2024-05-09T10:50:38.516 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T10:50:38.516 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T10:50:38.516 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2024-05-09T10:50:38.516 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-09T10:50:38.516 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-09T10:50:38.516 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T10:50:38.516 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T10:50:38.516 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T10:50:38.517 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-09T10:50:38.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:38.517 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2024-05-09T10:50:38.533 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T10:50:38.534 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T10:50:38.534 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T10:50:38.534 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:38.534 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-05-09T10:50:38.534 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:38.534 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T10:50:38.534 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2024-05-09T10:50:38.534 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2024-05-09T10:50:38.534 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T10:50:38.534 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2024-05-09T10:50:38.534 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2024-05-09T10:50:38.534 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T10:50:38.534 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2024-05-09T10:50:38.534 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T10:50:38.535 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2024-05-09T10:50:38.535 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-09T10:50:38.535 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42767sec preferred_lft 42767sec 2024-05-09T10:50:38.535 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2024-05-09T10:50:38.535 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:38.535 INFO:teuthology.orchestra.run.smithi105.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T10:50:38.535 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 76:1d:6d:2c:57:3f brd ff:ff:ff:ff:ff:ff 2024-05-09T10:50:38.535 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T10:50:38.535 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:38.535 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::6ccf:48ff:fe53:46aa/64 scope link 2024-05-09T10:50:38.535 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:38.535 INFO:teuthology.orchestra.run.smithi105.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T10:50:38.535 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 76:1d:6d:2c:57:3f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T10:50:38.535 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::741d:6dff:fe2c:573f/64 scope link tentative 2024-05-09T10:50:38.535 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:38.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:38.536 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T10:50:38.598 INFO:teuthology.orchestra.run.smithi105.stdout:1 2024-05-09T10:50:38.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:38.599 DEBUG:teuthology.orchestra.run.smithi105:> route 2024-05-09T10:50:38.657 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2024-05-09T10:50:38.657 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T10:50:38.657 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T10:50:38.657 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T10:50:38.657 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T10:50:38.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:38.657 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-05-09T10:50:38.657 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-05-09T10:50:38.658 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T10:50:38.658 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T10:50:38.658 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T10:50:38.658 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-05-09T10:50:38.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:38.743 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-05-09T10:50:38.759 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-09T10:50:38.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:38.759 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list-id 2024-05-09T10:50:38.814 INFO:teuthology.orchestra.run.smithi105.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-09T10:50:38.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:38.814 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-05-09T10:50:38.815 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-05-09T10:50:38.815 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-09T10:50:38.815 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-09T10:50:38.815 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-05-09T10:50:38.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:38.891 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-09T10:50:38.955 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T10:50:38.955 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T10:50:38.955 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T10:50:38.955 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:38.956 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-05-09T10:50:38.956 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:38.956 INFO:teuthology.orchestra.run.smithi105.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T10:50:38.956 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 02:69:15:81:fc:dc brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-09T10:50:38.956 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-09T10:50:38.956 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:38.956 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::69:15ff:fe81:fcdc/64 scope link 2024-05-09T10:50:38.956 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:38.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:38.957 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-09T10:50:39.021 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T10:50:39.022 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T10:50:39.022 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T10:50:39.022 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:39.022 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-05-09T10:50:39.022 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:39.022 INFO:teuthology.orchestra.run.smithi105.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T10:50:39.022 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 02:69:15:81:fc:dc brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-09T10:50:39.022 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-09T10:50:39.022 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:39.023 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::69:15ff:fe81:fcdc/64 scope link 2024-05-09T10:50:39.023 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:50:39.024 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-09T10:50:39.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:39.024 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-05-09T10:50:39.025 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-05-09T10:50:39.025 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-09T10:50:39.025 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-09T10:50:39.025 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-09T10:50:39.025 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-09T10:50:39.025 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-09T10:50:39.025 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-05-09T10:50:39.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:39.138 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-05-09T10:50:39.138 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-05-09T10:50:39.138 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.1 up 2024-05-09T10:50:39.138 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set dev brx.1 master ceph-brx 2024-05-09T10:50:39.138 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-05-09T10:50:39.176 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T10:50:39.176 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-09T10:50:39.232 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-09T10:50:39.232 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T10:50:39.233 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-09T10:50:39.288 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-09T10:50:39.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:39.288 DEBUG:teuthology.orchestra.run.smithi105:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-09T10:50:39.371 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-09T10:50:39.372 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-09T10:50:39.372 INFO:teuthology.orchestra.run.smithi105.stdout:invalid new device string format 2024-05-09T10:50:39.381 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-09T10:50:39.381 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-09T10:50:39.381 INFO:teuthology.orchestra.run.smithi105.stdout:invalid new device string format 2024-05-09T10:50:39.381 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.105:3300,172.21.15.105:3301" 2024-05-09T10:50:39.381 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-09T10:50:39.386 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-09T10:50:39.386 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-09T10:50:39.386 INFO:teuthology.orchestra.run.smithi105.stdout:invalid new device string format 2024-05-09T10:50:39.386 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.105:3300,172.21.15.105:3301" 2024-05-09T10:50:39.386 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-09T10:50:39.386 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: resolved to: "172.21.15.98:6789,172.21.15.105:6789,172.21.15.105:6790" 2024-05-09T10:50:39.386 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.98:6789,172.21.15.105:6789,172.21.15.105:6790:/ 2024-05-09T10:50:39.386 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=3f705980-2043-4fe5-8639-5415c9d3c944" will pass to kernel 2024-05-09T10:50:39.387 INFO:teuthology.orchestra.run.smithi105.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-05-09T10:50:39.387 INFO:teuthology.orchestra.run.smithi105.stdout: You just mounted a file system that supports labels which does not 2024-05-09T10:50:39.387 INFO:teuthology.orchestra.run.smithi105.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-09T10:50:39.387 INFO:teuthology.orchestra.run.smithi105.stdout: applications will generate AVC messages and not be allowed access to 2024-05-09T10:50:39.387 INFO:teuthology.orchestra.run.smithi105.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-09T10:50:39.388 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-09T10:50:39.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:39.389 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-09T10:50:39.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:39.455 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T10:50:39.512 INFO:teuthology.orchestra.run.smithi105.stdout:client4725 2024-05-09T10:50:39.513 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:50:39.513 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:50:39.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:39.570 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T10:50:39.627 INFO:teuthology.orchestra.run.smithi105.stdout:client4725 2024-05-09T10:50:39.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:39.628 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T10:50:39.685 INFO:teuthology.orchestra.run.smithi105.stdout:client4725 2024-05-09T10:50:39.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:39.685 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/3f705980-2043-4fe5-8639-5415c9d3c944.client4725/status') 2024-05-09T10:50:39.751 INFO:teuthology.orchestra.run.smithi105.stdout:instance: client.4725 (3)172.21.15.105:0/2186257253 2024-05-09T10:50:39.751 INFO:teuthology.orchestra.run.smithi105.stdout:blocklisted: false 2024-05-09T10:50:39.751 INFO:teuthology.orchestra.run.smithi105.stderr:0+1 records in 2024-05-09T10:50:39.751 INFO:teuthology.orchestra.run.smithi105.stderr:0+1 records out 2024-05-09T10:50:39.751 INFO:teuthology.orchestra.run.smithi105.stderr:71 bytes copied, 2.2055e-05 s, 3.2 MB/s 2024-05-09T10:50:39.752 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-09T10:50:39.764 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-09T10:50:39.799 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:39.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.917+0000 7f39f6dce640 1 Processor -- start 2024-05-09T10:50:39.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.917+0000 7f39f6dce640 1 -- start start 2024-05-09T10:50:39.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.918+0000 7f39f6dce640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f39f01443e0 0x7f39f00720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:39.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.918+0000 7f39f6dce640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f0143b20 con 0x7f39f0143740 2024-05-09T10:50:39.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.918+0000 7f39f6dce640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f015f460 con 0x7f39f0143d70 2024-05-09T10:50:39.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.918+0000 7f39f6dce640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f00b28c0 con 0x7f39f01443e0 2024-05-09T10:50:39.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.918+0000 7f39e7fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f39f01443e0 0x7f39f00720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:39.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.918+0000 7f39f4b6a640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f39f0143740 0x7f39f015e6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38068/0 (socket says 172.21.15.98:38068) 2024-05-09T10:50:39.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.918+0000 7f39f4b6a640 1 -- 172.21.15.98:0/1046026432 learned_addr learned my addr 172.21.15.98:0/1046026432 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:39.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.918+0000 7f39e7fff640 1 -- 172.21.15.98:0/1046026432 >> v1:172.21.15.105:6789/0 conn(0x7f39f0143d70 legacy=0x7f39f00b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:39.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.918+0000 7f39e7fff640 1 -- 172.21.15.98:0/1046026432 >> v1:172.21.15.98:6789/0 conn(0x7f39f0143740 legacy=0x7f39f015e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:39.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.918+0000 7f39e77fe640 1 -- 172.21.15.98:0/1046026432 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935908323 0 0) 0x7f39f0143b20 con 0x7f39f0143740 2024-05-09T10:50:39.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.918+0000 7f39e7fff640 1 -- 172.21.15.98:0/1046026432 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f00b2ba0 con 0x7f39f01443e0 2024-05-09T10:50:39.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.919+0000 7f39e7fff640 1 --2- 172.21.15.98:0/1046026432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39f01443e0 0x7f39f00720a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f39e00097a0 tx=0x7f39e002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae72d4b7b00e23ac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:39.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.919+0000 7f39e77fe640 1 -- 172.21.15.98:0/1046026432 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e0037070 con 0x7f39f01443e0 2024-05-09T10:50:39.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.919+0000 7f39e77fe640 1 -- 172.21.15.98:0/1046026432 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e0032440 con 0x7f39f01443e0 2024-05-09T10:50:39.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.919+0000 7f39e77fe640 1 -- 172.21.15.98:0/1046026432 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e0032810 con 0x7f39f01443e0 2024-05-09T10:50:39.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.920+0000 7f39f6dce640 1 -- 172.21.15.98:0/1046026432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39f01443e0 msgr2=0x7f39f00720a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:39.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.920+0000 7f39f6dce640 1 --2- 172.21.15.98:0/1046026432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39f01443e0 0x7f39f00720a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f39e00097a0 tx=0x7f39e002a230 comp rx=0 tx=0).stop 2024-05-09T10:50:39.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.920+0000 7f39f6dce640 1 -- 172.21.15.98:0/1046026432 shutdown_connections 2024-05-09T10:50:39.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.920+0000 7f39f6dce640 1 --2- 172.21.15.98:0/1046026432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39f01443e0 0x7f39f00720a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:39.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.920+0000 7f39f6dce640 1 -- 172.21.15.98:0/1046026432 >> 172.21.15.98:0/1046026432 conn(0x7f39f006abe0 msgr2=0x7f39f006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:39.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.920+0000 7f39f6dce640 1 -- 172.21.15.98:0/1046026432 shutdown_connections 2024-05-09T10:50:39.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.920+0000 7f39f6dce640 1 -- 172.21.15.98:0/1046026432 wait complete. 2024-05-09T10:50:39.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.921+0000 7f39f6dce640 1 Processor -- start 2024-05-09T10:50:39.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.921+0000 7f39f6dce640 1 -- start start 2024-05-09T10:50:39.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.922+0000 7f39f6dce640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39f0143740 0x7f39f00ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:39.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.922+0000 7f39f6dce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f0143d70 0x7f39f01f3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:39.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.922+0000 7f39f6dce640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39f01443e0 0x7f39f01f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:39.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.922+0000 7f39f6dce640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f015f1c0 con 0x7f39f0143740 2024-05-09T10:50:39.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.922+0000 7f39f6dce640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f015f040 con 0x7f39f0143d70 2024-05-09T10:50:39.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.922+0000 7f39f6dce640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f00b37d0 con 0x7f39f01443e0 2024-05-09T10:50:39.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.922+0000 7f39f4b6a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39f0143740 0x7f39f00ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:39.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.922+0000 7f39f4b6a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39f0143740 0x7f39f00ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48086/0 (socket says 172.21.15.98:48086) 2024-05-09T10:50:39.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.922+0000 7f39f4b6a640 1 -- 172.21.15.98:0/3525831315 learned_addr learned my addr 172.21.15.98:0/3525831315 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:39.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.922+0000 7f39e7fff640 1 --2- 172.21.15.98:0/3525831315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f0143d70 0x7f39f01f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:39.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.922+0000 7f39f536b640 1 --2- 172.21.15.98:0/3525831315 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39f01443e0 0x7f39f01f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:39.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.922+0000 7f39f4b6a640 1 -- 172.21.15.98:0/3525831315 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39f01443e0 msgr2=0x7f39f01f75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:39.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.922+0000 7f39f4b6a640 1 --2- 172.21.15.98:0/3525831315 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39f01443e0 0x7f39f01f75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:39.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.922+0000 7f39f4b6a640 1 -- 172.21.15.98:0/3525831315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f0143d70 msgr2=0x7f39f01f3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:39.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.922+0000 7f39f4b6a640 1 --2- 172.21.15.98:0/3525831315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f0143d70 0x7f39f01f3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:39.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.922+0000 7f39f4b6a640 1 -- 172.21.15.98:0/3525831315 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f00ce320 con 0x7f39f0143740 2024-05-09T10:50:39.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.923+0000 7f39f4b6a640 1 --2- 172.21.15.98:0/3525831315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39f0143740 0x7f39f00ca050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00c4a0 tx=0x7f39ec00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:39.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.923+0000 7f39e57fa640 1 -- 172.21.15.98:0/3525831315 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec018070 con 0x7f39f0143740 2024-05-09T10:50:39.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.923+0000 7f39e57fa640 1 -- 172.21.15.98:0/3525831315 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ec0074e0 con 0x7f39f0143740 2024-05-09T10:50:39.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.923+0000 7f39f6dce640 1 -- 172.21.15.98:0/3525831315 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f00d0190 con 0x7f39f0143740 2024-05-09T10:50:39.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.923+0000 7f39e57fa640 1 -- 172.21.15.98:0/3525831315 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec004a20 con 0x7f39f0143740 2024-05-09T10:50:39.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.923+0000 7f39f6dce640 1 -- 172.21.15.98:0/3525831315 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f00c9850 con 0x7f39f0143740 2024-05-09T10:50:39.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.924+0000 7f39f6dce640 1 -- 172.21.15.98:0/3525831315 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f0069cb0 con 0x7f39f0143740 2024-05-09T10:50:39.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.924+0000 7f39e57fa640 1 -- 172.21.15.98:0/3525831315 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f39ec01b070 con 0x7f39f0143740 2024-05-09T10:50:39.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.925+0000 7f39e57fa640 1 --2- 172.21.15.98:0/3525831315 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f39dc06b4d0 0x7f39dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:39.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.925+0000 7f39e7fff640 1 --2- 172.21.15.98:0/3525831315 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f39dc06b4d0 0x7f39dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:39.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.925+0000 7f39e57fa640 1 -- 172.21.15.98:0/3525831315 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39ec0893a0 con 0x7f39f0143740 2024-05-09T10:50:39.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.926+0000 7f39e7fff640 1 --2- 172.21.15.98:0/3525831315 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f39dc06b4d0 0x7f39dc06d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f39e0034000 tx=0x7f39e002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:39.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:39.929+0000 7f39e57fa640 1 -- 172.21.15.98:0/3525831315 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f39ec057c60 con 0x7f39f0143740 2024-05-09T10:50:40.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.105+0000 7f39f6dce640 1 -- 172.21.15.98:0/3525831315 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39f015e450 con 0x7f39f0143740 2024-05-09T10:50:40.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.106+0000 7f39e57fa640 1 -- 172.21.15.98:0/3525831315 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f39ec05bd00 con 0x7f39f0143740 2024-05-09T10:50:40.099 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:40.099 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":20,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:21.176419+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:50:17.323444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T10:50:18.482315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:40.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.108+0000 7f39f6dce640 1 -- 172.21.15.98:0/3525831315 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f39dc06b4d0 msgr2=0x7f39dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:40.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.108+0000 7f39f6dce640 1 --2- 172.21.15.98:0/3525831315 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f39dc06b4d0 0x7f39dc06d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f39e0034000 tx=0x7f39e002e000 comp rx=0 tx=0).stop 2024-05-09T10:50:40.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.108+0000 7f39f6dce640 1 -- 172.21.15.98:0/3525831315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39f0143740 msgr2=0x7f39f00ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:40.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.108+0000 7f39f6dce640 1 --2- 172.21.15.98:0/3525831315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39f0143740 0x7f39f00ca050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00c4a0 tx=0x7f39ec00c960 comp rx=0 tx=0).stop 2024-05-09T10:50:40.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.108+0000 7f39f6dce640 1 -- 172.21.15.98:0/3525831315 shutdown_connections 2024-05-09T10:50:40.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.108+0000 7f39f6dce640 1 --2- 172.21.15.98:0/3525831315 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f39dc06b4d0 0x7f39dc06d990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.108+0000 7f39f6dce640 1 --2- 172.21.15.98:0/3525831315 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39f01443e0 0x7f39f01f75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.108+0000 7f39f6dce640 1 --2- 172.21.15.98:0/3525831315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39f0143d70 0x7f39f01f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.108+0000 7f39f6dce640 1 --2- 172.21.15.98:0/3525831315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39f0143740 0x7f39f00ca050 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.108+0000 7f39f6dce640 1 -- 172.21.15.98:0/3525831315 >> 172.21.15.98:0/3525831315 conn(0x7f39f006abe0 msgr2=0x7f39f00702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:40.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.108+0000 7f39f6dce640 1 -- 172.21.15.98:0/3525831315 shutdown_connections 2024-05-09T10:50:40.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.108+0000 7f39f6dce640 1 -- 172.21.15.98:0/3525831315 wait complete. 2024-05-09T10:50:40.113 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T10:50:07.396423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T10:50:17.323444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T10:50:18.482315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T10:50:40.113 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T10:50:40.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.222+0000 7f8d5ad4c640 1 Processor -- start 2024-05-09T10:50:40.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.223+0000 7f8d5ad4c640 1 -- start start 2024-05-09T10:50:40.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.223+0000 7f8d5ad4c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8d54161450 0x7f8d540d4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:40.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.223+0000 7f8d5ad4c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d54161830 con 0x7f8d540b2a30 2024-05-09T10:50:40.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.223+0000 7f8d5ad4c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d540cf2f0 con 0x7f8d540d2820 2024-05-09T10:50:40.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.223+0000 7f8d5ad4c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d540b0760 con 0x7f8d54161450 2024-05-09T10:50:40.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.223+0000 7f8d53fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f8d540b2a30 0x7f8d540d2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38076/0 (socket says 172.21.15.98:38076) 2024-05-09T10:50:40.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.224+0000 7f8d53fff640 1 -- 172.21.15.98:0/3721025955 learned_addr learned my addr 172.21.15.98:0/3721025955 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:40.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.224+0000 7f8d58ae8640 1 --2- 172.21.15.98:0/3721025955 >> v2:172.21.15.105:3300/0 conn(0x7f8d54161450 0x7f8d540d4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:40.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.224+0000 7f8d537fe640 1 -- 172.21.15.98:0/3721025955 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2124002114 0 0) 0x7f8d3c001110 con 0x7f8d540b2a30 2024-05-09T10:50:40.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.224+0000 7f8d537fe640 1 -- 172.21.15.98:0/3721025955 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d40003520 con 0x7f8d540b2a30 2024-05-09T10:50:40.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.224+0000 7f8d58ae8640 1 -- 172.21.15.98:0/3721025955 >> v1:172.21.15.105:6789/0 conn(0x7f8d540d2820 legacy=0x7f8d540b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:40.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.224+0000 7f8d537fe640 1 -- 172.21.15.98:0/3721025955 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172951069 0 0) 0x7f8d540cf2f0 con 0x7f8d540d2820 2024-05-09T10:50:40.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.224+0000 7f8d58ae8640 1 -- 172.21.15.98:0/3721025955 >> v1:172.21.15.98:6789/0 conn(0x7f8d540b2a30 legacy=0x7f8d540d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:40.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.224+0000 7f8d58ae8640 1 -- 172.21.15.98:0/3721025955 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d540cf5d0 con 0x7f8d54161450 2024-05-09T10:50:40.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.225+0000 7f8d58ae8640 1 --2- 172.21.15.98:0/3721025955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d54161450 0x7f8d540d4650 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8d44009dc0 tx=0x7f8d4402a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce5012ec1cd7a055 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:40.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.225+0000 7f8d537fe640 1 -- 172.21.15.98:0/3721025955 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44007740 con 0x7f8d54161450 2024-05-09T10:50:40.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.225+0000 7f8d537fe640 1 -- 172.21.15.98:0/3721025955 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d440078e0 con 0x7f8d54161450 2024-05-09T10:50:40.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.225+0000 7f8d537fe640 1 -- 172.21.15.98:0/3721025955 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44007cb0 con 0x7f8d54161450 2024-05-09T10:50:40.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.225+0000 7f8d5ad4c640 1 -- 172.21.15.98:0/3721025955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d54161450 msgr2=0x7f8d540d4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:40.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.225+0000 7f8d5ad4c640 1 --2- 172.21.15.98:0/3721025955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d54161450 0x7f8d540d4650 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8d44009dc0 tx=0x7f8d4402a6a0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.225+0000 7f8d5ad4c640 1 -- 172.21.15.98:0/3721025955 shutdown_connections 2024-05-09T10:50:40.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.225+0000 7f8d5ad4c640 1 --2- 172.21.15.98:0/3721025955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d54161450 0x7f8d540d4650 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.225+0000 7f8d5ad4c640 1 -- 172.21.15.98:0/3721025955 >> 172.21.15.98:0/3721025955 conn(0x7f8d540d3b30 msgr2=0x7f8d540d3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:40.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.225+0000 7f8d5ad4c640 1 -- 172.21.15.98:0/3721025955 shutdown_connections 2024-05-09T10:50:40.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.225+0000 7f8d5ad4c640 1 -- 172.21.15.98:0/3721025955 wait complete. 2024-05-09T10:50:40.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.226+0000 7f8d5ad4c640 1 Processor -- start 2024-05-09T10:50:40.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.227+0000 7f8d5ad4c640 1 -- start start 2024-05-09T10:50:40.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.227+0000 7f8d5ad4c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d540b2a30 0x7f8d540cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:40.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.227+0000 7f8d5ad4c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d540d2820 0x7f8d5415df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:40.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.227+0000 7f8d5ad4c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d54161450 0x7f8d540b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:40.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.227+0000 7f8d5ad4c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540b15e0 con 0x7f8d540d2820 2024-05-09T10:50:40.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.227+0000 7f8d5ad4c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540b09b0 con 0x7f8d540b2a30 2024-05-09T10:50:40.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.227+0000 7f8d5ad4c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540b1760 con 0x7f8d54161450 2024-05-09T10:50:40.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.228+0000 7f8d53fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d540d2820 0x7f8d5415df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:40.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.228+0000 7f8d53fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d540d2820 0x7f8d5415df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48088/0 (socket says 172.21.15.98:48088) 2024-05-09T10:50:40.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.228+0000 7f8d53fff640 1 -- 172.21.15.98:0/3390905961 learned_addr learned my addr 172.21.15.98:0/3390905961 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:40.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.228+0000 7f8d58ae8640 1 --2- 172.21.15.98:0/3390905961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d540b2a30 0x7f8d540cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:40.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.228+0000 7f8d592e9640 1 --2- 172.21.15.98:0/3390905961 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d54161450 0x7f8d540b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:40.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.228+0000 7f8d53fff640 1 -- 172.21.15.98:0/3390905961 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d54161450 msgr2=0x7f8d540b8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:40.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.228+0000 7f8d53fff640 1 --2- 172.21.15.98:0/3390905961 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d54161450 0x7f8d540b8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.228+0000 7f8d53fff640 1 -- 172.21.15.98:0/3390905961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d540b2a30 msgr2=0x7f8d540cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:40.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.228+0000 7f8d53fff640 1 --2- 172.21.15.98:0/3390905961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d540b2a30 0x7f8d540cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.228+0000 7f8d53fff640 1 -- 172.21.15.98:0/3390905961 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d540cb450 con 0x7f8d540d2820 2024-05-09T10:50:40.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.228+0000 7f8d53fff640 1 --2- 172.21.15.98:0/3390905961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d540d2820 0x7f8d5415df40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00c760 tx=0x7f8d3c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:40.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.228+0000 7f8d517fa640 1 -- 172.21.15.98:0/3390905961 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c017070 con 0x7f8d540d2820 2024-05-09T10:50:40.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.228+0000 7f8d517fa640 1 -- 172.21.15.98:0/3390905961 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3c005ad0 con 0x7f8d540d2820 2024-05-09T10:50:40.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.228+0000 7f8d517fa640 1 -- 172.21.15.98:0/3390905961 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c007e20 con 0x7f8d540d2820 2024-05-09T10:50:40.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.228+0000 7f8d5ad4c640 1 -- 172.21.15.98:0/3390905961 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d540cb730 con 0x7f8d540d2820 2024-05-09T10:50:40.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.228+0000 7f8d5ad4c640 1 -- 172.21.15.98:0/3390905961 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d540cb140 con 0x7f8d540d2820 2024-05-09T10:50:40.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.229+0000 7f8d5ad4c640 1 -- 172.21.15.98:0/3390905961 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d540cbc50 con 0x7f8d540d2820 2024-05-09T10:50:40.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.230+0000 7f8d517fa640 1 -- 172.21.15.98:0/3390905961 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8d3c01a070 con 0x7f8d540d2820 2024-05-09T10:50:40.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.230+0000 7f8d517fa640 1 --2- 172.21.15.98:0/3390905961 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8d3406b4d0 0x7f8d3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:40.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.230+0000 7f8d517fa640 1 -- 172.21.15.98:0/3390905961 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d3c0894d0 con 0x7f8d540d2820 2024-05-09T10:50:40.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.230+0000 7f8d58ae8640 1 --2- 172.21.15.98:0/3390905961 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8d3406b4d0 0x7f8d3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:40.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.230+0000 7f8d58ae8640 1 --2- 172.21.15.98:0/3390905961 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8d3406b4d0 0x7f8d3406d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8d44034000 tx=0x7f8d4402f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:40.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.234+0000 7f8d517fa640 1 -- 172.21.15.98:0/3390905961 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8d540cbc50 con 0x7f8d540d2820 2024-05-09T10:50:40.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.406+0000 7f8d5ad4c640 1 -- 172.21.15.98:0/3390905961 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8d540a1ae0 con 0x7f8d540d2820 2024-05-09T10:50:40.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.406+0000 7f8d517fa640 1 -- 172.21.15.98:0/3390905961 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8d3c087e20 con 0x7f8d540d2820 2024-05-09T10:50:40.400 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-05-09T10:50:40.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.408+0000 7f8d5ad4c640 1 -- 172.21.15.98:0/3390905961 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8d3406b4d0 msgr2=0x7f8d3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:40.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.408+0000 7f8d5ad4c640 1 --2- 172.21.15.98:0/3390905961 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8d3406b4d0 0x7f8d3406d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8d44034000 tx=0x7f8d4402f000 comp rx=0 tx=0).stop 2024-05-09T10:50:40.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.408+0000 7f8d5ad4c640 1 -- 172.21.15.98:0/3390905961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d540d2820 msgr2=0x7f8d5415df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:40.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.408+0000 7f8d5ad4c640 1 --2- 172.21.15.98:0/3390905961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d540d2820 0x7f8d5415df40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00c760 tx=0x7f8d3c00cc20 comp rx=0 tx=0).stop 2024-05-09T10:50:40.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.408+0000 7f8d5ad4c640 1 -- 172.21.15.98:0/3390905961 shutdown_connections 2024-05-09T10:50:40.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.408+0000 7f8d5ad4c640 1 --2- 172.21.15.98:0/3390905961 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8d3406b4d0 0x7f8d3406d990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.408+0000 7f8d5ad4c640 1 --2- 172.21.15.98:0/3390905961 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d54161450 0x7f8d540b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.408+0000 7f8d5ad4c640 1 --2- 172.21.15.98:0/3390905961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d540d2820 0x7f8d5415df40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.408+0000 7f8d5ad4c640 1 --2- 172.21.15.98:0/3390905961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d540b2a30 0x7f8d540cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.408+0000 7f8d5ad4c640 1 -- 172.21.15.98:0/3390905961 >> 172.21.15.98:0/3390905961 conn(0x7f8d540d3b30 msgr2=0x7f8d540c7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:40.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.408+0000 7f8d5ad4c640 1 -- 172.21.15.98:0/3390905961 shutdown_connections 2024-05-09T10:50:40.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.408+0000 7f8d5ad4c640 1 -- 172.21.15.98:0/3390905961 wait complete. 2024-05-09T10:50:40.413 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T10:50:40.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.521+0000 7fe3d83c1640 1 Processor -- start 2024-05-09T10:50:40.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.521+0000 7fe3d83c1640 1 -- start start 2024-05-09T10:50:40.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.522+0000 7fe3d83c1640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3d015dd60 0x7fe3d00d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:40.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.522+0000 7fe3d83c1640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe3d015e140 0x7fe3d00cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:40.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.522+0000 7fe3d83c1640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3d00cf8e0 con 0x7fe3d00d1de0 2024-05-09T10:50:40.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.522+0000 7fe3d83c1640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3d00cf6b0 con 0x7fe3d015e140 2024-05-09T10:50:40.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.522+0000 7fe3d83c1640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d00b2f50 con 0x7fe3d015dd60 2024-05-09T10:50:40.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.523+0000 7fe3d595c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe3d015e140 0x7fe3d00cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:40.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.523+0000 7fe3d695e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe3d00d1de0 0x7fe3d00d0520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38088/0 (socket says 172.21.15.98:38088) 2024-05-09T10:50:40.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.523+0000 7fe3d695e640 1 -- 172.21.15.98:0/1057245655 learned_addr learned my addr 172.21.15.98:0/1057245655 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:40.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.523+0000 7fe3d615d640 1 --2- 172.21.15.98:0/1057245655 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3d015dd60 0x7fe3d00d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:40.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.523+0000 7fe3d595c640 1 -- 172.21.15.98:0/1057245655 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3d015dd60 msgr2=0x7fe3d00d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:40.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.523+0000 7fe3d595c640 1 --2- 172.21.15.98:0/1057245655 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3d015dd60 0x7fe3d00d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.523+0000 7fe3d595c640 1 -- 172.21.15.98:0/1057245655 >> v1:172.21.15.98:6789/0 conn(0x7fe3d00d1de0 legacy=0x7fe3d00d0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:40.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.523+0000 7fe3d595c640 1 -- 172.21.15.98:0/1057245655 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d00b31a0 con 0x7fe3d015e140 2024-05-09T10:50:40.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.524+0000 7fe3d595c640 1 --2- 172.21.15.98:0/1057245655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d015e140 0x7fe3d00cf170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe3c40097a0 tx=0x7fe3c402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3572c116abfb4fc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:40.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.524+0000 7fe3d515b640 1 -- 172.21.15.98:0/1057245655 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4037070 con 0x7fe3d015e140 2024-05-09T10:50:40.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.524+0000 7fe3d515b640 1 -- 172.21.15.98:0/1057245655 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c4032440 con 0x7fe3d015e140 2024-05-09T10:50:40.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.524+0000 7fe3d515b640 1 -- 172.21.15.98:0/1057245655 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4032810 con 0x7fe3d015e140 2024-05-09T10:50:40.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.524+0000 7fe3d83c1640 1 -- 172.21.15.98:0/1057245655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d015e140 msgr2=0x7fe3d00cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:40.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.524+0000 7fe3d83c1640 1 --2- 172.21.15.98:0/1057245655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d015e140 0x7fe3d00cf170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe3c40097a0 tx=0x7fe3c402a180 comp rx=0 tx=0).stop 2024-05-09T10:50:40.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.524+0000 7fe3d83c1640 1 -- 172.21.15.98:0/1057245655 shutdown_connections 2024-05-09T10:50:40.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.524+0000 7fe3d83c1640 1 --2- 172.21.15.98:0/1057245655 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d015e140 0x7fe3d00cf170 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.524+0000 7fe3d83c1640 1 --2- 172.21.15.98:0/1057245655 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3d015dd60 0x7fe3d00d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.524+0000 7fe3d83c1640 1 -- 172.21.15.98:0/1057245655 >> 172.21.15.98:0/1057245655 conn(0x7fe3d00d3b50 msgr2=0x7fe3d00d3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:40.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.524+0000 7fe3d83c1640 1 -- 172.21.15.98:0/1057245655 shutdown_connections 2024-05-09T10:50:40.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.524+0000 7fe3d83c1640 1 -- 172.21.15.98:0/1057245655 wait complete. 2024-05-09T10:50:40.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.525+0000 7fe3d83c1640 1 Processor -- start 2024-05-09T10:50:40.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.526+0000 7fe3d83c1640 1 -- start start 2024-05-09T10:50:40.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.526+0000 7fe3d83c1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3d00d1de0 0x7fe3d00b6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:40.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.526+0000 7fe3d83c1640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3d015dd60 0x7fe3d01f6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:40.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.526+0000 7fe3d83c1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d00c8570 0x7fe3d00c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:40.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.526+0000 7fe3d83c1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d00b4c90 con 0x7fe3d00c8570 2024-05-09T10:50:40.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.526+0000 7fe3d83c1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d00b3760 con 0x7fe3d00d1de0 2024-05-09T10:50:40.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.526+0000 7fe3d83c1640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d00b4e10 con 0x7fe3d015dd60 2024-05-09T10:50:40.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.526+0000 7fe3d695e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d00c8570 0x7fe3d00c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:40.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.526+0000 7fe3d695e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d00c8570 0x7fe3d00c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48106/0 (socket says 172.21.15.98:48106) 2024-05-09T10:50:40.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.526+0000 7fe3d695e640 1 -- 172.21.15.98:0/2592226297 learned_addr learned my addr 172.21.15.98:0/2592226297 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:40.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.526+0000 7fe3d615d640 1 --2- 172.21.15.98:0/2592226297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3d00d1de0 0x7fe3d00b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:40.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.526+0000 7fe3d595c640 1 --2- 172.21.15.98:0/2592226297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3d015dd60 0x7fe3d01f6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:40.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.527+0000 7fe3d695e640 1 -- 172.21.15.98:0/2592226297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3d015dd60 msgr2=0x7fe3d01f6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:40.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.527+0000 7fe3d695e640 1 --2- 172.21.15.98:0/2592226297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3d015dd60 0x7fe3d01f6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.527+0000 7fe3d695e640 1 -- 172.21.15.98:0/2592226297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3d00d1de0 msgr2=0x7fe3d00b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:40.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.527+0000 7fe3d695e640 1 --2- 172.21.15.98:0/2592226297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3d00d1de0 0x7fe3d00b6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.527+0000 7fe3d695e640 1 -- 172.21.15.98:0/2592226297 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d00c77b0 con 0x7fe3d00c8570 2024-05-09T10:50:40.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.527+0000 7fe3d695e640 1 --2- 172.21.15.98:0/2592226297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d00c8570 0x7fe3d00c8a20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000cbc0 tx=0x7fe3c000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:40.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.527+0000 7fe3ceffd640 1 -- 172.21.15.98:0/2592226297 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0017070 con 0x7fe3d00c8570 2024-05-09T10:50:40.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.527+0000 7fe3ceffd640 1 -- 172.21.15.98:0/2592226297 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c000ad40 con 0x7fe3d00c8570 2024-05-09T10:50:40.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.527+0000 7fe3d83c1640 1 -- 172.21.15.98:0/2592226297 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d00c7a90 con 0x7fe3d00c8570 2024-05-09T10:50:40.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.527+0000 7fe3ceffd640 1 -- 172.21.15.98:0/2592226297 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0012600 con 0x7fe3d00c8570 2024-05-09T10:50:40.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.527+0000 7fe3d83c1640 1 -- 172.21.15.98:0/2592226297 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d00c8050 con 0x7fe3d00c8570 2024-05-09T10:50:40.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.528+0000 7fe3d83c1640 1 -- 172.21.15.98:0/2592226297 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3d006c870 con 0x7fe3d00c8570 2024-05-09T10:50:40.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.528+0000 7fe3ceffd640 1 -- 172.21.15.98:0/2592226297 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe3c001a070 con 0x7fe3d00c8570 2024-05-09T10:50:40.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.529+0000 7fe3ceffd640 1 --2- 172.21.15.98:0/2592226297 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe3bc06b4d0 0x7fe3bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:40.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.529+0000 7fe3ceffd640 1 -- 172.21.15.98:0/2592226297 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3c0089de0 con 0x7fe3d00c8570 2024-05-09T10:50:40.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.529+0000 7fe3d615d640 1 --2- 172.21.15.98:0/2592226297 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe3bc06b4d0 0x7fe3bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:40.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.529+0000 7fe3d615d640 1 --2- 172.21.15.98:0/2592226297 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe3bc06b4d0 0x7fe3bc06d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8006fd0 tx=0x7fe3c8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:40.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.533+0000 7fe3ceffd640 1 -- 172.21.15.98:0/2592226297 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe3d006c870 con 0x7fe3d00c8570 2024-05-09T10:50:40.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.702+0000 7fe3d83c1640 1 -- 172.21.15.98:0/2592226297 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe3d00a1b60 con 0x7fe3d00c8570 2024-05-09T10:50:40.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.702+0000 7fe3ceffd640 1 -- 172.21.15.98:0/2592226297 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe3c00586a0 con 0x7fe3d00c8570 2024-05-09T10:50:40.696 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2024-05-09T10:50:40.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.704+0000 7fe3d83c1640 1 -- 172.21.15.98:0/2592226297 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe3bc06b4d0 msgr2=0x7fe3bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:40.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.704+0000 7fe3d83c1640 1 --2- 172.21.15.98:0/2592226297 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe3bc06b4d0 0x7fe3bc06d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8006fd0 tx=0x7fe3c8008040 comp rx=0 tx=0).stop 2024-05-09T10:50:40.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.704+0000 7fe3d83c1640 1 -- 172.21.15.98:0/2592226297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d00c8570 msgr2=0x7fe3d00c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:40.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.704+0000 7fe3d83c1640 1 --2- 172.21.15.98:0/2592226297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d00c8570 0x7fe3d00c8a20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000cbc0 tx=0x7fe3c000a430 comp rx=0 tx=0).stop 2024-05-09T10:50:40.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.704+0000 7fe3d83c1640 1 -- 172.21.15.98:0/2592226297 shutdown_connections 2024-05-09T10:50:40.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.704+0000 7fe3d83c1640 1 --2- 172.21.15.98:0/2592226297 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe3bc06b4d0 0x7fe3bc06d990 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.704+0000 7fe3d83c1640 1 --2- 172.21.15.98:0/2592226297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d00c8570 0x7fe3d00c8a20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.704+0000 7fe3d83c1640 1 --2- 172.21.15.98:0/2592226297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe3d015dd60 0x7fe3d01f6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.704+0000 7fe3d83c1640 1 --2- 172.21.15.98:0/2592226297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3d00d1de0 0x7fe3d00b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.704+0000 7fe3d83c1640 1 -- 172.21.15.98:0/2592226297 >> 172.21.15.98:0/2592226297 conn(0x7fe3d00d3b50 msgr2=0x7fe3d0160540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:40.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.704+0000 7fe3d83c1640 1 -- 172.21.15.98:0/2592226297 shutdown_connections 2024-05-09T10:50:40.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.704+0000 7fe3d83c1640 1 -- 172.21.15.98:0/2592226297 wait complete. 2024-05-09T10:50:40.709 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T10:50:40.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.819+0000 7fba41570640 1 Processor -- start 2024-05-09T10:50:40.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.819+0000 7fba41570640 1 -- start start 2024-05-09T10:50:40.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.820+0000 7fba41570640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fba3c161450 0x7fba3c0d4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:40.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.820+0000 7fba41570640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fba3c0d2820 0x7fba3c0b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:40.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.820+0000 7fba41570640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba3c161830 con 0x7fba3c0b2a30 2024-05-09T10:50:40.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.820+0000 7fba41570640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba3c0b0760 con 0x7fba3c161450 2024-05-09T10:50:40.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.820+0000 7fba41570640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba3c0cf2f0 con 0x7fba3c0d2820 2024-05-09T10:50:40.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.820+0000 7fba3affd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fba3c161450 0x7fba3c0d4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:40.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.820+0000 7fba33fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fba3c0b2a30 0x7fba3c0d2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38100/0 (socket says 172.21.15.98:38100) 2024-05-09T10:50:40.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.820+0000 7fba3affd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fba3c161450 0x7fba3c0d4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48112/0 (socket says 172.21.15.98:48112) 2024-05-09T10:50:40.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.820+0000 7fba33fff640 1 -- 172.21.15.98:0/1878636291 learned_addr learned my addr 172.21.15.98:0/1878636291 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:40.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.820+0000 7fba3b7fe640 1 --2- 172.21.15.98:0/1878636291 >> v2:172.21.15.105:3300/0 conn(0x7fba3c0d2820 0x7fba3c0b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:40.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.820+0000 7fba3affd640 1 -- 172.21.15.98:0/1878636291 >> v2:172.21.15.105:3300/0 conn(0x7fba3c0d2820 msgr2=0x7fba3c0b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:40.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.820+0000 7fba3affd640 1 --2- 172.21.15.98:0/1878636291 >> v2:172.21.15.105:3300/0 conn(0x7fba3c0d2820 0x7fba3c0b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.820+0000 7fba3affd640 1 -- 172.21.15.98:0/1878636291 >> v1:172.21.15.98:6789/0 conn(0x7fba3c0b2a30 legacy=0x7fba3c0d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:40.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.820+0000 7fba3affd640 1 -- 172.21.15.98:0/1878636291 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba3c0cf5d0 con 0x7fba3c161450 2024-05-09T10:50:40.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.820+0000 7fba3a7fc640 1 -- 172.21.15.98:0/1878636291 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1349267844 0 0) 0x7fba24001110 con 0x7fba3c0b2a30 2024-05-09T10:50:40.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.821+0000 7fba3affd640 1 --2- 172.21.15.98:0/1878636291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba3c161450 0x7fba3c0d4650 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0097a0 tx=0x7fba2c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f560a479e21dc719 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:40.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.821+0000 7fba3a7fc640 1 -- 172.21.15.98:0/1878636291 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c037070 con 0x7fba3c161450 2024-05-09T10:50:40.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.821+0000 7fba3a7fc640 1 -- 172.21.15.98:0/1878636291 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2c02ae50 con 0x7fba3c161450 2024-05-09T10:50:40.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.821+0000 7fba3a7fc640 1 -- 172.21.15.98:0/1878636291 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c032660 con 0x7fba3c161450 2024-05-09T10:50:40.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.822+0000 7fba41570640 1 -- 172.21.15.98:0/1878636291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba3c161450 msgr2=0x7fba3c0d4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:40.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.822+0000 7fba41570640 1 --2- 172.21.15.98:0/1878636291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba3c161450 0x7fba3c0d4650 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0097a0 tx=0x7fba2c02a140 comp rx=0 tx=0).stop 2024-05-09T10:50:40.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.822+0000 7fba41570640 1 -- 172.21.15.98:0/1878636291 shutdown_connections 2024-05-09T10:50:40.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.822+0000 7fba41570640 1 --2- 172.21.15.98:0/1878636291 >> v2:172.21.15.105:3300/0 conn(0x7fba3c0d2820 0x7fba3c0b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.822+0000 7fba41570640 1 --2- 172.21.15.98:0/1878636291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba3c161450 0x7fba3c0d4650 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.822+0000 7fba41570640 1 -- 172.21.15.98:0/1878636291 >> 172.21.15.98:0/1878636291 conn(0x7fba3c0d3b30 msgr2=0x7fba3c0d3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:40.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.822+0000 7fba41570640 1 -- 172.21.15.98:0/1878636291 shutdown_connections 2024-05-09T10:50:40.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.822+0000 7fba41570640 1 -- 172.21.15.98:0/1878636291 wait complete. 2024-05-09T10:50:40.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.823+0000 7fba41570640 1 Processor -- start 2024-05-09T10:50:40.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.823+0000 7fba41570640 1 -- start start 2024-05-09T10:50:40.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.824+0000 7fba41570640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba3c0b2a30 0x7fba3c0b8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:40.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.824+0000 7fba41570640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba3c0d2820 0x7fba3c0cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:40.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.824+0000 7fba41570640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba3c161450 0x7fba3c159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:40.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.824+0000 7fba41570640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c0b1410 con 0x7fba3c0d2820 2024-05-09T10:50:40.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.824+0000 7fba41570640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c0b09b0 con 0x7fba3c0b2a30 2024-05-09T10:50:40.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.824+0000 7fba41570640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c0b1590 con 0x7fba3c161450 2024-05-09T10:50:40.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.824+0000 7fba33fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba3c0d2820 0x7fba3c0cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:40.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.824+0000 7fba33fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba3c0d2820 0x7fba3c0cac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48128/0 (socket says 172.21.15.98:48128) 2024-05-09T10:50:40.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.824+0000 7fba33fff640 1 -- 172.21.15.98:0/2989907091 learned_addr learned my addr 172.21.15.98:0/2989907091 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:40.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.824+0000 7fba3affd640 1 --2- 172.21.15.98:0/2989907091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba3c0b2a30 0x7fba3c0b8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:40.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.824+0000 7fba3b7fe640 1 --2- 172.21.15.98:0/2989907091 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba3c161450 0x7fba3c159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:40.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.824+0000 7fba33fff640 1 -- 172.21.15.98:0/2989907091 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba3c161450 msgr2=0x7fba3c159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:40.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.824+0000 7fba33fff640 1 --2- 172.21.15.98:0/2989907091 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba3c161450 0x7fba3c159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.824+0000 7fba33fff640 1 -- 172.21.15.98:0/2989907091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba3c0b2a30 msgr2=0x7fba3c0b8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:40.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.824+0000 7fba33fff640 1 --2- 172.21.15.98:0/2989907091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba3c0b2a30 0x7fba3c0b8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.824+0000 7fba33fff640 1 -- 172.21.15.98:0/2989907091 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba3c0cbfb0 con 0x7fba3c0d2820 2024-05-09T10:50:40.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.824+0000 7fba3affd640 1 --2- 172.21.15.98:0/2989907091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba3c0b2a30 0x7fba3c0b8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:50:40.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.824+0000 7fba33fff640 1 --2- 172.21.15.98:0/2989907091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba3c0d2820 0x7fba3c0cac00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fba2400c780 tx=0x7fba2400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:40.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.824+0000 7fba337fe640 1 -- 172.21.15.98:0/2989907091 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba24018070 con 0x7fba3c0d2820 2024-05-09T10:50:40.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.824+0000 7fba337fe640 1 -- 172.21.15.98:0/2989907091 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba240072d0 con 0x7fba3c0d2820 2024-05-09T10:50:40.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.824+0000 7fba337fe640 1 -- 172.21.15.98:0/2989907091 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba240047e0 con 0x7fba3c0d2820 2024-05-09T10:50:40.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.824+0000 7fba41570640 1 -- 172.21.15.98:0/2989907091 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba3c0cb4e0 con 0x7fba3c0d2820 2024-05-09T10:50:40.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.824+0000 7fba41570640 1 -- 172.21.15.98:0/2989907091 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba3c0cbaa0 con 0x7fba3c0d2820 2024-05-09T10:50:40.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.825+0000 7fba41570640 1 -- 172.21.15.98:0/2989907091 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba3c0a1ae0 con 0x7fba3c0d2820 2024-05-09T10:50:40.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.826+0000 7fba337fe640 1 -- 172.21.15.98:0/2989907091 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba2401b070 con 0x7fba3c0d2820 2024-05-09T10:50:40.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.826+0000 7fba337fe640 1 --2- 172.21.15.98:0/2989907091 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fba1c06b4d0 0x7fba1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:40.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.826+0000 7fba3affd640 1 --2- 172.21.15.98:0/2989907091 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fba1c06b4d0 0x7fba1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:40.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.826+0000 7fba337fe640 1 -- 172.21.15.98:0/2989907091 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba240893d0 con 0x7fba3c0d2820 2024-05-09T10:50:40.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.827+0000 7fba3affd640 1 --2- 172.21.15.98:0/2989907091 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fba1c06b4d0 0x7fba1c06d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fba2c034000 tx=0x7fba2c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:40.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.830+0000 7fba337fe640 1 -- 172.21.15.98:0/2989907091 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fba3c161830 con 0x7fba3c0d2820 2024-05-09T10:50:40.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.999+0000 7fba41570640 1 -- 172.21.15.98:0/2989907091 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fba3c15e670 con 0x7fba3c0d2820 2024-05-09T10:50:40.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:40.999+0000 7fba337fe640 1 -- 172.21.15.98:0/2989907091 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fba24057c90 con 0x7fba3c0d2820 2024-05-09T10:50:40.993 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2024-05-09T10:50:40.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.001+0000 7fba41570640 1 -- 172.21.15.98:0/2989907091 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fba1c06b4d0 msgr2=0x7fba1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:40.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.001+0000 7fba41570640 1 --2- 172.21.15.98:0/2989907091 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fba1c06b4d0 0x7fba1c06d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fba2c034000 tx=0x7fba2c02e000 comp rx=0 tx=0).stop 2024-05-09T10:50:40.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.001+0000 7fba41570640 1 -- 172.21.15.98:0/2989907091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba3c0d2820 msgr2=0x7fba3c0cac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:40.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.001+0000 7fba41570640 1 --2- 172.21.15.98:0/2989907091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba3c0d2820 0x7fba3c0cac00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fba2400c780 tx=0x7fba2400cc40 comp rx=0 tx=0).stop 2024-05-09T10:50:40.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.001+0000 7fba41570640 1 -- 172.21.15.98:0/2989907091 shutdown_connections 2024-05-09T10:50:40.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.001+0000 7fba41570640 1 --2- 172.21.15.98:0/2989907091 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fba1c06b4d0 0x7fba1c06d990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.001+0000 7fba41570640 1 --2- 172.21.15.98:0/2989907091 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fba3c161450 0x7fba3c159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.001+0000 7fba41570640 1 --2- 172.21.15.98:0/2989907091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fba3c0d2820 0x7fba3c0cac00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.001+0000 7fba41570640 1 --2- 172.21.15.98:0/2989907091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba3c0b2a30 0x7fba3c0b8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:40.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.001+0000 7fba41570640 1 -- 172.21.15.98:0/2989907091 >> 172.21.15.98:0/2989907091 conn(0x7fba3c0d3b30 msgr2=0x7fba3c0c9670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:40.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.001+0000 7fba41570640 1 -- 172.21.15.98:0/2989907091 shutdown_connections 2024-05-09T10:50:40.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.001+0000 7fba41570640 1 -- 172.21.15.98:0/2989907091 wait complete. 2024-05-09T10:50:41.006 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:41.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.113+0000 7ff714837640 1 Processor -- start 2024-05-09T10:50:41.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.113+0000 7ff714837640 1 -- start start 2024-05-09T10:50:41.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.114+0000 7ff714837640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff71015dce0 0x7ff71006bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:41.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.114+0000 7ff714837640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff71015e0c0 0x7ff71006c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:41.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.114+0000 7ff714837640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff7100d0700 0x7ff71006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:41.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.114+0000 7ff714837640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7100b4350 con 0x7ff7100d0700 2024-05-09T10:50:41.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.114+0000 7ff714837640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7100d0c70 con 0x7ff71015dce0 2024-05-09T10:50:41.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.114+0000 7ff714837640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7100d0f50 con 0x7ff71015e0c0 2024-05-09T10:50:41.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.114+0000 7ff70ed9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff7100d0700 0x7ff71006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:41.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.114+0000 7ff70ed9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff7100d0700 0x7ff71006c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48140/0 (socket says 172.21.15.98:48140) 2024-05-09T10:50:41.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.114+0000 7ff70ed9d640 1 -- 172.21.15.98:0/571366107 learned_addr learned my addr 172.21.15.98:0/571366107 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:41.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.114+0000 7ff70e59c640 1 --2- 172.21.15.98:0/571366107 >> v2:172.21.15.105:3300/0 conn(0x7ff71015dce0 0x7ff71006bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:41.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.115+0000 7ff70dd9b640 1 --2- 172.21.15.98:0/571366107 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff71015e0c0 0x7ff71006c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:41.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.115+0000 7ff70ed9d640 1 -- 172.21.15.98:0/571366107 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff71015e0c0 msgr2=0x7ff71006c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:41.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.115+0000 7ff70ed9d640 1 --2- 172.21.15.98:0/571366107 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff71015e0c0 0x7ff71006c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.115+0000 7ff70ed9d640 1 -- 172.21.15.98:0/571366107 >> v2:172.21.15.105:3300/0 conn(0x7ff71015dce0 msgr2=0x7ff71006bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:41.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.115+0000 7ff70ed9d640 1 --2- 172.21.15.98:0/571366107 >> v2:172.21.15.105:3300/0 conn(0x7ff71015dce0 0x7ff71006bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.115+0000 7ff70ed9d640 1 -- 172.21.15.98:0/571366107 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7100b0fd0 con 0x7ff7100d0700 2024-05-09T10:50:41.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.115+0000 7ff70dd9b640 1 --2- 172.21.15.98:0/571366107 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff71015e0c0 0x7ff71006c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:41.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.115+0000 7ff70ed9d640 1 --2- 172.21.15.98:0/571366107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7100d0700 0x7ff71006c8f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff7100d29e0 tx=0x7ff70802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27424035c0029b88 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:41.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.115+0000 7ff70d59a640 1 -- 172.21.15.98:0/571366107 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff708036070 con 0x7ff7100d0700 2024-05-09T10:50:41.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.115+0000 7ff70d59a640 1 -- 172.21.15.98:0/571366107 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff708031600 con 0x7ff7100d0700 2024-05-09T10:50:41.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.115+0000 7ff70d59a640 1 -- 172.21.15.98:0/571366107 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7080319d0 con 0x7ff7100d0700 2024-05-09T10:50:41.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.116+0000 7ff714837640 1 -- 172.21.15.98:0/571366107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7100d0700 msgr2=0x7ff71006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:41.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.116+0000 7ff714837640 1 --2- 172.21.15.98:0/571366107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7100d0700 0x7ff71006c8f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff7100d29e0 tx=0x7ff70802a730 comp rx=0 tx=0).stop 2024-05-09T10:50:41.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.116+0000 7ff714837640 1 -- 172.21.15.98:0/571366107 shutdown_connections 2024-05-09T10:50:41.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.116+0000 7ff714837640 1 --2- 172.21.15.98:0/571366107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7100d0700 0x7ff71006c8f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.116+0000 7ff714837640 1 --2- 172.21.15.98:0/571366107 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff71015e0c0 0x7ff71006c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.116+0000 7ff714837640 1 --2- 172.21.15.98:0/571366107 >> v2:172.21.15.105:3300/0 conn(0x7ff71015dce0 0x7ff71006bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.116+0000 7ff714837640 1 -- 172.21.15.98:0/571366107 >> 172.21.15.98:0/571366107 conn(0x7ff7100b34e0 msgr2=0x7ff7100b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:41.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.116+0000 7ff714837640 1 -- 172.21.15.98:0/571366107 shutdown_connections 2024-05-09T10:50:41.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.116+0000 7ff714837640 1 -- 172.21.15.98:0/571366107 wait complete. 2024-05-09T10:50:41.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.117+0000 7ff714837640 1 Processor -- start 2024-05-09T10:50:41.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.117+0000 7ff714837640 1 -- start start 2024-05-09T10:50:41.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.118+0000 7ff714837640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7100d0700 0x7ff7100b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:41.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.118+0000 7ff714837640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff71015dce0 0x7ff7101603a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:41.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.118+0000 7ff714837640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff71015e0c0 0x7ff71006ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:41.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.118+0000 7ff714837640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff710161a00 con 0x7ff71015e0c0 2024-05-09T10:50:41.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.118+0000 7ff714837640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7100b1590 con 0x7ff71015dce0 2024-05-09T10:50:41.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.118+0000 7ff714837640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff710161b80 con 0x7ff7100d0700 2024-05-09T10:50:41.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.118+0000 7ff70ed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff71015e0c0 0x7ff71006ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:41.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.118+0000 7ff70ed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff71015e0c0 0x7ff71006ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48142/0 (socket says 172.21.15.98:48142) 2024-05-09T10:50:41.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.118+0000 7ff70ed9d640 1 -- 172.21.15.98:0/3372857007 learned_addr learned my addr 172.21.15.98:0/3372857007 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:41.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.118+0000 7ff70e59c640 1 --2- 172.21.15.98:0/3372857007 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7100d0700 0x7ff7100b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:41.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.118+0000 7ff70dd9b640 1 --2- 172.21.15.98:0/3372857007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff71015dce0 0x7ff7101603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:41.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.119+0000 7ff70ed9d640 1 -- 172.21.15.98:0/3372857007 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7100d0700 msgr2=0x7ff7100b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:41.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.119+0000 7ff70ed9d640 1 --2- 172.21.15.98:0/3372857007 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7100d0700 0x7ff7100b51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.119+0000 7ff70ed9d640 1 -- 172.21.15.98:0/3372857007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff71015dce0 msgr2=0x7ff7101603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:41.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.119+0000 7ff70ed9d640 1 --2- 172.21.15.98:0/3372857007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff71015dce0 0x7ff7101603a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.119+0000 7ff70ed9d640 1 -- 172.21.15.98:0/3372857007 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7100b57d0 con 0x7ff71015e0c0 2024-05-09T10:50:41.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.119+0000 7ff70ed9d640 1 --2- 172.21.15.98:0/3372857007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff71015e0c0 0x7ff71006ec80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff708009f40 tx=0x7ff708004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:41.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.119+0000 7ff6feffd640 1 -- 172.21.15.98:0/3372857007 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff708036050 con 0x7ff71015e0c0 2024-05-09T10:50:41.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.119+0000 7ff6feffd640 1 -- 172.21.15.98:0/3372857007 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff708042070 con 0x7ff71015e0c0 2024-05-09T10:50:41.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.119+0000 7ff714837640 1 -- 172.21.15.98:0/3372857007 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7100b5ab0 con 0x7ff71015e0c0 2024-05-09T10:50:41.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.119+0000 7ff6feffd640 1 -- 172.21.15.98:0/3372857007 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff708009510 con 0x7ff71015e0c0 2024-05-09T10:50:41.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.119+0000 7ff714837640 1 -- 172.21.15.98:0/3372857007 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7100b6070 con 0x7ff71015e0c0 2024-05-09T10:50:41.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.120+0000 7ff714837640 1 -- 172.21.15.98:0/3372857007 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff710069cb0 con 0x7ff71015e0c0 2024-05-09T10:50:41.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.121+0000 7ff6feffd640 1 -- 172.21.15.98:0/3372857007 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff708007bf0 con 0x7ff71015e0c0 2024-05-09T10:50:41.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.121+0000 7ff6feffd640 1 --2- 172.21.15.98:0/3372857007 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff6f006b4d0 0x7ff6f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:41.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.121+0000 7ff6feffd640 1 -- 172.21.15.98:0/3372857007 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7080aadf0 con 0x7ff71015e0c0 2024-05-09T10:50:41.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.121+0000 7ff70e59c640 1 --2- 172.21.15.98:0/3372857007 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff6f006b4d0 0x7ff6f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:41.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.121+0000 7ff70e59c640 1 --2- 172.21.15.98:0/3372857007 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff6f006b4d0 0x7ff6f006d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff700006fd0 tx=0x7ff700008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:41.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.125+0000 7ff6feffd640 1 -- 172.21.15.98:0/3372857007 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff7080796b0 con 0x7ff71015e0c0 2024-05-09T10:50:41.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.298+0000 7ff714837640 1 -- 172.21.15.98:0/3372857007 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7100b7930 con 0x7ff71015e0c0 2024-05-09T10:50:41.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.299+0000 7ff6feffd640 1 -- 172.21.15.98:0/3372857007 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7ff70807d750 con 0x7ff71015e0c0 2024-05-09T10:50:41.293 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:41.293 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":20,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:21.176419+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:50:17.323444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T10:50:18.482315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:41.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.301+0000 7ff714837640 1 -- 172.21.15.98:0/3372857007 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff6f006b4d0 msgr2=0x7ff6f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:41.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.301+0000 7ff714837640 1 --2- 172.21.15.98:0/3372857007 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff6f006b4d0 0x7ff6f006d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff700006fd0 tx=0x7ff700008040 comp rx=0 tx=0).stop 2024-05-09T10:50:41.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.301+0000 7ff714837640 1 -- 172.21.15.98:0/3372857007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff71015e0c0 msgr2=0x7ff71006ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:41.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.301+0000 7ff714837640 1 --2- 172.21.15.98:0/3372857007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff71015e0c0 0x7ff71006ec80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff708009f40 tx=0x7ff708004a40 comp rx=0 tx=0).stop 2024-05-09T10:50:41.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.301+0000 7ff714837640 1 -- 172.21.15.98:0/3372857007 shutdown_connections 2024-05-09T10:50:41.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.301+0000 7ff714837640 1 --2- 172.21.15.98:0/3372857007 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff6f006b4d0 0x7ff6f006d990 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.301+0000 7ff714837640 1 --2- 172.21.15.98:0/3372857007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff71015e0c0 0x7ff71006ec80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.301+0000 7ff714837640 1 --2- 172.21.15.98:0/3372857007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff71015dce0 0x7ff7101603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.301+0000 7ff714837640 1 --2- 172.21.15.98:0/3372857007 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7100d0700 0x7ff7100b51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.301+0000 7ff714837640 1 -- 172.21.15.98:0/3372857007 >> 172.21.15.98:0/3372857007 conn(0x7ff7100b34e0 msgr2=0x7ff71006fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:41.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.301+0000 7ff714837640 1 -- 172.21.15.98:0/3372857007 shutdown_connections 2024-05-09T10:50:41.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.301+0000 7ff714837640 1 -- 172.21.15.98:0/3372857007 wait complete. 2024-05-09T10:50:41.306 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T10:50:07.396423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T10:50:17.323444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T10:50:18.482315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T10:50:41.306 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T10:50:41.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.415+0000 7f6ede01f640 1 Processor -- start 2024-05-09T10:50:41.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.415+0000 7f6ede01f640 1 -- start start 2024-05-09T10:50:41.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.416+0000 7f6ede01f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6ed815bce0 0x7f6ed80af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:41.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.416+0000 7f6ede01f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ed80d1320 0x7f6ed8165600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:41.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.416+0000 7f6ede01f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ed80b0080 con 0x7f6ed806abe0 2024-05-09T10:50:41.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.416+0000 7f6ede01f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ed806fe30 con 0x7f6ed815bce0 2024-05-09T10:50:41.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.416+0000 7f6ede01f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed806f900 con 0x7f6ed80d1320 2024-05-09T10:50:41.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.416+0000 7f6ed77fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f6ed806abe0 0x7f6ed806afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38116/0 (socket says 172.21.15.98:38116) 2024-05-09T10:50:41.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.416+0000 7f6ed77fe640 1 -- 172.21.15.98:0/683431723 learned_addr learned my addr 172.21.15.98:0/683431723 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:41.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6eceffd640 1 --2- 172.21.15.98:0/683431723 >> v2:172.21.15.105:3300/0 conn(0x7f6ed815bce0 0x7f6ed80af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:41.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6ed7fff640 1 --2- 172.21.15.98:0/683431723 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ed80d1320 0x7f6ed8165600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:41.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6ed6ffd640 1 -- 172.21.15.98:0/683431723 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3016695470 0 0) 0x7f6ec8001110 con 0x7f6ed806abe0 2024-05-09T10:50:41.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6ed6ffd640 1 -- 172.21.15.98:0/683431723 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ec4003520 con 0x7f6ed806abe0 2024-05-09T10:50:41.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6ed6ffd640 1 -- 172.21.15.98:0/683431723 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1641983797 0 0) 0x7f6ec4003520 con 0x7f6ed806abe0 2024-05-09T10:50:41.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6ed6ffd640 1 -- 172.21.15.98:0/683431723 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ed80d1320 msgr2=0x7f6ed8165600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:41.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6ed6ffd640 1 --2- 172.21.15.98:0/683431723 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ed80d1320 0x7f6ed8165600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6ed6ffd640 1 -- 172.21.15.98:0/683431723 >> v2:172.21.15.105:3300/0 conn(0x7f6ed815bce0 msgr2=0x7f6ed80af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:41.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6ed6ffd640 1 --2- 172.21.15.98:0/683431723 >> v2:172.21.15.105:3300/0 conn(0x7f6ed815bce0 0x7f6ed80af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6ed6ffd640 1 -- 172.21.15.98:0/683431723 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed806fb50 con 0x7f6ed806abe0 2024-05-09T10:50:41.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6ed6ffd640 1 -- 172.21.15.98:0/683431723 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f6ec8001840 con 0x7f6ed806abe0 2024-05-09T10:50:41.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6eceffd640 1 --2- 172.21.15.98:0/683431723 >> v2:172.21.15.105:3300/0 conn(0x7f6ed815bce0 0x7f6ed80af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:50:41.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6ed6ffd640 1 -- 172.21.15.98:0/683431723 >> v1:172.21.15.98:6789/0 conn(0x7f6ed806abe0 legacy=0x7f6ed806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:41.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6ed6ffd640 1 --2- 172.21.15.98:0/683431723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ec40039b0 0x7f6ec4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:41.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6ed6ffd640 1 --2- 172.21.15.98:0/683431723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec4004800 0x7f6ec4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:41.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6ed6ffd640 1 --2- 172.21.15.98:0/683431723 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ed80d1320 0x7f6ec40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:41.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6ed6ffd640 1 -- 172.21.15.98:0/683431723 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed806fe30 con 0x7f6ec4004800 2024-05-09T10:50:41.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6ed6ffd640 1 -- 172.21.15.98:0/683431723 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed806f900 con 0x7f6ec40039b0 2024-05-09T10:50:41.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6ed6ffd640 1 -- 172.21.15.98:0/683431723 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec400b9c0 con 0x7f6ed80d1320 2024-05-09T10:50:41.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6eceffd640 1 --2- 172.21.15.98:0/683431723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec4004800 0x7f6ec4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:41.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6ed77fe640 1 --2- 172.21.15.98:0/683431723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ec40039b0 0x7f6ec4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:41.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6ed7fff640 1 --2- 172.21.15.98:0/683431723 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ed80d1320 0x7f6ec40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:41.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6eceffd640 1 -- 172.21.15.98:0/683431723 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ed80d1320 msgr2=0x7f6ec40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:41.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6eceffd640 1 --2- 172.21.15.98:0/683431723 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ed80d1320 0x7f6ec40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6eceffd640 1 -- 172.21.15.98:0/683431723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ec40039b0 msgr2=0x7f6ec4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:41.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6eceffd640 1 --2- 172.21.15.98:0/683431723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ec40039b0 0x7f6ec4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.417+0000 7f6eceffd640 1 -- 172.21.15.98:0/683431723 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec4003520 con 0x7f6ec4004800 2024-05-09T10:50:41.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.418+0000 7f6eceffd640 1 --2- 172.21.15.98:0/683431723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec4004800 0x7f6ec4004c80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0009c90 tx=0x7f6ec4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa9236114013bf1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:41.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.418+0000 7f6ed6ffd640 1 -- 172.21.15.98:0/683431723 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0036070 con 0x7f6ec4004800 2024-05-09T10:50:41.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.418+0000 7f6ed6ffd640 1 -- 172.21.15.98:0/683431723 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec0031400 con 0x7f6ec4004800 2024-05-09T10:50:41.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.418+0000 7f6ed6ffd640 1 -- 172.21.15.98:0/683431723 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0009860 con 0x7f6ec4004800 2024-05-09T10:50:41.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.419+0000 7f6ede01f640 1 -- 172.21.15.98:0/683431723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec4004800 msgr2=0x7f6ec4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:41.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.419+0000 7f6ede01f640 1 --2- 172.21.15.98:0/683431723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec4004800 0x7f6ec4004c80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0009c90 tx=0x7f6ec4005830 comp rx=0 tx=0).stop 2024-05-09T10:50:41.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.419+0000 7f6ed7fff640 1 -- 172.21.15.98:0/683431723 reap_dead start 2024-05-09T10:50:41.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.419+0000 7f6ede01f640 1 -- 172.21.15.98:0/683431723 shutdown_connections 2024-05-09T10:50:41.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.419+0000 7f6ede01f640 1 -- 172.21.15.98:0/683431723 >> 172.21.15.98:0/683431723 conn(0x7f6ed80d0830 msgr2=0x7f6ed80d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:41.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.419+0000 7f6ede01f640 1 -- 172.21.15.98:0/683431723 shutdown_connections 2024-05-09T10:50:41.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.419+0000 7f6ede01f640 1 -- 172.21.15.98:0/683431723 wait complete. 2024-05-09T10:50:41.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.420+0000 7f6ede01f640 1 Processor -- start 2024-05-09T10:50:41.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.420+0000 7f6ede01f640 1 -- start start 2024-05-09T10:50:41.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.421+0000 7f6ede01f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec4004800 0x7f6ed813d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:41.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.421+0000 7f6ede01f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ed814b940 0x7f6ed8149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:41.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.421+0000 7f6ede01f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ed814a320 0x7f6ed806ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:41.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.421+0000 7f6ede01f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed806d170 con 0x7f6ec4004800 2024-05-09T10:50:41.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.421+0000 7f6ede01f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed80cd3f0 con 0x7f6ed814b940 2024-05-09T10:50:41.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.421+0000 7f6ede01f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed806d2f0 con 0x7f6ed814a320 2024-05-09T10:50:41.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.421+0000 7f6ed77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec4004800 0x7f6ed813d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:41.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.421+0000 7f6ed77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec4004800 0x7f6ed813d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48160/0 (socket says 172.21.15.98:48160) 2024-05-09T10:50:41.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.421+0000 7f6ed77fe640 1 -- 172.21.15.98:0/1454377267 learned_addr learned my addr 172.21.15.98:0/1454377267 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:41.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.421+0000 7f6ed77fe640 1 -- 172.21.15.98:0/1454377267 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ed814a320 msgr2=0x7f6ed806ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:41.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.421+0000 7f6ed77fe640 1 --2- 172.21.15.98:0/1454377267 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ed814a320 0x7f6ed806ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.421+0000 7f6ed77fe640 1 -- 172.21.15.98:0/1454377267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ed814b940 msgr2=0x7f6ed8149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:41.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.421+0000 7f6ed77fe640 1 --2- 172.21.15.98:0/1454377267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ed814b940 0x7f6ed8149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.421+0000 7f6ed77fe640 1 -- 172.21.15.98:0/1454377267 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed814a8a0 con 0x7f6ec4004800 2024-05-09T10:50:41.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.421+0000 7f6ed77fe640 1 --2- 172.21.15.98:0/1454377267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec4004800 0x7f6ed813d320 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80072c0 tx=0x7f6ec8007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:41.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.422+0000 7f6ed4ff9640 1 -- 172.21.15.98:0/1454377267 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec800f960 con 0x7f6ec4004800 2024-05-09T10:50:41.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.422+0000 7f6ed4ff9640 1 -- 172.21.15.98:0/1454377267 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec8018070 con 0x7f6ec4004800 2024-05-09T10:50:41.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.422+0000 7f6ede01f640 1 -- 172.21.15.98:0/1454377267 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ed814ab80 con 0x7f6ec4004800 2024-05-09T10:50:41.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.422+0000 7f6ed4ff9640 1 -- 172.21.15.98:0/1454377267 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec800fd40 con 0x7f6ec4004800 2024-05-09T10:50:41.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.422+0000 7f6ede01f640 1 -- 172.21.15.98:0/1454377267 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed814b140 con 0x7f6ec4004800 2024-05-09T10:50:41.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.423+0000 7f6ede01f640 1 -- 172.21.15.98:0/1454377267 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ed8072ec0 con 0x7f6ec4004800 2024-05-09T10:50:41.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.423+0000 7f6ed4ff9640 1 -- 172.21.15.98:0/1454377267 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ec801d460 con 0x7f6ec4004800 2024-05-09T10:50:41.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.423+0000 7f6ed4ff9640 1 --2- 172.21.15.98:0/1454377267 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6eb806b4d0 0x7f6eb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:41.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.424+0000 7f6ed4ff9640 1 -- 172.21.15.98:0/1454377267 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ec808a160 con 0x7f6ec4004800 2024-05-09T10:50:41.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.424+0000 7f6eceffd640 1 --2- 172.21.15.98:0/1454377267 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6eb806b4d0 0x7f6eb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:41.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.424+0000 7f6eceffd640 1 --2- 172.21.15.98:0/1454377267 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6eb806b4d0 0x7f6eb806d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0033000 tx=0x7f6ec002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:41.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.428+0000 7f6ed4ff9640 1 -- 172.21.15.98:0/1454377267 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6ed80b0080 con 0x7f6ec4004800 2024-05-09T10:50:41.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.600+0000 7f6ede01f640 1 -- 172.21.15.98:0/1454377267 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6ed815c8c0 con 0x7f6ec4004800 2024-05-09T10:50:41.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.600+0000 7f6ed4ff9640 1 -- 172.21.15.98:0/1454377267 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6ec8059070 con 0x7f6ec4004800 2024-05-09T10:50:41.594 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-05-09T10:50:41.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.602+0000 7f6ede01f640 1 -- 172.21.15.98:0/1454377267 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6eb806b4d0 msgr2=0x7f6eb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:41.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.602+0000 7f6ede01f640 1 --2- 172.21.15.98:0/1454377267 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6eb806b4d0 0x7f6eb806d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0033000 tx=0x7f6ec002e000 comp rx=0 tx=0).stop 2024-05-09T10:50:41.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.602+0000 7f6ede01f640 1 -- 172.21.15.98:0/1454377267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec4004800 msgr2=0x7f6ed813d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:41.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.602+0000 7f6ede01f640 1 --2- 172.21.15.98:0/1454377267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec4004800 0x7f6ed813d320 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80072c0 tx=0x7f6ec8007780 comp rx=0 tx=0).stop 2024-05-09T10:50:41.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.603+0000 7f6ede01f640 1 -- 172.21.15.98:0/1454377267 shutdown_connections 2024-05-09T10:50:41.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.603+0000 7f6ede01f640 1 --2- 172.21.15.98:0/1454377267 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6eb806b4d0 0x7f6eb806d990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.603+0000 7f6ede01f640 1 --2- 172.21.15.98:0/1454377267 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ed814a320 0x7f6ed806ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.603+0000 7f6ede01f640 1 --2- 172.21.15.98:0/1454377267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ed814b940 0x7f6ed8149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.603+0000 7f6ede01f640 1 --2- 172.21.15.98:0/1454377267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec4004800 0x7f6ed813d320 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.603+0000 7f6ede01f640 1 -- 172.21.15.98:0/1454377267 >> 172.21.15.98:0/1454377267 conn(0x7f6ed80d0830 msgr2=0x7f6ed806eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:41.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.603+0000 7f6ede01f640 1 -- 172.21.15.98:0/1454377267 shutdown_connections 2024-05-09T10:50:41.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.603+0000 7f6ede01f640 1 -- 172.21.15.98:0/1454377267 wait complete. 2024-05-09T10:50:41.607 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T10:50:41.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.717+0000 7f25764b6640 1 Processor -- start 2024-05-09T10:50:41.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.717+0000 7f25764b6640 1 -- start start 2024-05-09T10:50:41.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.718+0000 7f25764b6640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f25700b2d40 0x7f257006c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:41.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.718+0000 7f25764b6640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25700d0090 con 0x7f25700cfbb0 2024-05-09T10:50:41.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.718+0000 7f25764b6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25700d0370 con 0x7f2570162240 2024-05-09T10:50:41.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.718+0000 7f25764b6640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25700d0650 con 0x7f25700b2d40 2024-05-09T10:50:41.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.718+0000 7f256ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f25700b2d40 0x7f257006c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:41.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.718+0000 7f2567fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f25700cfbb0 0x7f2570161b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38118/0 (socket says 172.21.15.98:38118) 2024-05-09T10:50:41.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.718+0000 7f2567fff640 1 -- 172.21.15.98:0/2222300508 learned_addr learned my addr 172.21.15.98:0/2222300508 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:41.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.718+0000 7f256ffff640 1 -- 172.21.15.98:0/2222300508 >> v1:172.21.15.105:6789/0 conn(0x7f2570162240 legacy=0x7f257015d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:41.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.719+0000 7f256ffff640 1 -- 172.21.15.98:0/2222300508 >> v1:172.21.15.98:6789/0 conn(0x7f25700cfbb0 legacy=0x7f2570161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:41.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.719+0000 7f256f7fe640 1 -- 172.21.15.98:0/2222300508 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4250613851 0 0) 0x7f25700d0090 con 0x7f25700cfbb0 2024-05-09T10:50:41.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.719+0000 7f256ffff640 1 -- 172.21.15.98:0/2222300508 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25700d0930 con 0x7f25700b2d40 2024-05-09T10:50:41.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.719+0000 7f256ffff640 1 --2- 172.21.15.98:0/2222300508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25700b2d40 0x7f257006c910 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f25600097a0 tx=0x7f256002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348e2ab4f0f3b087 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:41.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.719+0000 7f256f7fe640 1 -- 172.21.15.98:0/2222300508 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2560037070 con 0x7f25700b2d40 2024-05-09T10:50:41.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.719+0000 7f256f7fe640 1 -- 172.21.15.98:0/2222300508 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2560032440 con 0x7f25700b2d40 2024-05-09T10:50:41.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.719+0000 7f256f7fe640 1 -- 172.21.15.98:0/2222300508 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2560032810 con 0x7f25700b2d40 2024-05-09T10:50:41.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.720+0000 7f25764b6640 1 -- 172.21.15.98:0/2222300508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25700b2d40 msgr2=0x7f257006c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:41.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.720+0000 7f25764b6640 1 --2- 172.21.15.98:0/2222300508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25700b2d40 0x7f257006c910 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f25600097a0 tx=0x7f256002a1a0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.720+0000 7f25764b6640 1 -- 172.21.15.98:0/2222300508 shutdown_connections 2024-05-09T10:50:41.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.720+0000 7f25764b6640 1 --2- 172.21.15.98:0/2222300508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25700b2d40 0x7f257006c910 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.720+0000 7f25764b6640 1 -- 172.21.15.98:0/2222300508 >> 172.21.15.98:0/2222300508 conn(0x7f25700d1970 msgr2=0x7f25700d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:41.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.720+0000 7f25764b6640 1 -- 172.21.15.98:0/2222300508 shutdown_connections 2024-05-09T10:50:41.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.720+0000 7f25764b6640 1 -- 172.21.15.98:0/2222300508 wait complete. 2024-05-09T10:50:41.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.721+0000 7f25764b6640 1 Processor -- start 2024-05-09T10:50:41.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.721+0000 7f25764b6640 1 -- start start 2024-05-09T10:50:41.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.722+0000 7f25764b6640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25700b2d40 0x7f2570155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:41.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.722+0000 7f25764b6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25700cfbb0 0x7f2570157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:41.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.722+0000 7f25764b6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2570162240 0x7f2570159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:41.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.722+0000 7f25764b6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25700b32a0 con 0x7f2570162240 2024-05-09T10:50:41.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.722+0000 7f25764b6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25700b3120 con 0x7f25700cfbb0 2024-05-09T10:50:41.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.722+0000 7f25764b6640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f257006ce50 con 0x7f25700b2d40 2024-05-09T10:50:41.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.722+0000 7f2574a53640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2570162240 0x7f2570159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:41.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.722+0000 7f2574a53640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2570162240 0x7f2570159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48184/0 (socket says 172.21.15.98:48184) 2024-05-09T10:50:41.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.722+0000 7f2574a53640 1 -- 172.21.15.98:0/3925372658 learned_addr learned my addr 172.21.15.98:0/3925372658 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:41.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.722+0000 7f256ffff640 1 --2- 172.21.15.98:0/3925372658 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25700b2d40 0x7f2570155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:41.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.722+0000 7f2567fff640 1 --2- 172.21.15.98:0/3925372658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25700cfbb0 0x7f2570157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:41.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.722+0000 7f2574a53640 1 -- 172.21.15.98:0/3925372658 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25700b2d40 msgr2=0x7f2570155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:41.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.722+0000 7f2574a53640 1 --2- 172.21.15.98:0/3925372658 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25700b2d40 0x7f2570155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.722+0000 7f2574a53640 1 -- 172.21.15.98:0/3925372658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25700cfbb0 msgr2=0x7f2570157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:41.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.722+0000 7f2574a53640 1 --2- 172.21.15.98:0/3925372658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25700cfbb0 0x7f2570157f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.722+0000 7f2574a53640 1 -- 172.21.15.98:0/3925372658 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25701563e0 con 0x7f2570162240 2024-05-09T10:50:41.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.722+0000 7f2574a53640 1 --2- 172.21.15.98:0/3925372658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2570162240 0x7f2570159a20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f256800c680 tx=0x7f256800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:41.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.723+0000 7f256d7fa640 1 -- 172.21.15.98:0/3925372658 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2568003b70 con 0x7f2570162240 2024-05-09T10:50:41.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.723+0000 7f256d7fa640 1 -- 172.21.15.98:0/3925372658 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2568003d10 con 0x7f2570162240 2024-05-09T10:50:41.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.723+0000 7f25764b6640 1 -- 172.21.15.98:0/3925372658 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2570158a20 con 0x7f2570162240 2024-05-09T10:50:41.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.723+0000 7f256d7fa640 1 -- 172.21.15.98:0/3925372658 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2568005310 con 0x7f2570162240 2024-05-09T10:50:41.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.723+0000 7f25764b6640 1 -- 172.21.15.98:0/3925372658 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25700af7b0 con 0x7f2570162240 2024-05-09T10:50:41.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.724+0000 7f25764b6640 1 -- 172.21.15.98:0/3925372658 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f257009fa40 con 0x7f2570162240 2024-05-09T10:50:41.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.724+0000 7f256d7fa640 1 -- 172.21.15.98:0/3925372658 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f25680054b0 con 0x7f2570162240 2024-05-09T10:50:41.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.724+0000 7f256d7fa640 1 --2- 172.21.15.98:0/3925372658 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f255006b500 0x7f255006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:41.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.725+0000 7f256d7fa640 1 -- 172.21.15.98:0/3925372658 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2568089e70 con 0x7f2570162240 2024-05-09T10:50:41.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.725+0000 7f256ffff640 1 --2- 172.21.15.98:0/3925372658 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f255006b500 0x7f255006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:41.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.725+0000 7f256ffff640 1 --2- 172.21.15.98:0/3925372658 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f255006b500 0x7f255006d9c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2560034000 tx=0x7f256002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:41.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.729+0000 7f256d7fa640 1 -- 172.21.15.98:0/3925372658 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2568058730 con 0x7f2570162240 2024-05-09T10:50:41.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.901+0000 7f25764b6640 1 -- 172.21.15.98:0/3925372658 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f25700727e0 con 0x7f2570162240 2024-05-09T10:50:41.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.902+0000 7f256d7fa640 1 -- 172.21.15.98:0/3925372658 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f256805c7d0 con 0x7f2570162240 2024-05-09T10:50:41.895 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2024-05-09T10:50:41.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.904+0000 7f25764b6640 1 -- 172.21.15.98:0/3925372658 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f255006b500 msgr2=0x7f255006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:41.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.904+0000 7f25764b6640 1 --2- 172.21.15.98:0/3925372658 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f255006b500 0x7f255006d9c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2560034000 tx=0x7f256002e000 comp rx=0 tx=0).stop 2024-05-09T10:50:41.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.904+0000 7f25764b6640 1 -- 172.21.15.98:0/3925372658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2570162240 msgr2=0x7f2570159a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:41.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.904+0000 7f25764b6640 1 --2- 172.21.15.98:0/3925372658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2570162240 0x7f2570159a20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f256800c680 tx=0x7f256800cb40 comp rx=0 tx=0).stop 2024-05-09T10:50:41.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.904+0000 7f25764b6640 1 -- 172.21.15.98:0/3925372658 shutdown_connections 2024-05-09T10:50:41.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.904+0000 7f25764b6640 1 --2- 172.21.15.98:0/3925372658 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f255006b500 0x7f255006d9c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.904+0000 7f25764b6640 1 --2- 172.21.15.98:0/3925372658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2570162240 0x7f2570159a20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.904+0000 7f25764b6640 1 --2- 172.21.15.98:0/3925372658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25700cfbb0 0x7f2570157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.904+0000 7f25764b6640 1 --2- 172.21.15.98:0/3925372658 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25700b2d40 0x7f2570155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:41.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.904+0000 7f25764b6640 1 -- 172.21.15.98:0/3925372658 >> 172.21.15.98:0/3925372658 conn(0x7f25700d1970 msgr2=0x7f2570156720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:41.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.904+0000 7f25764b6640 1 -- 172.21.15.98:0/3925372658 shutdown_connections 2024-05-09T10:50:41.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:41.904+0000 7f25764b6640 1 -- 172.21.15.98:0/3925372658 wait complete. 2024-05-09T10:50:41.908 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T10:50:42.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.017+0000 7fc27a153640 1 Processor -- start 2024-05-09T10:50:42.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.017+0000 7fc27a153640 1 -- start start 2024-05-09T10:50:42.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.017+0000 7fc27a153640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc2740cf950 0x7fc2740cec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.017+0000 7fc27a153640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2740b0a60 con 0x7fc27415e0c0 2024-05-09T10:50:42.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.017+0000 7fc27a153640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2740b0d00 con 0x7fc2740d1970 2024-05-09T10:50:42.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.017+0000 7fc27a153640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc27415e4a0 con 0x7fc2740cf950 2024-05-09T10:50:42.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.018+0000 7fc2737fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc27415e0c0 0x7fc2740d11d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38132/0 (socket says 172.21.15.98:38132) 2024-05-09T10:50:42.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.018+0000 7fc2737fe640 1 -- 172.21.15.98:0/355425112 learned_addr learned my addr 172.21.15.98:0/355425112 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:42.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.018+0000 7fc272ffd640 1 --2- 172.21.15.98:0/355425112 >> v2:172.21.15.105:3300/0 conn(0x7fc2740cf950 0x7fc2740cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:42.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.018+0000 7fc2727fc640 1 -- 172.21.15.98:0/355425112 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3121877639 0 0) 0x7fc2740b0a60 con 0x7fc27415e0c0 2024-05-09T10:50:42.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.018+0000 7fc2727fc640 1 -- 172.21.15.98:0/355425112 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc25c003520 con 0x7fc27415e0c0 2024-05-09T10:50:42.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.018+0000 7fc272ffd640 1 -- 172.21.15.98:0/355425112 >> v1:172.21.15.105:6789/0 conn(0x7fc2740d1970 legacy=0x7fc2740b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:42.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.018+0000 7fc272ffd640 1 -- 172.21.15.98:0/355425112 >> v1:172.21.15.98:6789/0 conn(0x7fc27415e0c0 legacy=0x7fc2740d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:42.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.018+0000 7fc272ffd640 1 -- 172.21.15.98:0/355425112 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2740b0fe0 con 0x7fc2740cf950 2024-05-09T10:50:42.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.019+0000 7fc272ffd640 1 --2- 172.21.15.98:0/355425112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2740cf950 0x7fc2740cec10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc2680097a0 tx=0x7fc26802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a04c92f405580826 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:42.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.019+0000 7fc2727fc640 1 -- 172.21.15.98:0/355425112 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268037070 con 0x7fc2740cf950 2024-05-09T10:50:42.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.019+0000 7fc2727fc640 1 -- 172.21.15.98:0/355425112 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc268032440 con 0x7fc2740cf950 2024-05-09T10:50:42.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.019+0000 7fc2727fc640 1 -- 172.21.15.98:0/355425112 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2680327f0 con 0x7fc2740cf950 2024-05-09T10:50:42.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.020+0000 7fc27a153640 1 -- 172.21.15.98:0/355425112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2740cf950 msgr2=0x7fc2740cec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:42.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.020+0000 7fc27a153640 1 --2- 172.21.15.98:0/355425112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2740cf950 0x7fc2740cec10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc2680097a0 tx=0x7fc26802a1a0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.020+0000 7fc27a153640 1 -- 172.21.15.98:0/355425112 shutdown_connections 2024-05-09T10:50:42.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.020+0000 7fc27a153640 1 --2- 172.21.15.98:0/355425112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2740cf950 0x7fc2740cec10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.020+0000 7fc27a153640 1 -- 172.21.15.98:0/355425112 >> 172.21.15.98:0/355425112 conn(0x7fc2740b6db0 msgr2=0x7fc2740b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:42.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.020+0000 7fc27a153640 1 -- 172.21.15.98:0/355425112 shutdown_connections 2024-05-09T10:50:42.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.020+0000 7fc27a153640 1 -- 172.21.15.98:0/355425112 wait complete. 2024-05-09T10:50:42.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.021+0000 7fc27a153640 1 Processor -- start 2024-05-09T10:50:42.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.021+0000 7fc27a153640 1 -- start start 2024-05-09T10:50:42.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.022+0000 7fc27a153640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2740cf950 0x7fc2740b5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.022+0000 7fc27a153640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc2740d1970 0x7fc274160720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.022+0000 7fc27a153640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc27415e0c0 0x7fc274160c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.022+0000 7fc27a153640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2740cf150 con 0x7fc27415e0c0 2024-05-09T10:50:42.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.022+0000 7fc27a153640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2740b3180 con 0x7fc2740cf950 2024-05-09T10:50:42.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.022+0000 7fc27a153640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2740b19d0 con 0x7fc2740d1970 2024-05-09T10:50:42.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.022+0000 7fc273fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc27415e0c0 0x7fc274160c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:42.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.022+0000 7fc273fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc27415e0c0 0x7fc274160c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48194/0 (socket says 172.21.15.98:48194) 2024-05-09T10:50:42.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.022+0000 7fc273fff640 1 -- 172.21.15.98:0/3153618120 learned_addr learned my addr 172.21.15.98:0/3153618120 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:42.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.022+0000 7fc2737fe640 1 --2- 172.21.15.98:0/3153618120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2740cf950 0x7fc2740b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:42.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.022+0000 7fc272ffd640 1 --2- 172.21.15.98:0/3153618120 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc2740d1970 0x7fc274160720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:42.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.022+0000 7fc273fff640 1 -- 172.21.15.98:0/3153618120 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc2740d1970 msgr2=0x7fc274160720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:42.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.022+0000 7fc273fff640 1 --2- 172.21.15.98:0/3153618120 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc2740d1970 0x7fc274160720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.022+0000 7fc273fff640 1 -- 172.21.15.98:0/3153618120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2740cf950 msgr2=0x7fc2740b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:42.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.022+0000 7fc273fff640 1 --2- 172.21.15.98:0/3153618120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2740cf950 0x7fc2740b5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.022+0000 7fc273fff640 1 -- 172.21.15.98:0/3153618120 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2741f3b80 con 0x7fc27415e0c0 2024-05-09T10:50:42.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.022+0000 7fc273fff640 1 --2- 172.21.15.98:0/3153618120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc27415e0c0 0x7fc274160c60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00c7a0 tx=0x7fc26c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:42.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.022+0000 7fc25bfff640 1 -- 172.21.15.98:0/3153618120 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c003b70 con 0x7fc27415e0c0 2024-05-09T10:50:42.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.022+0000 7fc25bfff640 1 -- 172.21.15.98:0/3153618120 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26c003d30 con 0x7fc27415e0c0 2024-05-09T10:50:42.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.022+0000 7fc27a153640 1 -- 172.21.15.98:0/3153618120 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2741f3ec0 con 0x7fc27415e0c0 2024-05-09T10:50:42.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.022+0000 7fc25bfff640 1 -- 172.21.15.98:0/3153618120 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c00aa90 con 0x7fc27415e0c0 2024-05-09T10:50:42.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.022+0000 7fc27a153640 1 -- 172.21.15.98:0/3153618120 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2741611a0 con 0x7fc27415e0c0 2024-05-09T10:50:42.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.023+0000 7fc27a153640 1 -- 172.21.15.98:0/3153618120 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2741f4910 con 0x7fc27415e0c0 2024-05-09T10:50:42.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.024+0000 7fc25bfff640 1 -- 172.21.15.98:0/3153618120 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc26c00ac30 con 0x7fc27415e0c0 2024-05-09T10:50:42.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.024+0000 7fc25bfff640 1 --2- 172.21.15.98:0/3153618120 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc26006b4d0 0x7fc26006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.025+0000 7fc2737fe640 1 --2- 172.21.15.98:0/3153618120 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc26006b4d0 0x7fc26006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:42.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.025+0000 7fc25bfff640 1 -- 172.21.15.98:0/3153618120 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc26c08a0b0 con 0x7fc27415e0c0 2024-05-09T10:50:42.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.025+0000 7fc2737fe640 1 --2- 172.21.15.98:0/3153618120 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc26006b4d0 0x7fc26006d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc2640011e0 tx=0x7fc264009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:42.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.029+0000 7fc25bfff640 1 -- 172.21.15.98:0/3153618120 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc2741f3ec0 con 0x7fc27415e0c0 2024-05-09T10:50:42.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.200+0000 7fc27a153640 1 -- 172.21.15.98:0/3153618120 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc27409fbb0 con 0x7fc27415e0c0 2024-05-09T10:50:42.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.200+0000 7fc25bfff640 1 -- 172.21.15.98:0/3153618120 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc26c058fc0 con 0x7fc27415e0c0 2024-05-09T10:50:42.194 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2024-05-09T10:50:42.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.202+0000 7fc27a153640 1 -- 172.21.15.98:0/3153618120 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc26006b4d0 msgr2=0x7fc26006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:42.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.202+0000 7fc27a153640 1 --2- 172.21.15.98:0/3153618120 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc26006b4d0 0x7fc26006d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc2640011e0 tx=0x7fc264009040 comp rx=0 tx=0).stop 2024-05-09T10:50:42.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.202+0000 7fc27a153640 1 -- 172.21.15.98:0/3153618120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc27415e0c0 msgr2=0x7fc274160c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:42.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.202+0000 7fc27a153640 1 --2- 172.21.15.98:0/3153618120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc27415e0c0 0x7fc274160c60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00c7a0 tx=0x7fc26c00cc60 comp rx=0 tx=0).stop 2024-05-09T10:50:42.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.202+0000 7fc27a153640 1 -- 172.21.15.98:0/3153618120 shutdown_connections 2024-05-09T10:50:42.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.202+0000 7fc27a153640 1 --2- 172.21.15.98:0/3153618120 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc26006b4d0 0x7fc26006d990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.202+0000 7fc27a153640 1 --2- 172.21.15.98:0/3153618120 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc27415e0c0 0x7fc274160c60 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.202+0000 7fc27a153640 1 --2- 172.21.15.98:0/3153618120 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc2740d1970 0x7fc274160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.202+0000 7fc27a153640 1 --2- 172.21.15.98:0/3153618120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2740cf950 0x7fc2740b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.202+0000 7fc27a153640 1 -- 172.21.15.98:0/3153618120 >> 172.21.15.98:0/3153618120 conn(0x7fc2740b6db0 msgr2=0x7fc27406e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:42.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.202+0000 7fc27a153640 1 -- 172.21.15.98:0/3153618120 shutdown_connections 2024-05-09T10:50:42.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.202+0000 7fc27a153640 1 -- 172.21.15.98:0/3153618120 wait complete. 2024-05-09T10:50:42.207 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:42.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.313+0000 7f650fa2a640 1 Processor -- start 2024-05-09T10:50:42.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.313+0000 7f650fa2a640 1 -- start start 2024-05-09T10:50:42.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.314+0000 7f650fa2a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f650815f4e0 0x7f65080ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.314+0000 7f650fa2a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f65080afe40 0x7f65080b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.314+0000 7f650fa2a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65080aeda0 con 0x7f65080b0810 2024-05-09T10:50:42.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.314+0000 7f650fa2a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65080af080 con 0x7f650815f4e0 2024-05-09T10:50:42.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.314+0000 7f650fa2a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65080af360 con 0x7f65080afe40 2024-05-09T10:50:42.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.314+0000 7f650d7c6640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f650815f4e0 0x7f65080ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:42.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.314+0000 7f650d7c6640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f650815f4e0 0x7f65080ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48206/0 (socket says 172.21.15.98:48206) 2024-05-09T10:50:42.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.314+0000 7f650d7c6640 1 -- 172.21.15.98:0/718751094 learned_addr learned my addr 172.21.15.98:0/718751094 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:42.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.315+0000 7f650d7c6640 1 -- 172.21.15.98:0/718751094 >> v2:172.21.15.105:3300/0 conn(0x7f65080afe40 msgr2=0x7f65080b02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:42.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.315+0000 7f650d7c6640 1 --2- 172.21.15.98:0/718751094 >> v2:172.21.15.105:3300/0 conn(0x7f65080afe40 0x7f65080b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.315+0000 7f650d7c6640 1 -- 172.21.15.98:0/718751094 >> v1:172.21.15.105:6789/0 conn(0x7f65080b0810 legacy=0x7f6508164c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:42.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.315+0000 7f650d7c6640 1 -- 172.21.15.98:0/718751094 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65080af640 con 0x7f650815f4e0 2024-05-09T10:50:42.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.315+0000 7f650d7c6640 1 --2- 172.21.15.98:0/718751094 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f650815f4e0 0x7f65080ae7c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6508071990 tx=0x7f650402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8da5023c62592a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:42.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.315+0000 7f64fffff640 1 -- 172.21.15.98:0/718751094 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6504036070 con 0x7f650815f4e0 2024-05-09T10:50:42.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.316+0000 7f64fffff640 1 -- 172.21.15.98:0/718751094 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6504031600 con 0x7f650815f4e0 2024-05-09T10:50:42.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.316+0000 7f64fffff640 1 -- 172.21.15.98:0/718751094 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65040319d0 con 0x7f650815f4e0 2024-05-09T10:50:42.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.316+0000 7f650fa2a640 1 -- 172.21.15.98:0/718751094 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f650815f4e0 msgr2=0x7f65080ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:42.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.316+0000 7f650fa2a640 1 --2- 172.21.15.98:0/718751094 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f650815f4e0 0x7f65080ae7c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6508071990 tx=0x7f650402a660 comp rx=0 tx=0).stop 2024-05-09T10:50:42.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.316+0000 7f650fa2a640 1 -- 172.21.15.98:0/718751094 shutdown_connections 2024-05-09T10:50:42.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.316+0000 7f650fa2a640 1 --2- 172.21.15.98:0/718751094 >> v2:172.21.15.105:3300/0 conn(0x7f65080afe40 0x7f65080b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.316+0000 7f650fa2a640 1 --2- 172.21.15.98:0/718751094 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f650815f4e0 0x7f65080ae7c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.316+0000 7f650fa2a640 1 -- 172.21.15.98:0/718751094 >> 172.21.15.98:0/718751094 conn(0x7f65080b34e0 msgr2=0x7f65080b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:42.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.316+0000 7f650fa2a640 1 -- 172.21.15.98:0/718751094 shutdown_connections 2024-05-09T10:50:42.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.316+0000 7f650fa2a640 1 -- 172.21.15.98:0/718751094 wait complete. 2024-05-09T10:50:42.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.317+0000 7f650fa2a640 1 Processor -- start 2024-05-09T10:50:42.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.317+0000 7f650fa2a640 1 -- start start 2024-05-09T10:50:42.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.318+0000 7f650fa2a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65080afe40 0x7f650814bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.318+0000 7f650fa2a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65080b0810 0x7f65080b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.318+0000 7f650fa2a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f650815f4e0 0x7f650815cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.318+0000 7f650fa2a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650815f8c0 con 0x7f65080afe40 2024-05-09T10:50:42.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.318+0000 7f650fa2a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65080cbcf0 con 0x7f650815f4e0 2024-05-09T10:50:42.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.318+0000 7f650fa2a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65080b0c20 con 0x7f65080b0810 2024-05-09T10:50:42.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.318+0000 7f650d7c6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65080afe40 0x7f650814bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:42.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.318+0000 7f650d7c6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65080afe40 0x7f650814bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48210/0 (socket says 172.21.15.98:48210) 2024-05-09T10:50:42.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.318+0000 7f650d7c6640 1 -- 172.21.15.98:0/439347495 learned_addr learned my addr 172.21.15.98:0/439347495 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:42.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.318+0000 7f650dfc7640 1 --2- 172.21.15.98:0/439347495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f650815f4e0 0x7f650815cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:42.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.318+0000 7f650cfc5640 1 --2- 172.21.15.98:0/439347495 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65080b0810 0x7f65080b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:42.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.318+0000 7f650d7c6640 1 -- 172.21.15.98:0/439347495 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65080b0810 msgr2=0x7f65080b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:42.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.318+0000 7f650d7c6640 1 --2- 172.21.15.98:0/439347495 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65080b0810 0x7f65080b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.318+0000 7f650d7c6640 1 -- 172.21.15.98:0/439347495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f650815f4e0 msgr2=0x7f650815cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:42.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.318+0000 7f650d7c6640 1 --2- 172.21.15.98:0/439347495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f650815f4e0 0x7f650815cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.318+0000 7f650d7c6640 1 -- 172.21.15.98:0/439347495 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65080b5c70 con 0x7f65080afe40 2024-05-09T10:50:42.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.318+0000 7f650cfc5640 1 --2- 172.21.15.98:0/439347495 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65080b0810 0x7f65080b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:42.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.318+0000 7f650dfc7640 1 --2- 172.21.15.98:0/439347495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f650815f4e0 0x7f650815cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:42.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.319+0000 7f650d7c6640 1 --2- 172.21.15.98:0/439347495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65080afe40 0x7f650814bce0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f65040093e0 tx=0x7f6504004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:42.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.319+0000 7f64fdffb640 1 -- 172.21.15.98:0/439347495 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6504041020 con 0x7f65080afe40 2024-05-09T10:50:42.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.319+0000 7f64fdffb640 1 -- 172.21.15.98:0/439347495 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6504007820 con 0x7f65080afe40 2024-05-09T10:50:42.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.319+0000 7f650fa2a640 1 -- 172.21.15.98:0/439347495 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65080b5f50 con 0x7f65080afe40 2024-05-09T10:50:42.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.319+0000 7f64fdffb640 1 -- 172.21.15.98:0/439347495 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6504036040 con 0x7f65080afe40 2024-05-09T10:50:42.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.319+0000 7f650fa2a640 1 -- 172.21.15.98:0/439347495 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65080b6510 con 0x7f65080afe40 2024-05-09T10:50:42.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.320+0000 7f650fa2a640 1 -- 172.21.15.98:0/439347495 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6508068a60 con 0x7f65080afe40 2024-05-09T10:50:42.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.321+0000 7f64fdffb640 1 -- 172.21.15.98:0/439347495 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6504044020 con 0x7f65080afe40 2024-05-09T10:50:42.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.321+0000 7f64fdffb640 1 --2- 172.21.15.98:0/439347495 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f64ec06b500 0x7f64ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.321+0000 7f64fdffb640 1 -- 172.21.15.98:0/439347495 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65040ab0e0 con 0x7f65080afe40 2024-05-09T10:50:42.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.321+0000 7f650cfc5640 1 --2- 172.21.15.98:0/439347495 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f64ec06b500 0x7f64ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:42.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.321+0000 7f650cfc5640 1 --2- 172.21.15.98:0/439347495 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f64ec06b500 0x7f64ec06d9c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6508156a60 tx=0x7f64f8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:42.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.325+0000 7f64fdffb640 1 -- 172.21.15.98:0/439347495 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6508068a60 con 0x7f65080afe40 2024-05-09T10:50:42.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.497+0000 7f650fa2a640 1 -- 172.21.15.98:0/439347495 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65080cbad0 con 0x7f65080afe40 2024-05-09T10:50:42.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.498+0000 7f64fdffb640 1 -- 172.21.15.98:0/439347495 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f650404b030 con 0x7f65080afe40 2024-05-09T10:50:42.492 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:42.492 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":20,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:21.176419+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:50:17.323444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T10:50:18.482315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:42.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.500+0000 7f650fa2a640 1 -- 172.21.15.98:0/439347495 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f64ec06b500 msgr2=0x7f64ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:42.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.500+0000 7f650fa2a640 1 --2- 172.21.15.98:0/439347495 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f64ec06b500 0x7f64ec06d9c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6508156a60 tx=0x7f64f8008040 comp rx=0 tx=0).stop 2024-05-09T10:50:42.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.500+0000 7f650fa2a640 1 -- 172.21.15.98:0/439347495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65080afe40 msgr2=0x7f650814bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:42.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.500+0000 7f650fa2a640 1 --2- 172.21.15.98:0/439347495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65080afe40 0x7f650814bce0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f65040093e0 tx=0x7f6504004a40 comp rx=0 tx=0).stop 2024-05-09T10:50:42.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.500+0000 7f650fa2a640 1 -- 172.21.15.98:0/439347495 shutdown_connections 2024-05-09T10:50:42.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.500+0000 7f650fa2a640 1 --2- 172.21.15.98:0/439347495 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f64ec06b500 0x7f64ec06d9c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.500+0000 7f650fa2a640 1 --2- 172.21.15.98:0/439347495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f650815f4e0 0x7f650815cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.500+0000 7f650fa2a640 1 --2- 172.21.15.98:0/439347495 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65080b0810 0x7f65080b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.500+0000 7f650fa2a640 1 --2- 172.21.15.98:0/439347495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65080afe40 0x7f650814bce0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.500+0000 7f650fa2a640 1 -- 172.21.15.98:0/439347495 >> 172.21.15.98:0/439347495 conn(0x7f65080b34e0 msgr2=0x7f65080ce450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:42.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.500+0000 7f650fa2a640 1 -- 172.21.15.98:0/439347495 shutdown_connections 2024-05-09T10:50:42.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.500+0000 7f650fa2a640 1 -- 172.21.15.98:0/439347495 wait complete. 2024-05-09T10:50:42.505 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T10:50:07.396423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T10:50:17.323444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T10:50:18.482315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T10:50:42.505 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T10:50:42.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.614+0000 7f6aeb222640 1 Processor -- start 2024-05-09T10:50:42.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.614+0000 7f6aeb222640 1 -- start start 2024-05-09T10:50:42.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.614+0000 7f6aeb222640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6ae40b4d80 0x7f6ae40713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.614+0000 7f6aeb222640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ae4071930 0x7f6ae406d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.614+0000 7f6aeb222640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae40b01a0 con 0x7f6ae41584e0 2024-05-09T10:50:42.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.614+0000 7f6aeb222640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ae4071d70 con 0x7f6ae40b4d80 2024-05-09T10:50:42.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.614+0000 7f6aeb222640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae40b42a0 con 0x7f6ae4071930 2024-05-09T10:50:42.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae8fbe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f6ae41584e0 0x7f6ae40af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38134/0 (socket says 172.21.15.98:38134) 2024-05-09T10:50:42.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae8fbe640 1 -- 172.21.15.98:0/1875789099 learned_addr learned my addr 172.21.15.98:0/1875789099 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:42.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae3fff640 1 --2- 172.21.15.98:0/1875789099 >> v2:172.21.15.105:3300/0 conn(0x7f6ae40b4d80 0x7f6ae40713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:42.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae97bf640 1 --2- 172.21.15.98:0/1875789099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ae4071930 0x7f6ae406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:42.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae37fe640 1 -- 172.21.15.98:0/1875789099 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2036951330 0 0) 0x7f6ae40b01a0 con 0x7f6ae41584e0 2024-05-09T10:50:42.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae37fe640 1 -- 172.21.15.98:0/1875789099 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ad0003520 con 0x7f6ae41584e0 2024-05-09T10:50:42.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae37fe640 1 -- 172.21.15.98:0/1875789099 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2956343457 0 0) 0x7f6ad0003520 con 0x7f6ae41584e0 2024-05-09T10:50:42.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae37fe640 1 -- 172.21.15.98:0/1875789099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ae4071930 msgr2=0x7f6ae406d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:42.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae37fe640 1 --2- 172.21.15.98:0/1875789099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ae4071930 0x7f6ae406d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae37fe640 1 -- 172.21.15.98:0/1875789099 >> v2:172.21.15.105:3300/0 conn(0x7f6ae40b4d80 msgr2=0x7f6ae40713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:42.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae37fe640 1 --2- 172.21.15.98:0/1875789099 >> v2:172.21.15.105:3300/0 conn(0x7f6ae40b4d80 0x7f6ae40713f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae37fe640 1 -- 172.21.15.98:0/1875789099 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae40b4580 con 0x7f6ae41584e0 2024-05-09T10:50:42.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae37fe640 1 -- 172.21.15.98:0/1875789099 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f6ad40019f0 con 0x7f6ae41584e0 2024-05-09T10:50:42.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae37fe640 1 -- 172.21.15.98:0/1875789099 >> v1:172.21.15.98:6789/0 conn(0x7f6ae41584e0 legacy=0x7f6ae40af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:42.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae37fe640 1 --2- 172.21.15.98:0/1875789099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ad00039b0 0x7f6ad0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae37fe640 1 --2- 172.21.15.98:0/1875789099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ae4071930 0x7f6ad0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae37fe640 1 --2- 172.21.15.98:0/1875789099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ad0004d40 0x7f6ad00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae37fe640 1 -- 172.21.15.98:0/1875789099 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae4071d70 con 0x7f6ad00039b0 2024-05-09T10:50:42.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae37fe640 1 -- 172.21.15.98:0/1875789099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae40b42a0 con 0x7f6ad0004d40 2024-05-09T10:50:42.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae37fe640 1 -- 172.21.15.98:0/1875789099 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad000b9f0 con 0x7f6ae4071930 2024-05-09T10:50:42.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae8fbe640 1 --2- 172.21.15.98:0/1875789099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ad00039b0 0x7f6ad0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:42.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae3fff640 1 --2- 172.21.15.98:0/1875789099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ae4071930 0x7f6ad0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:42.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae97bf640 1 --2- 172.21.15.98:0/1875789099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ad0004d40 0x7f6ad00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:42.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae8fbe640 1 -- 172.21.15.98:0/1875789099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ae4071930 msgr2=0x7f6ad0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:42.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae8fbe640 1 --2- 172.21.15.98:0/1875789099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ae4071930 0x7f6ad0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae8fbe640 1 -- 172.21.15.98:0/1875789099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ad0004d40 msgr2=0x7f6ad00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:42.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae8fbe640 1 --2- 172.21.15.98:0/1875789099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ad0004d40 0x7f6ad00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae8fbe640 1 -- 172.21.15.98:0/1875789099 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad0003520 con 0x7f6ad00039b0 2024-05-09T10:50:42.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.615+0000 7f6ae97bf640 1 --2- 172.21.15.98:0/1875789099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ad0004d40 0x7f6ad00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:42.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.616+0000 7f6ae8fbe640 1 --2- 172.21.15.98:0/1875789099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ad00039b0 0x7f6ad0003dd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400a870 tx=0x7f6ad0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795dd7e79a9de5cd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:42.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.616+0000 7f6ae37fe640 1 -- 172.21.15.98:0/1875789099 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad400a4c0 con 0x7f6ad00039b0 2024-05-09T10:50:42.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.616+0000 7f6ae37fe640 1 -- 172.21.15.98:0/1875789099 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad402f070 con 0x7f6ad00039b0 2024-05-09T10:50:42.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.616+0000 7f6ae37fe640 1 -- 172.21.15.98:0/1875789099 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad402bcc0 con 0x7f6ad00039b0 2024-05-09T10:50:42.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.617+0000 7f6aeb222640 1 -- 172.21.15.98:0/1875789099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ad00039b0 msgr2=0x7f6ad0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:42.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.617+0000 7f6aeb222640 1 --2- 172.21.15.98:0/1875789099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ad00039b0 0x7f6ad0003dd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400a870 tx=0x7f6ad0005830 comp rx=0 tx=0).stop 2024-05-09T10:50:42.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.617+0000 7f6ae97bf640 1 -- 172.21.15.98:0/1875789099 reap_dead start 2024-05-09T10:50:42.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.617+0000 7f6aeb222640 1 -- 172.21.15.98:0/1875789099 shutdown_connections 2024-05-09T10:50:42.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.617+0000 7f6aeb222640 1 -- 172.21.15.98:0/1875789099 >> 172.21.15.98:0/1875789099 conn(0x7f6ae40d0860 msgr2=0x7f6ae40d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:42.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.617+0000 7f6aeb222640 1 -- 172.21.15.98:0/1875789099 shutdown_connections 2024-05-09T10:50:42.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.617+0000 7f6aeb222640 1 -- 172.21.15.98:0/1875789099 wait complete. 2024-05-09T10:50:42.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.618+0000 7f6aeb222640 1 Processor -- start 2024-05-09T10:50:42.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.618+0000 7f6aeb222640 1 -- start start 2024-05-09T10:50:42.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.619+0000 7f6aeb222640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ad00039b0 0x7f6ae4155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.619+0000 7f6aeb222640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ae41539e0 0x7f6ae414b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.619+0000 7f6aeb222640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ae406abe0 0x7f6ae406b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.619+0000 7f6aeb222640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae415bde0 con 0x7f6ae41539e0 2024-05-09T10:50:42.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.619+0000 7f6aeb222640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae41588c0 con 0x7f6ad00039b0 2024-05-09T10:50:42.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.619+0000 7f6aeb222640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae40afdf0 con 0x7f6ae406abe0 2024-05-09T10:50:42.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.619+0000 7f6ae3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ae41539e0 0x7f6ae414b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:42.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.619+0000 7f6ae3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ae41539e0 0x7f6ae414b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48230/0 (socket says 172.21.15.98:48230) 2024-05-09T10:50:42.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.619+0000 7f6ae3fff640 1 -- 172.21.15.98:0/495185735 learned_addr learned my addr 172.21.15.98:0/495185735 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:42.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.619+0000 7f6ae8fbe640 1 --2- 172.21.15.98:0/495185735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ad00039b0 0x7f6ae4155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:42.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.619+0000 7f6ae97bf640 1 --2- 172.21.15.98:0/495185735 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ae406abe0 0x7f6ae406b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:42.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.619+0000 7f6ae3fff640 1 -- 172.21.15.98:0/495185735 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ae406abe0 msgr2=0x7f6ae406b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:42.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.619+0000 7f6ae3fff640 1 --2- 172.21.15.98:0/495185735 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ae406abe0 0x7f6ae406b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.619+0000 7f6ae3fff640 1 -- 172.21.15.98:0/495185735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ad00039b0 msgr2=0x7f6ae4155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:42.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.619+0000 7f6ae3fff640 1 --2- 172.21.15.98:0/495185735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ad00039b0 0x7f6ae4155f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.619+0000 7f6ae3fff640 1 -- 172.21.15.98:0/495185735 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae4156530 con 0x7f6ae41539e0 2024-05-09T10:50:42.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.619+0000 7f6ae8fbe640 1 --2- 172.21.15.98:0/495185735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ad00039b0 0x7f6ae4155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:42.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.619+0000 7f6ae3fff640 1 --2- 172.21.15.98:0/495185735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ae41539e0 0x7f6ae414b2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6acc011a70 tx=0x7f6acc011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:42.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.619+0000 7f6ae17fa640 1 -- 172.21.15.98:0/495185735 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc00fb50 con 0x7f6ae41539e0 2024-05-09T10:50:42.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.620+0000 7f6ae17fa640 1 -- 172.21.15.98:0/495185735 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6acc00fcf0 con 0x7f6ae41539e0 2024-05-09T10:50:42.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.620+0000 7f6aeb222640 1 -- 172.21.15.98:0/495185735 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae4156810 con 0x7f6ae41539e0 2024-05-09T10:50:42.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.620+0000 7f6ae17fa640 1 -- 172.21.15.98:0/495185735 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc00fea0 con 0x7f6ae41539e0 2024-05-09T10:50:42.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.620+0000 7f6aeb222640 1 -- 172.21.15.98:0/495185735 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae4156dd0 con 0x7f6ae41539e0 2024-05-09T10:50:42.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.620+0000 7f6aeb222640 1 -- 172.21.15.98:0/495185735 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae40673a0 con 0x7f6ae41539e0 2024-05-09T10:50:42.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.621+0000 7f6ae17fa640 1 -- 172.21.15.98:0/495185735 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6acc012040 con 0x7f6ae41539e0 2024-05-09T10:50:42.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.621+0000 7f6ae17fa640 1 --2- 172.21.15.98:0/495185735 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6ac406b500 0x7f6ac406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.621+0000 7f6ae17fa640 1 -- 172.21.15.98:0/495185735 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6acc08a810 con 0x7f6ae41539e0 2024-05-09T10:50:42.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.621+0000 7f6ae8fbe640 1 --2- 172.21.15.98:0/495185735 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6ac406b500 0x7f6ac406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:42.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.622+0000 7f6ae8fbe640 1 --2- 172.21.15.98:0/495185735 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6ac406b500 0x7f6ac406d9c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4033000 tx=0x7f6ad4007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:42.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.626+0000 7f6ae17fa640 1 -- 172.21.15.98:0/495185735 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6acc019030 con 0x7f6ae41539e0 2024-05-09T10:50:42.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.794+0000 7f6aeb222640 1 -- 172.21.15.98:0/495185735 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6ae4072f30 con 0x7f6ae41539e0 2024-05-09T10:50:42.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.795+0000 7f6ae17fa640 1 -- 172.21.15.98:0/495185735 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6acc0590d0 con 0x7f6ae41539e0 2024-05-09T10:50:42.788 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-05-09T10:50:42.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.797+0000 7f6aeb222640 1 -- 172.21.15.98:0/495185735 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6ac406b500 msgr2=0x7f6ac406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:42.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.797+0000 7f6aeb222640 1 --2- 172.21.15.98:0/495185735 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6ac406b500 0x7f6ac406d9c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4033000 tx=0x7f6ad4007000 comp rx=0 tx=0).stop 2024-05-09T10:50:42.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.797+0000 7f6aeb222640 1 -- 172.21.15.98:0/495185735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ae41539e0 msgr2=0x7f6ae414b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:42.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.797+0000 7f6aeb222640 1 --2- 172.21.15.98:0/495185735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ae41539e0 0x7f6ae414b2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6acc011a70 tx=0x7f6acc011f30 comp rx=0 tx=0).stop 2024-05-09T10:50:42.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.797+0000 7f6aeb222640 1 -- 172.21.15.98:0/495185735 shutdown_connections 2024-05-09T10:50:42.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.797+0000 7f6aeb222640 1 --2- 172.21.15.98:0/495185735 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f6ac406b500 0x7f6ac406d9c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.797+0000 7f6aeb222640 1 --2- 172.21.15.98:0/495185735 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6ae406abe0 0x7f6ae406b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.797+0000 7f6aeb222640 1 --2- 172.21.15.98:0/495185735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ae41539e0 0x7f6ae414b2f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.797+0000 7f6aeb222640 1 --2- 172.21.15.98:0/495185735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ad00039b0 0x7f6ae4155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.797+0000 7f6aeb222640 1 -- 172.21.15.98:0/495185735 >> 172.21.15.98:0/495185735 conn(0x7f6ae40d0860 msgr2=0x7f6ae4150410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:42.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.797+0000 7f6aeb222640 1 -- 172.21.15.98:0/495185735 shutdown_connections 2024-05-09T10:50:42.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.797+0000 7f6aeb222640 1 -- 172.21.15.98:0/495185735 wait complete. 2024-05-09T10:50:42.801 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T10:50:42.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.909+0000 7f36a3afc640 1 Processor -- start 2024-05-09T10:50:42.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.909+0000 7f36a3afc640 1 -- start start 2024-05-09T10:50:42.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.910+0000 7f36a3afc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f369c0cf950 0x7f369c15bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.910+0000 7f36a3afc640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f369c0b2bb0 con 0x7f369c0d1fc0 2024-05-09T10:50:42.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.910+0000 7f36a3afc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f369c0b2e90 con 0x7f369c0d1be0 2024-05-09T10:50:42.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.910+0000 7f36a3afc640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f369c0af840 con 0x7f369c0cf950 2024-05-09T10:50:42.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.910+0000 7f36a1097640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f369c0cf950 0x7f369c15bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:42.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.910+0000 7f36a2099640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f369c0d1fc0 0x7f369c15c0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38142/0 (socket says 172.21.15.98:38142) 2024-05-09T10:50:42.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.911+0000 7f36a2099640 1 -- 172.21.15.98:0/1714237506 learned_addr learned my addr 172.21.15.98:0/1714237506 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:42.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.911+0000 7f36a1097640 1 -- 172.21.15.98:0/1714237506 >> v1:172.21.15.105:6789/0 conn(0x7f369c0d1be0 legacy=0x7f369c0d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:42.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.911+0000 7f36a0896640 1 -- 172.21.15.98:0/1714237506 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528881753 0 0) 0x7f369c0b2bb0 con 0x7f369c0d1fc0 2024-05-09T10:50:42.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.911+0000 7f36a1097640 1 -- 172.21.15.98:0/1714237506 >> v1:172.21.15.98:6789/0 conn(0x7f369c0d1fc0 legacy=0x7f369c15c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:42.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.911+0000 7f36a1097640 1 -- 172.21.15.98:0/1714237506 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369c0b3170 con 0x7f369c0cf950 2024-05-09T10:50:42.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.911+0000 7f36a1097640 1 --2- 172.21.15.98:0/1714237506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369c0cf950 0x7f369c15bb90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f36900097a0 tx=0x7f369002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca85e03e0c75ec26 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:42.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.912+0000 7f36a0896640 1 -- 172.21.15.98:0/1714237506 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3690037070 con 0x7f369c0cf950 2024-05-09T10:50:42.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.912+0000 7f36a0896640 1 -- 172.21.15.98:0/1714237506 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3690032440 con 0x7f369c0cf950 2024-05-09T10:50:42.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.912+0000 7f36a0896640 1 -- 172.21.15.98:0/1714237506 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3690032810 con 0x7f369c0cf950 2024-05-09T10:50:42.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.912+0000 7f36a3afc640 1 -- 172.21.15.98:0/1714237506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369c0cf950 msgr2=0x7f369c15bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:42.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.912+0000 7f36a3afc640 1 --2- 172.21.15.98:0/1714237506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369c0cf950 0x7f369c15bb90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f36900097a0 tx=0x7f369002a230 comp rx=0 tx=0).stop 2024-05-09T10:50:42.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.912+0000 7f36a3afc640 1 -- 172.21.15.98:0/1714237506 shutdown_connections 2024-05-09T10:50:42.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.912+0000 7f36a3afc640 1 --2- 172.21.15.98:0/1714237506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369c0cf950 0x7f369c15bb90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.912+0000 7f36a3afc640 1 -- 172.21.15.98:0/1714237506 >> 172.21.15.98:0/1714237506 conn(0x7f369c0b6db0 msgr2=0x7f369c0b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:42.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.912+0000 7f36a3afc640 1 -- 172.21.15.98:0/1714237506 shutdown_connections 2024-05-09T10:50:42.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.912+0000 7f36a3afc640 1 -- 172.21.15.98:0/1714237506 wait complete. 2024-05-09T10:50:42.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.913+0000 7f36a3afc640 1 Processor -- start 2024-05-09T10:50:42.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.914+0000 7f36a3afc640 1 -- start start 2024-05-09T10:50:42.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.914+0000 7f36a3afc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f369c0cf950 0x7f369c15dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.914+0000 7f36a3afc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369c0d1be0 0x7f369c1e8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.914+0000 7f36a3afc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f369c0d1fc0 0x7f369c141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.914+0000 7f36a3afc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369c0b26b0 con 0x7f369c0d1be0 2024-05-09T10:50:42.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.914+0000 7f36a3afc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369c0b2190 con 0x7f369c0cf950 2024-05-09T10:50:42.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.914+0000 7f36a3afc640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369c15e5b0 con 0x7f369c0d1fc0 2024-05-09T10:50:42.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.914+0000 7f36a1097640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369c0d1be0 0x7f369c1e8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:42.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.914+0000 7f36a1097640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369c0d1be0 0x7f369c1e8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48248/0 (socket says 172.21.15.98:48248) 2024-05-09T10:50:42.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.914+0000 7f36a1097640 1 -- 172.21.15.98:0/3663307260 learned_addr learned my addr 172.21.15.98:0/3663307260 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:42.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.915+0000 7f36a1898640 1 --2- 172.21.15.98:0/3663307260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f369c0cf950 0x7f369c15dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:42.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.915+0000 7f36a1097640 1 -- 172.21.15.98:0/3663307260 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f369c0d1fc0 msgr2=0x7f369c141ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:42.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.915+0000 7f36a1097640 1 --2- 172.21.15.98:0/3663307260 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f369c0d1fc0 0x7f369c141ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.915+0000 7f36a1097640 1 -- 172.21.15.98:0/3663307260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f369c0cf950 msgr2=0x7f369c15dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:42.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.915+0000 7f36a1097640 1 --2- 172.21.15.98:0/3663307260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f369c0cf950 0x7f369c15dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:42.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.915+0000 7f36a1097640 1 -- 172.21.15.98:0/3663307260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369c15e890 con 0x7f369c0d1be0 2024-05-09T10:50:42.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.915+0000 7f36a1898640 1 --2- 172.21.15.98:0/3663307260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f369c0cf950 0x7f369c15dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:42.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.915+0000 7f36a1097640 1 --2- 172.21.15.98:0/3663307260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369c0d1be0 0x7f369c1e8c50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f36900097a0 tx=0x7f36900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:42.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.915+0000 7f369a7fc640 1 -- 172.21.15.98:0/3663307260 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3690037050 con 0x7f369c0d1be0 2024-05-09T10:50:42.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.915+0000 7f369a7fc640 1 -- 172.21.15.98:0/3663307260 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3690038040 con 0x7f369c0d1be0 2024-05-09T10:50:42.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.915+0000 7f36a3afc640 1 -- 172.21.15.98:0/3663307260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f369c15eb70 con 0x7f369c0d1be0 2024-05-09T10:50:42.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.915+0000 7f369a7fc640 1 -- 172.21.15.98:0/3663307260 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3690004490 con 0x7f369c0d1be0 2024-05-09T10:50:42.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.915+0000 7f36a3afc640 1 -- 172.21.15.98:0/3663307260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f369c06bfd0 con 0x7f369c0d1be0 2024-05-09T10:50:42.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.916+0000 7f36a3afc640 1 -- 172.21.15.98:0/3663307260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f369c09fbb0 con 0x7f369c0d1be0 2024-05-09T10:50:42.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.916+0000 7f369a7fc640 1 -- 172.21.15.98:0/3663307260 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3690030900 con 0x7f369c0d1be0 2024-05-09T10:50:42.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.917+0000 7f369a7fc640 1 --2- 172.21.15.98:0/3663307260 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f368806b4d0 0x7f368806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:42.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.917+0000 7f369a7fc640 1 -- 172.21.15.98:0/3663307260 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36900aae10 con 0x7f369c0d1be0 2024-05-09T10:50:42.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.917+0000 7f36a1898640 1 --2- 172.21.15.98:0/3663307260 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f368806b4d0 0x7f368806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:42.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.917+0000 7f36a1898640 1 --2- 172.21.15.98:0/3663307260 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f368806b4d0 0x7f368806d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f368c001d70 tx=0x7f368c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:42.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:42.921+0000 7f369a7fc640 1 -- 172.21.15.98:0/3663307260 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3690079db0 con 0x7f369c0d1be0 2024-05-09T10:50:43.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.088+0000 7f36a3afc640 1 -- 172.21.15.98:0/3663307260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f369c0b2830 con 0x7f369c0d1be0 2024-05-09T10:50:43.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.089+0000 7f369a7fc640 1 -- 172.21.15.98:0/3663307260 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f36900796b0 con 0x7f369c0d1be0 2024-05-09T10:50:43.082 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2024-05-09T10:50:43.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.091+0000 7f36a3afc640 1 -- 172.21.15.98:0/3663307260 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f368806b4d0 msgr2=0x7f368806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:43.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.091+0000 7f36a3afc640 1 --2- 172.21.15.98:0/3663307260 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f368806b4d0 0x7f368806d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f368c001d70 tx=0x7f368c009040 comp rx=0 tx=0).stop 2024-05-09T10:50:43.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.091+0000 7f36a3afc640 1 -- 172.21.15.98:0/3663307260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369c0d1be0 msgr2=0x7f369c1e8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:43.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.091+0000 7f36a3afc640 1 --2- 172.21.15.98:0/3663307260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369c0d1be0 0x7f369c1e8c50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f36900097a0 tx=0x7f36900046c0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.091+0000 7f36a3afc640 1 -- 172.21.15.98:0/3663307260 shutdown_connections 2024-05-09T10:50:43.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.091+0000 7f36a3afc640 1 --2- 172.21.15.98:0/3663307260 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f368806b4d0 0x7f368806d990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.091+0000 7f36a3afc640 1 --2- 172.21.15.98:0/3663307260 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f369c0d1fc0 0x7f369c141ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.091+0000 7f36a3afc640 1 --2- 172.21.15.98:0/3663307260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369c0d1be0 0x7f369c1e8c50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.091+0000 7f36a3afc640 1 --2- 172.21.15.98:0/3663307260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f369c0cf950 0x7f369c15dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.091+0000 7f36a3afc640 1 -- 172.21.15.98:0/3663307260 >> 172.21.15.98:0/3663307260 conn(0x7f369c0b6db0 msgr2=0x7f369c15ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:43.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.091+0000 7f36a3afc640 1 -- 172.21.15.98:0/3663307260 shutdown_connections 2024-05-09T10:50:43.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.091+0000 7f36a3afc640 1 -- 172.21.15.98:0/3663307260 wait complete. 2024-05-09T10:50:43.095 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T10:50:43.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.206+0000 7fe356fb0640 1 Processor -- start 2024-05-09T10:50:43.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.206+0000 7fe356fb0640 1 -- start start 2024-05-09T10:50:43.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.207+0000 7fe356fb0640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe3500d2870 0x7fe350168b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:43.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.207+0000 7fe356fb0640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3500b3070 con 0x7fe35015dd00 2024-05-09T10:50:43.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.207+0000 7fe356fb0640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3500b32e0 con 0x7fe35015e0e0 2024-05-09T10:50:43.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.207+0000 7fe356fb0640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3500b35c0 con 0x7fe3500d2870 2024-05-09T10:50:43.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.207+0000 7fe35554d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe3500d2870 0x7fe350168b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:43.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.207+0000 7fe354d4c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe35015dd00 0x7fe3500cf230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38156/0 (socket says 172.21.15.98:38156) 2024-05-09T10:50:43.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.207+0000 7fe354d4c640 1 -- 172.21.15.98:0/932367680 learned_addr learned my addr 172.21.15.98:0/932367680 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:43.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.207+0000 7fe35554d640 1 -- 172.21.15.98:0/932367680 >> v1:172.21.15.105:6789/0 conn(0x7fe35015e0e0 legacy=0x7fe3500d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:43.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.207+0000 7fe35554d640 1 -- 172.21.15.98:0/932367680 >> v1:172.21.15.98:6789/0 conn(0x7fe35015dd00 legacy=0x7fe3500cf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:43.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.207+0000 7fe35554d640 1 -- 172.21.15.98:0/932367680 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3500b38a0 con 0x7fe3500d2870 2024-05-09T10:50:43.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.207+0000 7fe3477fe640 1 -- 172.21.15.98:0/932367680 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1345165996 0 0) 0x7fe3500b3070 con 0x7fe35015dd00 2024-05-09T10:50:43.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.208+0000 7fe35554d640 1 --2- 172.21.15.98:0/932367680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3500d2870 0x7fe350168b10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe348009d90 tx=0x7fe34802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d77b38cc58d1f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:43.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.208+0000 7fe3477fe640 1 -- 172.21.15.98:0/932367680 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe348007740 con 0x7fe3500d2870 2024-05-09T10:50:43.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.208+0000 7fe3477fe640 1 -- 172.21.15.98:0/932367680 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3480078e0 con 0x7fe3500d2870 2024-05-09T10:50:43.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.208+0000 7fe3477fe640 1 -- 172.21.15.98:0/932367680 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe348007cb0 con 0x7fe3500d2870 2024-05-09T10:50:43.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.208+0000 7fe356fb0640 1 -- 172.21.15.98:0/932367680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3500d2870 msgr2=0x7fe350168b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:43.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.208+0000 7fe356fb0640 1 --2- 172.21.15.98:0/932367680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3500d2870 0x7fe350168b10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe348009d90 tx=0x7fe34802a6f0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.209+0000 7fe356fb0640 1 -- 172.21.15.98:0/932367680 shutdown_connections 2024-05-09T10:50:43.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.209+0000 7fe356fb0640 1 --2- 172.21.15.98:0/932367680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3500d2870 0x7fe350168b10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.209+0000 7fe356fb0640 1 -- 172.21.15.98:0/932367680 >> 172.21.15.98:0/932367680 conn(0x7fe35009ce50 msgr2=0x7fe3500d3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:43.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.209+0000 7fe356fb0640 1 -- 172.21.15.98:0/932367680 shutdown_connections 2024-05-09T10:50:43.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.209+0000 7fe356fb0640 1 -- 172.21.15.98:0/932367680 wait complete. 2024-05-09T10:50:43.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.210+0000 7fe356fb0640 1 Processor -- start 2024-05-09T10:50:43.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.210+0000 7fe356fb0640 1 -- start start 2024-05-09T10:50:43.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.211+0000 7fe356fb0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3500d2870 0x7fe3500c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:43.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.211+0000 7fe356fb0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe35015dd00 0x7fe3500c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:43.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.211+0000 7fe356fb0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe35015e0e0 0x7fe3500b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:43.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.211+0000 7fe356fb0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3500b4f30 con 0x7fe3500d2870 2024-05-09T10:50:43.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.211+0000 7fe356fb0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3500b4db0 con 0x7fe35015dd00 2024-05-09T10:50:43.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.211+0000 7fe356fb0640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3500b50b0 con 0x7fe35015e0e0 2024-05-09T10:50:43.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.211+0000 7fe354d4c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3500d2870 0x7fe3500c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:43.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.211+0000 7fe354d4c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3500d2870 0x7fe3500c78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48276/0 (socket says 172.21.15.98:48276) 2024-05-09T10:50:43.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.211+0000 7fe354d4c640 1 -- 172.21.15.98:0/4163921615 learned_addr learned my addr 172.21.15.98:0/4163921615 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:43.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.211+0000 7fe347fff640 1 --2- 172.21.15.98:0/4163921615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe35015dd00 0x7fe3500c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:43.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.211+0000 7fe35554d640 1 --2- 172.21.15.98:0/4163921615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe35015e0e0 0x7fe3500b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:43.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.211+0000 7fe354d4c640 1 -- 172.21.15.98:0/4163921615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe35015e0e0 msgr2=0x7fe3500b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:43.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.211+0000 7fe354d4c640 1 --2- 172.21.15.98:0/4163921615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe35015e0e0 0x7fe3500b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.211+0000 7fe354d4c640 1 -- 172.21.15.98:0/4163921615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe35015dd00 msgr2=0x7fe3500c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:43.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.211+0000 7fe354d4c640 1 --2- 172.21.15.98:0/4163921615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe35015dd00 0x7fe3500c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.211+0000 7fe354d4c640 1 -- 172.21.15.98:0/4163921615 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3500c8060 con 0x7fe3500d2870 2024-05-09T10:50:43.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.211+0000 7fe35554d640 1 --2- 172.21.15.98:0/4163921615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe35015e0e0 0x7fe3500b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:43.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.211+0000 7fe354d4c640 1 --2- 172.21.15.98:0/4163921615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3500d2870 0x7fe3500c78f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00c4a0 tx=0x7fe34c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:43.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.211+0000 7fe3457fa640 1 -- 172.21.15.98:0/4163921615 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c018070 con 0x7fe3500d2870 2024-05-09T10:50:43.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.211+0000 7fe3457fa640 1 -- 172.21.15.98:0/4163921615 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c0074e0 con 0x7fe3500d2870 2024-05-09T10:50:43.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.211+0000 7fe356fb0640 1 -- 172.21.15.98:0/4163921615 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3500c8340 con 0x7fe3500d2870 2024-05-09T10:50:43.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.211+0000 7fe3457fa640 1 -- 172.21.15.98:0/4163921615 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c004a20 con 0x7fe3500d2870 2024-05-09T10:50:43.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.211+0000 7fe356fb0640 1 -- 172.21.15.98:0/4163921615 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3500c8900 con 0x7fe3500d2870 2024-05-09T10:50:43.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.213+0000 7fe356fb0640 1 -- 172.21.15.98:0/4163921615 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3500a1b30 con 0x7fe3500d2870 2024-05-09T10:50:43.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.213+0000 7fe3457fa640 1 -- 172.21.15.98:0/4163921615 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe34c01b070 con 0x7fe3500d2870 2024-05-09T10:50:43.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.213+0000 7fe3457fa640 1 --2- 172.21.15.98:0/4163921615 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe33c06b4d0 0x7fe33c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:43.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.214+0000 7fe347fff640 1 --2- 172.21.15.98:0/4163921615 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe33c06b4d0 0x7fe33c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:43.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.214+0000 7fe3457fa640 1 -- 172.21.15.98:0/4163921615 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe34c089410 con 0x7fe3500d2870 2024-05-09T10:50:43.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.214+0000 7fe347fff640 1 --2- 172.21.15.98:0/4163921615 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe33c06b4d0 0x7fe33c06d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe3500b81b0 tx=0x7fe340009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:43.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.218+0000 7fe3457fa640 1 -- 172.21.15.98:0/4163921615 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe34c057c40 con 0x7fe3500d2870 2024-05-09T10:50:43.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.392+0000 7fe356fb0640 1 -- 172.21.15.98:0/4163921615 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe35006afa0 con 0x7fe3500d2870 2024-05-09T10:50:43.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.392+0000 7fe3457fa640 1 -- 172.21.15.98:0/4163921615 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe34c05bce0 con 0x7fe3500d2870 2024-05-09T10:50:43.385 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2024-05-09T10:50:43.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.394+0000 7fe356fb0640 1 -- 172.21.15.98:0/4163921615 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe33c06b4d0 msgr2=0x7fe33c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:43.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.394+0000 7fe356fb0640 1 --2- 172.21.15.98:0/4163921615 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe33c06b4d0 0x7fe33c06d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe3500b81b0 tx=0x7fe340009040 comp rx=0 tx=0).stop 2024-05-09T10:50:43.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.394+0000 7fe356fb0640 1 -- 172.21.15.98:0/4163921615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3500d2870 msgr2=0x7fe3500c78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:43.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.394+0000 7fe356fb0640 1 --2- 172.21.15.98:0/4163921615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3500d2870 0x7fe3500c78f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00c4a0 tx=0x7fe34c00c960 comp rx=0 tx=0).stop 2024-05-09T10:50:43.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.394+0000 7fe356fb0640 1 -- 172.21.15.98:0/4163921615 shutdown_connections 2024-05-09T10:50:43.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.394+0000 7fe356fb0640 1 --2- 172.21.15.98:0/4163921615 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe33c06b4d0 0x7fe33c06d990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.394+0000 7fe356fb0640 1 --2- 172.21.15.98:0/4163921615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe35015e0e0 0x7fe3500b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.394+0000 7fe356fb0640 1 --2- 172.21.15.98:0/4163921615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe35015dd00 0x7fe3500c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.394+0000 7fe356fb0640 1 --2- 172.21.15.98:0/4163921615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3500d2870 0x7fe3500c78f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.394+0000 7fe356fb0640 1 -- 172.21.15.98:0/4163921615 >> 172.21.15.98:0/4163921615 conn(0x7fe35009ce50 msgr2=0x7fe3500d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:43.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.394+0000 7fe356fb0640 1 -- 172.21.15.98:0/4163921615 shutdown_connections 2024-05-09T10:50:43.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.394+0000 7fe356fb0640 1 -- 172.21.15.98:0/4163921615 wait complete. 2024-05-09T10:50:43.398 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:43.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.505+0000 7f7b5be31640 1 Processor -- start 2024-05-09T10:50:43.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.505+0000 7f7b5be31640 1 -- start start 2024-05-09T10:50:43.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.506+0000 7f7b5be31640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7b5415de40 0x7f7b540d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:43.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.506+0000 7f7b5be31640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7b540cf250 0x7f7b540cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:43.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.506+0000 7f7b5be31640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b5415e220 con 0x7f7b540d0470 2024-05-09T10:50:43.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.506+0000 7f7b5be31640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b540d1790 con 0x7f7b540cf250 2024-05-09T10:50:43.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.506+0000 7f7b5be31640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540d19e0 con 0x7f7b5415de40 2024-05-09T10:50:43.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.506+0000 7f7b59bcd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7b5415de40 0x7f7b540d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:43.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.506+0000 7f7b593cc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7b540cf250 0x7f7b540cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:43.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.506+0000 7f7b59bcd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7b5415de40 0x7f7b540d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:43104/0 (socket says 172.21.15.98:43104) 2024-05-09T10:50:43.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.506+0000 7f7b59bcd640 1 -- 172.21.15.98:0/2775714918 learned_addr learned my addr 172.21.15.98:0/2775714918 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:43.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.506+0000 7f7b59bcd640 1 -- 172.21.15.98:0/2775714918 >> v2:172.21.15.105:3300/0 conn(0x7f7b540cf250 msgr2=0x7f7b540cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:43.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.506+0000 7f7b59bcd640 1 --2- 172.21.15.98:0/2775714918 >> v2:172.21.15.105:3300/0 conn(0x7f7b540cf250 0x7f7b540cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.506+0000 7f7b59bcd640 1 -- 172.21.15.98:0/2775714918 >> v1:172.21.15.105:6789/0 conn(0x7f7b540d0470 legacy=0x7f7b540b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:43.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.506+0000 7f7b59bcd640 1 -- 172.21.15.98:0/2775714918 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b540d1cc0 con 0x7f7b5415de40 2024-05-09T10:50:43.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.506+0000 7f7b593cc640 1 --2- 172.21.15.98:0/2775714918 >> v2:172.21.15.105:3300/0 conn(0x7f7b540cf250 0x7f7b540cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:50:43.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.507+0000 7f7b59bcd640 1 --2- 172.21.15.98:0/2775714918 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7b5415de40 0x7f7b540d27a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7b50009870 tx=0x7f7b5002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57b04f4b918beede server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:43.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.507+0000 7f7b58bcb640 1 -- 172.21.15.98:0/2775714918 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b50037070 con 0x7f7b5415de40 2024-05-09T10:50:43.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.507+0000 7f7b58bcb640 1 -- 172.21.15.98:0/2775714918 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b50032440 con 0x7f7b5415de40 2024-05-09T10:50:43.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.507+0000 7f7b58bcb640 1 -- 172.21.15.98:0/2775714918 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b50032810 con 0x7f7b5415de40 2024-05-09T10:50:43.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.508+0000 7f7b5be31640 1 -- 172.21.15.98:0/2775714918 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7b5415de40 msgr2=0x7f7b540d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:43.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.508+0000 7f7b5be31640 1 --2- 172.21.15.98:0/2775714918 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7b5415de40 0x7f7b540d27a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7b50009870 tx=0x7f7b5002a270 comp rx=0 tx=0).stop 2024-05-09T10:50:43.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.508+0000 7f7b5be31640 1 -- 172.21.15.98:0/2775714918 shutdown_connections 2024-05-09T10:50:43.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.508+0000 7f7b5be31640 1 --2- 172.21.15.98:0/2775714918 >> v2:172.21.15.105:3300/0 conn(0x7f7b540cf250 0x7f7b540cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.508+0000 7f7b5be31640 1 --2- 172.21.15.98:0/2775714918 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7b5415de40 0x7f7b540d27a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.508+0000 7f7b5be31640 1 -- 172.21.15.98:0/2775714918 >> 172.21.15.98:0/2775714918 conn(0x7f7b5409ce90 msgr2=0x7f7b540d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:43.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.508+0000 7f7b5be31640 1 -- 172.21.15.98:0/2775714918 shutdown_connections 2024-05-09T10:50:43.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.508+0000 7f7b5be31640 1 -- 172.21.15.98:0/2775714918 wait complete. 2024-05-09T10:50:43.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.509+0000 7f7b5be31640 1 Processor -- start 2024-05-09T10:50:43.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.509+0000 7f7b5be31640 1 -- start start 2024-05-09T10:50:43.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.510+0000 7f7b5be31640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7b540cf250 0x7f7b5415b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:43.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.510+0000 7f7b5be31640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b540d0470 0x7f7b540cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:43.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.510+0000 7f7b5be31640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b5415de40 0x7f7b541e8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:43.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.510+0000 7f7b5be31640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540d4680 con 0x7f7b5415de40 2024-05-09T10:50:43.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.510+0000 7f7b5be31640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540d2280 con 0x7f7b540d0470 2024-05-09T10:50:43.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.510+0000 7f7b5be31640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540d4800 con 0x7f7b540cf250 2024-05-09T10:50:43.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.510+0000 7f7b5a3ce640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b5415de40 0x7f7b541e8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:43.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.510+0000 7f7b5a3ce640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b5415de40 0x7f7b541e8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33978/0 (socket says 172.21.15.98:33978) 2024-05-09T10:50:43.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.510+0000 7f7b5a3ce640 1 -- 172.21.15.98:0/3966971556 learned_addr learned my addr 172.21.15.98:0/3966971556 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:43.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.510+0000 7f7b593cc640 1 --2- 172.21.15.98:0/3966971556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b540d0470 0x7f7b540cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:43.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.510+0000 7f7b59bcd640 1 --2- 172.21.15.98:0/3966971556 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7b540cf250 0x7f7b5415b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:43.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.511+0000 7f7b5a3ce640 1 -- 172.21.15.98:0/3966971556 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7b540cf250 msgr2=0x7f7b5415b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:43.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.511+0000 7f7b5a3ce640 1 --2- 172.21.15.98:0/3966971556 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7b540cf250 0x7f7b5415b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.511+0000 7f7b5a3ce640 1 -- 172.21.15.98:0/3966971556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b540d0470 msgr2=0x7f7b540cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:43.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.511+0000 7f7b5a3ce640 1 --2- 172.21.15.98:0/3966971556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b540d0470 0x7f7b540cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.511+0000 7f7b5a3ce640 1 -- 172.21.15.98:0/3966971556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b541f3a00 con 0x7f7b5415de40 2024-05-09T10:50:43.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.511+0000 7f7b5a3ce640 1 --2- 172.21.15.98:0/3966971556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b5415de40 0x7f7b541e8ea0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00c7d0 tx=0x7f7b4c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:43.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.511+0000 7f7b4a7fc640 1 -- 172.21.15.98:0/3966971556 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c003b70 con 0x7f7b5415de40 2024-05-09T10:50:43.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.511+0000 7f7b4a7fc640 1 -- 172.21.15.98:0/3966971556 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4c003d30 con 0x7f7b5415de40 2024-05-09T10:50:43.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.511+0000 7f7b5be31640 1 -- 172.21.15.98:0/3966971556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b540cc160 con 0x7f7b5415de40 2024-05-09T10:50:43.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.511+0000 7f7b4a7fc640 1 -- 172.21.15.98:0/3966971556 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c00aa90 con 0x7f7b5415de40 2024-05-09T10:50:43.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.511+0000 7f7b5be31640 1 -- 172.21.15.98:0/3966971556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b540cc720 con 0x7f7b5415de40 2024-05-09T10:50:43.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.512+0000 7f7b5be31640 1 -- 172.21.15.98:0/3966971556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b540b82b0 con 0x7f7b5415de40 2024-05-09T10:50:43.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.512+0000 7f7b4a7fc640 1 -- 172.21.15.98:0/3966971556 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b4c00ac30 con 0x7f7b5415de40 2024-05-09T10:50:43.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.513+0000 7f7b4a7fc640 1 --2- 172.21.15.98:0/3966971556 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f7b3806b4d0 0x7f7b3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:43.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.513+0000 7f7b4a7fc640 1 -- 172.21.15.98:0/3966971556 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b4c089fd0 con 0x7f7b5415de40 2024-05-09T10:50:43.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.513+0000 7f7b59bcd640 1 --2- 172.21.15.98:0/3966971556 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f7b3806b4d0 0x7f7b3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:43.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.513+0000 7f7b59bcd640 1 --2- 172.21.15.98:0/3966971556 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f7b3806b4d0 0x7f7b3806d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7b50034000 tx=0x7f7b5002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:43.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.517+0000 7f7b4a7fc640 1 -- 172.21.15.98:0/3966971556 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7b540b82b0 con 0x7f7b5415de40 2024-05-09T10:50:43.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.688+0000 7f7b5be31640 1 -- 172.21.15.98:0/3966971556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b54069cb0 con 0x7f7b5415de40 2024-05-09T10:50:43.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.689+0000 7f7b4a7fc640 1 -- 172.21.15.98:0/3966971556 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f7b4c058800 con 0x7f7b5415de40 2024-05-09T10:50:43.683 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:43.683 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":20,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:21.176419+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:50:17.323444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T10:50:18.482315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:43.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.691+0000 7f7b5be31640 1 -- 172.21.15.98:0/3966971556 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f7b3806b4d0 msgr2=0x7f7b3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:43.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.691+0000 7f7b5be31640 1 --2- 172.21.15.98:0/3966971556 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f7b3806b4d0 0x7f7b3806d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7b50034000 tx=0x7f7b5002e000 comp rx=0 tx=0).stop 2024-05-09T10:50:43.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.691+0000 7f7b5be31640 1 -- 172.21.15.98:0/3966971556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b5415de40 msgr2=0x7f7b541e8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:43.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.691+0000 7f7b5be31640 1 --2- 172.21.15.98:0/3966971556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b5415de40 0x7f7b541e8ea0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00c7d0 tx=0x7f7b4c00cc90 comp rx=0 tx=0).stop 2024-05-09T10:50:43.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.691+0000 7f7b5be31640 1 -- 172.21.15.98:0/3966971556 shutdown_connections 2024-05-09T10:50:43.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.691+0000 7f7b5be31640 1 --2- 172.21.15.98:0/3966971556 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f7b3806b4d0 0x7f7b3806d990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.691+0000 7f7b5be31640 1 --2- 172.21.15.98:0/3966971556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b5415de40 0x7f7b541e8ea0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.691+0000 7f7b5be31640 1 --2- 172.21.15.98:0/3966971556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b540d0470 0x7f7b540cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.691+0000 7f7b5be31640 1 --2- 172.21.15.98:0/3966971556 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7b540cf250 0x7f7b5415b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.691+0000 7f7b5be31640 1 -- 172.21.15.98:0/3966971556 >> 172.21.15.98:0/3966971556 conn(0x7f7b5409ce90 msgr2=0x7f7b540b6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:43.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.692+0000 7f7b5be31640 1 -- 172.21.15.98:0/3966971556 shutdown_connections 2024-05-09T10:50:43.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.692+0000 7f7b5be31640 1 -- 172.21.15.98:0/3966971556 wait complete. 2024-05-09T10:50:43.696 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T10:50:07.396423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T10:50:17.323444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T10:50:18.482315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T10:50:43.696 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T10:50:43.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.805+0000 7fde2fb6a640 1 Processor -- start 2024-05-09T10:50:43.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.805+0000 7fde2fb6a640 1 -- start start 2024-05-09T10:50:43.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.806+0000 7fde2fb6a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fde2815a8f0 0x7fde2815f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:43.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.806+0000 7fde2fb6a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde280ca530 0x7fde280ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:43.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.806+0000 7fde2fb6a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fde2815acd0 0x7fde280cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.806+0000 7fde2fb6a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde2815f5d0 con 0x7fde2815a8f0 2024-05-09T10:50:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.806+0000 7fde2fb6a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde280cb540 con 0x7fde2815acd0 2024-05-09T10:50:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.806+0000 7fde2fb6a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde280ae960 con 0x7fde280ca530 2024-05-09T10:50:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.806+0000 7fde2d906640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fde2815a8f0 0x7fde2815f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.806+0000 7fde2d906640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fde2815a8f0 0x7fde2815f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33994/0 (socket says 172.21.15.98:33994) 2024-05-09T10:50:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.806+0000 7fde2d906640 1 -- 172.21.15.98:0/831176465 learned_addr learned my addr 172.21.15.98:0/831176465 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.806+0000 7fde2e107640 1 --2- 172.21.15.98:0/831176465 >> v2:172.21.15.105:3300/0 conn(0x7fde2815acd0 0x7fde280cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.806+0000 7fde2d105640 1 --2- 172.21.15.98:0/831176465 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde280ca530 0x7fde280ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.806+0000 7fde2d906640 1 -- 172.21.15.98:0/831176465 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde280ca530 msgr2=0x7fde280ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.806+0000 7fde2d906640 1 --2- 172.21.15.98:0/831176465 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde280ca530 0x7fde280ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.806+0000 7fde2d906640 1 -- 172.21.15.98:0/831176465 >> v2:172.21.15.105:3300/0 conn(0x7fde2815acd0 msgr2=0x7fde280cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.806+0000 7fde2d906640 1 --2- 172.21.15.98:0/831176465 >> v2:172.21.15.105:3300/0 conn(0x7fde2815acd0 0x7fde280cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.806+0000 7fde2d906640 1 -- 172.21.15.98:0/831176465 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde280ca9c0 con 0x7fde2815a8f0 2024-05-09T10:50:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.806+0000 7fde2d105640 1 --2- 172.21.15.98:0/831176465 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde280ca530 0x7fde280ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.807+0000 7fde2d906640 1 --2- 172.21.15.98:0/831176465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde2815a8f0 0x7fde2815f090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fde280af190 tx=0x7fde2402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd026b321fa7137 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:43.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.807+0000 7fde2c904640 1 -- 172.21.15.98:0/831176465 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde24037070 con 0x7fde2815a8f0 2024-05-09T10:50:43.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.807+0000 7fde2c904640 1 -- 172.21.15.98:0/831176465 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde24032440 con 0x7fde2815a8f0 2024-05-09T10:50:43.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.807+0000 7fde2c904640 1 -- 172.21.15.98:0/831176465 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde24032810 con 0x7fde2815a8f0 2024-05-09T10:50:43.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.808+0000 7fde2fb6a640 1 -- 172.21.15.98:0/831176465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde2815a8f0 msgr2=0x7fde2815f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:43.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.808+0000 7fde2fb6a640 1 --2- 172.21.15.98:0/831176465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde2815a8f0 0x7fde2815f090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fde280af190 tx=0x7fde2402a1e0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.808+0000 7fde2fb6a640 1 -- 172.21.15.98:0/831176465 shutdown_connections 2024-05-09T10:50:43.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.808+0000 7fde2fb6a640 1 --2- 172.21.15.98:0/831176465 >> v2:172.21.15.105:3300/0 conn(0x7fde2815acd0 0x7fde280cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.808+0000 7fde2fb6a640 1 --2- 172.21.15.98:0/831176465 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde280ca530 0x7fde280ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.808+0000 7fde2fb6a640 1 --2- 172.21.15.98:0/831176465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde2815a8f0 0x7fde2815f090 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.808+0000 7fde2fb6a640 1 -- 172.21.15.98:0/831176465 >> 172.21.15.98:0/831176465 conn(0x7fde280cf760 msgr2=0x7fde280cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:43.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.808+0000 7fde2fb6a640 1 -- 172.21.15.98:0/831176465 shutdown_connections 2024-05-09T10:50:43.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.808+0000 7fde2fb6a640 1 -- 172.21.15.98:0/831176465 wait complete. 2024-05-09T10:50:43.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.809+0000 7fde2fb6a640 1 Processor -- start 2024-05-09T10:50:43.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.809+0000 7fde2fb6a640 1 -- start start 2024-05-09T10:50:43.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.810+0000 7fde2fb6a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde2815a8f0 0x7fde280c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:43.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.810+0000 7fde2fb6a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde2815acd0 0x7fde2814f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:43.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.810+0000 7fde2fb6a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde2815d310 0x7fde2815d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:43.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.810+0000 7fde2fb6a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde280cb260 con 0x7fde2815acd0 2024-05-09T10:50:43.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.810+0000 7fde2fb6a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde280caf80 con 0x7fde2815a8f0 2024-05-09T10:50:43.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.810+0000 7fde2fb6a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2806abe0 con 0x7fde2815d310 2024-05-09T10:50:43.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.810+0000 7fde2d105640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde2815acd0 0x7fde2814f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:43.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.810+0000 7fde2d105640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde2815acd0 0x7fde2814f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34010/0 (socket says 172.21.15.98:34010) 2024-05-09T10:50:43.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.810+0000 7fde2d105640 1 -- 172.21.15.98:0/2418159184 learned_addr learned my addr 172.21.15.98:0/2418159184 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:43.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.810+0000 7fde2d906640 1 --2- 172.21.15.98:0/2418159184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde2815a8f0 0x7fde280c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:43.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.810+0000 7fde2e107640 1 --2- 172.21.15.98:0/2418159184 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde2815d310 0x7fde2815d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:43.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.810+0000 7fde2d105640 1 -- 172.21.15.98:0/2418159184 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde2815d310 msgr2=0x7fde2815d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:43.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.810+0000 7fde2d105640 1 --2- 172.21.15.98:0/2418159184 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde2815d310 0x7fde2815d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.810+0000 7fde2d105640 1 -- 172.21.15.98:0/2418159184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde2815a8f0 msgr2=0x7fde280c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:43.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.810+0000 7fde2d105640 1 --2- 172.21.15.98:0/2418159184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde2815a8f0 0x7fde280c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.810+0000 7fde2d105640 1 -- 172.21.15.98:0/2418159184 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde280c6bd0 con 0x7fde2815acd0 2024-05-09T10:50:43.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.810+0000 7fde2e107640 1 --2- 172.21.15.98:0/2418159184 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde2815d310 0x7fde2815d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:43.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.810+0000 7fde2d105640 1 --2- 172.21.15.98:0/2418159184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde2815acd0 0x7fde2814f210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fde1800c7f0 tx=0x7fde1800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:43.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.810+0000 7fde1e7fc640 1 -- 172.21.15.98:0/2418159184 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde18015070 con 0x7fde2815acd0 2024-05-09T10:50:43.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.810+0000 7fde1e7fc640 1 -- 172.21.15.98:0/2418159184 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde18007a30 con 0x7fde2815acd0 2024-05-09T10:50:43.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.810+0000 7fde1e7fc640 1 -- 172.21.15.98:0/2418159184 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde18007e00 con 0x7fde2815acd0 2024-05-09T10:50:43.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.810+0000 7fde2fb6a640 1 -- 172.21.15.98:0/2418159184 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde280c6eb0 con 0x7fde2815acd0 2024-05-09T10:50:43.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.811+0000 7fde2fb6a640 1 -- 172.21.15.98:0/2418159184 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde280c7470 con 0x7fde2815acd0 2024-05-09T10:50:43.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.811+0000 7fde2fb6a640 1 -- 172.21.15.98:0/2418159184 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde28072c80 con 0x7fde2815acd0 2024-05-09T10:50:43.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.812+0000 7fde1e7fc640 1 -- 172.21.15.98:0/2418159184 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde18018070 con 0x7fde2815acd0 2024-05-09T10:50:43.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.812+0000 7fde1e7fc640 1 --2- 172.21.15.98:0/2418159184 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fde0c06b4d0 0x7fde0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:43.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.812+0000 7fde1e7fc640 1 -- 172.21.15.98:0/2418159184 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde1808a190 con 0x7fde2815acd0 2024-05-09T10:50:43.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.812+0000 7fde2d906640 1 --2- 172.21.15.98:0/2418159184 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fde0c06b4d0 0x7fde0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:43.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.813+0000 7fde2d906640 1 --2- 172.21.15.98:0/2418159184 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fde0c06b4d0 0x7fde0c06d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fde24034000 tx=0x7fde2402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:43.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.816+0000 7fde1e7fc640 1 -- 172.21.15.98:0/2418159184 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fde18058930 con 0x7fde2815acd0 2024-05-09T10:50:43.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.989+0000 7fde2fb6a640 1 -- 172.21.15.98:0/2418159184 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde2804f000 con 0x7fde2815acd0 2024-05-09T10:50:43.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.989+0000 7fde1e7fc640 1 -- 172.21.15.98:0/2418159184 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde1805c9d0 con 0x7fde2815acd0 2024-05-09T10:50:43.983 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-05-09T10:50:43.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.991+0000 7fde2fb6a640 1 -- 172.21.15.98:0/2418159184 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fde0c06b4d0 msgr2=0x7fde0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:43.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.991+0000 7fde2fb6a640 1 --2- 172.21.15.98:0/2418159184 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fde0c06b4d0 0x7fde0c06d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fde24034000 tx=0x7fde2402e000 comp rx=0 tx=0).stop 2024-05-09T10:50:43.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.991+0000 7fde2fb6a640 1 -- 172.21.15.98:0/2418159184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde2815acd0 msgr2=0x7fde2814f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:43.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.991+0000 7fde2fb6a640 1 --2- 172.21.15.98:0/2418159184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde2815acd0 0x7fde2814f210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fde1800c7f0 tx=0x7fde1800cbf0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.991+0000 7fde2fb6a640 1 -- 172.21.15.98:0/2418159184 shutdown_connections 2024-05-09T10:50:43.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.991+0000 7fde2fb6a640 1 --2- 172.21.15.98:0/2418159184 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fde0c06b4d0 0x7fde0c06d990 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.991+0000 7fde2fb6a640 1 --2- 172.21.15.98:0/2418159184 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fde2815d310 0x7fde2815d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.991+0000 7fde2fb6a640 1 --2- 172.21.15.98:0/2418159184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde2815acd0 0x7fde2814f210 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.991+0000 7fde2fb6a640 1 --2- 172.21.15.98:0/2418159184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde2815a8f0 0x7fde280c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:43.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.991+0000 7fde2fb6a640 1 -- 172.21.15.98:0/2418159184 >> 172.21.15.98:0/2418159184 conn(0x7fde280cf760 msgr2=0x7fde2815ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:43.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.991+0000 7fde2fb6a640 1 -- 172.21.15.98:0/2418159184 shutdown_connections 2024-05-09T10:50:43.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:43.991+0000 7fde2fb6a640 1 -- 172.21.15.98:0/2418159184 wait complete. 2024-05-09T10:50:43.996 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T10:50:44.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.105+0000 7fe1f4d9e640 1 Processor -- start 2024-05-09T10:50:44.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.105+0000 7fe1f4d9e640 1 -- start start 2024-05-09T10:50:44.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.106+0000 7fe1f4d9e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe1f00b25f0 0x7fe1f00cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:44.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.106+0000 7fe1f4d9e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f00cf0c0 con 0x7fe1f00b0f10 2024-05-09T10:50:44.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.106+0000 7fe1f4d9e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f00cff60 con 0x7fe1f00cece0 2024-05-09T10:50:44.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.106+0000 7fe1f4d9e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1f00d0800 con 0x7fe1f00b25f0 2024-05-09T10:50:44.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.106+0000 7fe1eed9d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe1f00b0f10 0x7fe1f00b1360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41038/0 (socket says 172.21.15.98:41038) 2024-05-09T10:50:44.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.106+0000 7fe1eed9d640 1 -- 172.21.15.98:0/1433834256 learned_addr learned my addr 172.21.15.98:0/1433834256 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:44.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.106+0000 7fe1edd9b640 1 --2- 172.21.15.98:0/1433834256 >> v2:172.21.15.105:3300/0 conn(0x7fe1f00b25f0 0x7fe1f00cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:44.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.106+0000 7fe1ed59a640 1 -- 172.21.15.98:0/1433834256 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3460229062 0 0) 0x7fe1f00cf0c0 con 0x7fe1f00b0f10 2024-05-09T10:50:44.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.106+0000 7fe1ed59a640 1 -- 172.21.15.98:0/1433834256 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1d8003520 con 0x7fe1f00b0f10 2024-05-09T10:50:44.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.107+0000 7fe1ed59a640 1 -- 172.21.15.98:0/1433834256 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1427908378 0 0) 0x7fe1d8003520 con 0x7fe1f00b0f10 2024-05-09T10:50:44.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.107+0000 7fe1ed59a640 1 -- 172.21.15.98:0/1433834256 >> v2:172.21.15.105:3300/0 conn(0x7fe1f00b25f0 msgr2=0x7fe1f00cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:44.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.107+0000 7fe1ed59a640 1 --2- 172.21.15.98:0/1433834256 >> v2:172.21.15.105:3300/0 conn(0x7fe1f00b25f0 0x7fe1f00cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.107+0000 7fe1ed59a640 1 -- 172.21.15.98:0/1433834256 >> v1:172.21.15.105:6789/0 conn(0x7fe1f00cece0 legacy=0x7fe1f015f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:44.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.107+0000 7fe1ed59a640 1 -- 172.21.15.98:0/1433834256 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f00d0240 con 0x7fe1f00b0f10 2024-05-09T10:50:44.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.107+0000 7fe1ed59a640 1 -- 172.21.15.98:0/1433834256 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7fe1e80019f0 con 0x7fe1f00b0f10 2024-05-09T10:50:44.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.107+0000 7fe1ed59a640 1 -- 172.21.15.98:0/1433834256 >> v1:172.21.15.98:6789/0 conn(0x7fe1f00b0f10 legacy=0x7fe1f00b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:44.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.107+0000 7fe1ed59a640 1 --2- 172.21.15.98:0/1433834256 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe1d80039b0 0x7fe1d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:44.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.107+0000 7fe1ed59a640 1 --2- 172.21.15.98:0/1433834256 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1d8004800 0x7fe1d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:44.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.107+0000 7fe1ed59a640 1 --2- 172.21.15.98:0/1433834256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1d80051c0 0x7fe1d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:44.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.107+0000 7fe1ed59a640 1 -- 172.21.15.98:0/1433834256 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d800c030 con 0x7fe1d80051c0 2024-05-09T10:50:44.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.107+0000 7fe1ed59a640 1 -- 172.21.15.98:0/1433834256 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d800c1e0 con 0x7fe1d8004800 2024-05-09T10:50:44.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.107+0000 7fe1ed59a640 1 -- 172.21.15.98:0/1433834256 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d800c3e0 con 0x7fe1d80039b0 2024-05-09T10:50:44.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.107+0000 7fe1edd9b640 1 --2- 172.21.15.98:0/1433834256 >> v2:172.21.15.105:3300/0 conn(0x7fe1f00b25f0 0x7fe1f00cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:50:44.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.107+0000 7fe1eed9d640 1 --2- 172.21.15.98:0/1433834256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1d80051c0 0x7fe1d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:44.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.107+0000 7fe1ee59c640 1 --2- 172.21.15.98:0/1433834256 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe1d80039b0 0x7fe1d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:44.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.107+0000 7fe1edd9b640 1 --2- 172.21.15.98:0/1433834256 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1d8004800 0x7fe1d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:44.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.107+0000 7fe1eed9d640 1 -- 172.21.15.98:0/1433834256 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe1d80039b0 msgr2=0x7fe1d8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:44.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.107+0000 7fe1eed9d640 1 --2- 172.21.15.98:0/1433834256 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe1d80039b0 0x7fe1d8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.107+0000 7fe1eed9d640 1 -- 172.21.15.98:0/1433834256 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1d8004800 msgr2=0x7fe1d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:44.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.107+0000 7fe1eed9d640 1 --2- 172.21.15.98:0/1433834256 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1d8004800 0x7fe1d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.107+0000 7fe1eed9d640 1 -- 172.21.15.98:0/1433834256 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d8003520 con 0x7fe1d80051c0 2024-05-09T10:50:44.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.108+0000 7fe1eed9d640 1 -- 172.21.15.98:0/1433834256 reap_dead start 2024-05-09T10:50:44.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.108+0000 7fe1eed9d640 1 --2- 172.21.15.98:0/1433834256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1d80051c0 0x7fe1d800b9c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800a860 tx=0x7fe1d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84d95fe7a3267e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:44.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.108+0000 7fe1ed59a640 1 -- 172.21.15.98:0/1433834256 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e800a480 con 0x7fe1d80051c0 2024-05-09T10:50:44.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.108+0000 7fe1ed59a640 1 -- 172.21.15.98:0/1433834256 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e802f070 con 0x7fe1d80051c0 2024-05-09T10:50:44.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.108+0000 7fe1ed59a640 1 -- 172.21.15.98:0/1433834256 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e802bca0 con 0x7fe1d80051c0 2024-05-09T10:50:44.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.109+0000 7fe1f4d9e640 1 -- 172.21.15.98:0/1433834256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1d80051c0 msgr2=0x7fe1d800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:44.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.109+0000 7fe1f4d9e640 1 --2- 172.21.15.98:0/1433834256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1d80051c0 0x7fe1d800b9c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800a860 tx=0x7fe1d8005830 comp rx=0 tx=0).stop 2024-05-09T10:50:44.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.109+0000 7fe1f4d9e640 1 -- 172.21.15.98:0/1433834256 shutdown_connections 2024-05-09T10:50:44.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.109+0000 7fe1f4d9e640 1 --2- 172.21.15.98:0/1433834256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1d80051c0 0x7fe1d800b9c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.109+0000 7fe1f4d9e640 1 -- 172.21.15.98:0/1433834256 >> 172.21.15.98:0/1433834256 conn(0x7fe1f00b34e0 msgr2=0x7fe1f00b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:44.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.109+0000 7fe1f4d9e640 1 -- 172.21.15.98:0/1433834256 shutdown_connections 2024-05-09T10:50:44.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.109+0000 7fe1f4d9e640 1 -- 172.21.15.98:0/1433834256 wait complete. 2024-05-09T10:50:44.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.110+0000 7fe1f4d9e640 1 Processor -- start 2024-05-09T10:50:44.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.110+0000 7fe1f4d9e640 1 -- start start 2024-05-09T10:50:44.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.111+0000 7fe1f4d9e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1d80051c0 0x7fe1f015a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:44.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.111+0000 7fe1f4d9e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1f00c5dd0 0x7fe1f00c5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:44.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.111+0000 7fe1f4d9e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe1f015c550 0x7fe1f015c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:44.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.111+0000 7fe1f4d9e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f00b3120 con 0x7fe1d80051c0 2024-05-09T10:50:44.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.111+0000 7fe1f4d9e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f015eb60 con 0x7fe1f00c5dd0 2024-05-09T10:50:44.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.111+0000 7fe1f4d9e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f00cf980 con 0x7fe1f015c550 2024-05-09T10:50:44.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.111+0000 7fe1ee59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1d80051c0 0x7fe1f015a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:44.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.111+0000 7fe1ee59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1d80051c0 0x7fe1f015a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34022/0 (socket says 172.21.15.98:34022) 2024-05-09T10:50:44.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.111+0000 7fe1ee59c640 1 -- 172.21.15.98:0/2579405200 learned_addr learned my addr 172.21.15.98:0/2579405200 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:44.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.111+0000 7fe1edd9b640 1 --2- 172.21.15.98:0/2579405200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1f00c5dd0 0x7fe1f00c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:44.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.111+0000 7fe1eed9d640 1 --2- 172.21.15.98:0/2579405200 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe1f015c550 0x7fe1f015c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:44.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.111+0000 7fe1ee59c640 1 -- 172.21.15.98:0/2579405200 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe1f015c550 msgr2=0x7fe1f015c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:44.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.111+0000 7fe1ee59c640 1 --2- 172.21.15.98:0/2579405200 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe1f015c550 0x7fe1f015c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.111+0000 7fe1ee59c640 1 -- 172.21.15.98:0/2579405200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1f00c5dd0 msgr2=0x7fe1f00c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:44.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.111+0000 7fe1ee59c640 1 --2- 172.21.15.98:0/2579405200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1f00c5dd0 0x7fe1f00c5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.111+0000 7fe1ee59c640 1 -- 172.21.15.98:0/2579405200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f00c6440 con 0x7fe1d80051c0 2024-05-09T10:50:44.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.111+0000 7fe1edd9b640 1 --2- 172.21.15.98:0/2579405200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1f00c5dd0 0x7fe1f00c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:44.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.111+0000 7fe1ee59c640 1 --2- 172.21.15.98:0/2579405200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1d80051c0 0x7fe1f015a7c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000f5f0 tx=0x7fe1e000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:44.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.112+0000 7fe1deffd640 1 -- 172.21.15.98:0/2579405200 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e001a070 con 0x7fe1d80051c0 2024-05-09T10:50:44.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.112+0000 7fe1deffd640 1 -- 172.21.15.98:0/2579405200 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e0005320 con 0x7fe1d80051c0 2024-05-09T10:50:44.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.112+0000 7fe1deffd640 1 -- 172.21.15.98:0/2579405200 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0007270 con 0x7fe1d80051c0 2024-05-09T10:50:44.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.112+0000 7fe1f4d9e640 1 -- 172.21.15.98:0/2579405200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f006bcf0 con 0x7fe1d80051c0 2024-05-09T10:50:44.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.112+0000 7fe1f4d9e640 1 -- 172.21.15.98:0/2579405200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f006c2b0 con 0x7fe1d80051c0 2024-05-09T10:50:44.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.113+0000 7fe1f4d9e640 1 -- 172.21.15.98:0/2579405200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f00c61b0 con 0x7fe1d80051c0 2024-05-09T10:50:44.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.113+0000 7fe1deffd640 1 -- 172.21.15.98:0/2579405200 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1e001d070 con 0x7fe1d80051c0 2024-05-09T10:50:44.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.114+0000 7fe1deffd640 1 --2- 172.21.15.98:0/2579405200 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe1d006b500 0x7fe1d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:44.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.114+0000 7fe1deffd640 1 -- 172.21.15.98:0/2579405200 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1e0089950 con 0x7fe1d80051c0 2024-05-09T10:50:44.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.114+0000 7fe1edd9b640 1 --2- 172.21.15.98:0/2579405200 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe1d006b500 0x7fe1d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:44.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.114+0000 7fe1edd9b640 1 --2- 172.21.15.98:0/2579405200 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe1d006b500 0x7fe1d006d9c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe1f01f6ee0 tx=0x7fe1e4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:44.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.118+0000 7fe1deffd640 1 -- 172.21.15.98:0/2579405200 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe1e00587d0 con 0x7fe1d80051c0 2024-05-09T10:50:44.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.295+0000 7fe1f4d9e640 1 -- 172.21.15.98:0/2579405200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe1f004f000 con 0x7fe1d80051c0 2024-05-09T10:50:44.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.296+0000 7fe1deffd640 1 -- 172.21.15.98:0/2579405200 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe1e00580d0 con 0x7fe1d80051c0 2024-05-09T10:50:44.290 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2024-05-09T10:50:44.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.298+0000 7fe1f4d9e640 1 -- 172.21.15.98:0/2579405200 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe1d006b500 msgr2=0x7fe1d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:44.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.298+0000 7fe1f4d9e640 1 --2- 172.21.15.98:0/2579405200 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe1d006b500 0x7fe1d006d9c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe1f01f6ee0 tx=0x7fe1e4008040 comp rx=0 tx=0).stop 2024-05-09T10:50:44.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.298+0000 7fe1f4d9e640 1 -- 172.21.15.98:0/2579405200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1d80051c0 msgr2=0x7fe1f015a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:44.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.298+0000 7fe1f4d9e640 1 --2- 172.21.15.98:0/2579405200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1d80051c0 0x7fe1f015a7c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000f5f0 tx=0x7fe1e000fab0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.298+0000 7fe1f4d9e640 1 -- 172.21.15.98:0/2579405200 shutdown_connections 2024-05-09T10:50:44.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.298+0000 7fe1f4d9e640 1 --2- 172.21.15.98:0/2579405200 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe1d006b500 0x7fe1d006d9c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.298+0000 7fe1f4d9e640 1 --2- 172.21.15.98:0/2579405200 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe1f015c550 0x7fe1f015c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.298+0000 7fe1f4d9e640 1 --2- 172.21.15.98:0/2579405200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1f00c5dd0 0x7fe1f00c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.298+0000 7fe1f4d9e640 1 --2- 172.21.15.98:0/2579405200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1d80051c0 0x7fe1f015a7c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.298+0000 7fe1f4d9e640 1 -- 172.21.15.98:0/2579405200 >> 172.21.15.98:0/2579405200 conn(0x7fe1f00b34e0 msgr2=0x7fe1f00ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:44.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.299+0000 7fe1f4d9e640 1 -- 172.21.15.98:0/2579405200 shutdown_connections 2024-05-09T10:50:44.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.299+0000 7fe1f4d9e640 1 -- 172.21.15.98:0/2579405200 wait complete. 2024-05-09T10:50:44.303 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T10:50:44.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.412+0000 7f3934abc640 1 Processor -- start 2024-05-09T10:50:44.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.412+0000 7f3934abc640 1 -- start start 2024-05-09T10:50:44.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.413+0000 7f3934abc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f39300ca530 0x7f393015f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:44.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.413+0000 7f3934abc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39300cee80 0x7f39300c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:44.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.413+0000 7f3934abc640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39300ae960 con 0x7f393015a8f0 2024-05-09T10:50:44.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.413+0000 7f3934abc640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f393015f5d0 con 0x7f39300ca530 2024-05-09T10:50:44.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.413+0000 7f3934abc640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39300cb540 con 0x7f39300cee80 2024-05-09T10:50:44.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.413+0000 7f3925d9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f39300ca530 0x7f393015f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:44.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.413+0000 7f392e59c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f393015a8f0 0x7f39300ce770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41040/0 (socket says 172.21.15.98:41040) 2024-05-09T10:50:44.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.413+0000 7f3925d9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f39300ca530 0x7f393015f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34030/0 (socket says 172.21.15.98:34030) 2024-05-09T10:50:44.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.413+0000 7f392e59c640 1 -- 172.21.15.98:0/2284371394 learned_addr learned my addr 172.21.15.98:0/2284371394 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:44.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.413+0000 7f3925d9b640 1 -- 172.21.15.98:0/2284371394 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39300cee80 msgr2=0x7f39300c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:44.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.413+0000 7f392ed9d640 1 --2- 172.21.15.98:0/2284371394 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39300cee80 0x7f39300c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:44.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.413+0000 7f392dd9b640 1 -- 172.21.15.98:0/2284371394 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439900566 0 0) 0x7f39300ae960 con 0x7f393015a8f0 2024-05-09T10:50:44.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.413+0000 7f3925d9b640 1 --2- 172.21.15.98:0/2284371394 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39300cee80 0x7f39300c5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.413+0000 7f3925d9b640 1 -- 172.21.15.98:0/2284371394 >> v1:172.21.15.98:6789/0 conn(0x7f393015a8f0 legacy=0x7f39300ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:44.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.413+0000 7f3925d9b640 1 -- 172.21.15.98:0/2284371394 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39300ca9c0 con 0x7f39300ca530 2024-05-09T10:50:44.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.414+0000 7f3925d9b640 1 --2- 172.21.15.98:0/2284371394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39300ca530 0x7f393015f090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f39180097a0 tx=0x7f391802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa6a7f27c1dcb3e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:44.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.414+0000 7f392dd9b640 1 -- 172.21.15.98:0/2284371394 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3918037070 con 0x7f39300ca530 2024-05-09T10:50:44.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.414+0000 7f392dd9b640 1 -- 172.21.15.98:0/2284371394 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3918032440 con 0x7f39300ca530 2024-05-09T10:50:44.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.414+0000 7f392dd9b640 1 -- 172.21.15.98:0/2284371394 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3918032810 con 0x7f39300ca530 2024-05-09T10:50:44.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.414+0000 7f3934abc640 1 -- 172.21.15.98:0/2284371394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39300ca530 msgr2=0x7f393015f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:44.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.414+0000 7f3934abc640 1 --2- 172.21.15.98:0/2284371394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39300ca530 0x7f393015f090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f39180097a0 tx=0x7f391802a180 comp rx=0 tx=0).stop 2024-05-09T10:50:44.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.414+0000 7f3934abc640 1 -- 172.21.15.98:0/2284371394 shutdown_connections 2024-05-09T10:50:44.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.414+0000 7f3934abc640 1 --2- 172.21.15.98:0/2284371394 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f39300cee80 0x7f39300c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.414+0000 7f3934abc640 1 --2- 172.21.15.98:0/2284371394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39300ca530 0x7f393015f090 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.414+0000 7f3934abc640 1 -- 172.21.15.98:0/2284371394 >> 172.21.15.98:0/2284371394 conn(0x7f39300cf760 msgr2=0x7f39300cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:44.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.414+0000 7f3934abc640 1 -- 172.21.15.98:0/2284371394 shutdown_connections 2024-05-09T10:50:44.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.415+0000 7f3934abc640 1 -- 172.21.15.98:0/2284371394 wait complete. 2024-05-09T10:50:44.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.416+0000 7f3934abc640 1 Processor -- start 2024-05-09T10:50:44.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.416+0000 7f3934abc640 1 -- start start 2024-05-09T10:50:44.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.417+0000 7f3934abc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39300cee80 0x7f39300b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:44.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.417+0000 7f3934abc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f393015a8f0 0x7f3930151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:44.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.417+0000 7f3934abc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3930152120 0x7f39300af300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:44.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.417+0000 7f3934abc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39300cb260 con 0x7f393015a8f0 2024-05-09T10:50:44.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.417+0000 7f3934abc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39300caf80 con 0x7f39300cee80 2024-05-09T10:50:44.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.417+0000 7f3934abc640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f393015acd0 con 0x7f3930152120 2024-05-09T10:50:44.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.417+0000 7f3925d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f393015a8f0 0x7f3930151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:44.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.417+0000 7f3925d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f393015a8f0 0x7f3930151980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34044/0 (socket says 172.21.15.98:34044) 2024-05-09T10:50:44.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.417+0000 7f3925d9b640 1 -- 172.21.15.98:0/3956539689 learned_addr learned my addr 172.21.15.98:0/3956539689 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:44.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.417+0000 7f392e59c640 1 --2- 172.21.15.98:0/3956539689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39300cee80 0x7f39300b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:44.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.417+0000 7f392ed9d640 1 --2- 172.21.15.98:0/3956539689 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3930152120 0x7f39300af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:44.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.417+0000 7f3925d9b640 1 -- 172.21.15.98:0/3956539689 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3930152120 msgr2=0x7f39300af300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:44.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.417+0000 7f3925d9b640 1 --2- 172.21.15.98:0/3956539689 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3930152120 0x7f39300af300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.417+0000 7f3925d9b640 1 -- 172.21.15.98:0/3956539689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39300cee80 msgr2=0x7f39300b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:44.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.417+0000 7f3925d9b640 1 --2- 172.21.15.98:0/3956539689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39300cee80 0x7f39300b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.417+0000 7f3925d9b640 1 -- 172.21.15.98:0/3956539689 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39300afc70 con 0x7f393015a8f0 2024-05-09T10:50:44.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.417+0000 7f392e59c640 1 --2- 172.21.15.98:0/3956539689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39300cee80 0x7f39300b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:44.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.417+0000 7f3925d9b640 1 --2- 172.21.15.98:0/3956539689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f393015a8f0 0x7f3930151980 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3918035040 tx=0x7f39180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:44.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.417+0000 7f39277fe640 1 -- 172.21.15.98:0/3956539689 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3918037040 con 0x7f393015a8f0 2024-05-09T10:50:44.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.418+0000 7f39277fe640 1 -- 172.21.15.98:0/3956539689 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3918030770 con 0x7f393015a8f0 2024-05-09T10:50:44.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.418+0000 7f3934abc640 1 -- 172.21.15.98:0/3956539689 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39300aff50 con 0x7f393015a8f0 2024-05-09T10:50:44.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.418+0000 7f39277fe640 1 -- 172.21.15.98:0/3956539689 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3918030b40 con 0x7f393015a8f0 2024-05-09T10:50:44.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.418+0000 7f3934abc640 1 -- 172.21.15.98:0/3956539689 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39300b0510 con 0x7f393015a8f0 2024-05-09T10:50:44.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.419+0000 7f3934abc640 1 -- 172.21.15.98:0/3956539689 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3930072c80 con 0x7f393015a8f0 2024-05-09T10:50:44.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.419+0000 7f39277fe640 1 -- 172.21.15.98:0/3956539689 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3918044020 con 0x7f393015a8f0 2024-05-09T10:50:44.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.419+0000 7f39277fe640 1 --2- 172.21.15.98:0/3956539689 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f391006b500 0x7f391006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:44.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.420+0000 7f392e59c640 1 --2- 172.21.15.98:0/3956539689 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f391006b500 0x7f391006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:44.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.420+0000 7f39277fe640 1 -- 172.21.15.98:0/3956539689 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39180aa740 con 0x7f393015a8f0 2024-05-09T10:50:44.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.420+0000 7f392e59c640 1 --2- 172.21.15.98:0/3956539689 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f391006b500 0x7f391006d9c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3920000f60 tx=0x7f3920007ca0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:44.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.423+0000 7f39277fe640 1 -- 172.21.15.98:0/3956539689 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3918032e20 con 0x7f393015a8f0 2024-05-09T10:50:44.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.594+0000 7f3934abc640 1 -- 172.21.15.98:0/3956539689 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f393004f000 con 0x7f393015a8f0 2024-05-09T10:50:44.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.594+0000 7f39277fe640 1 -- 172.21.15.98:0/3956539689 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f39180795d0 con 0x7f393015a8f0 2024-05-09T10:50:44.588 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2024-05-09T10:50:44.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.596+0000 7f3934abc640 1 -- 172.21.15.98:0/3956539689 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f391006b500 msgr2=0x7f391006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:44.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.596+0000 7f3934abc640 1 --2- 172.21.15.98:0/3956539689 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f391006b500 0x7f391006d9c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3920000f60 tx=0x7f3920007ca0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.596+0000 7f3934abc640 1 -- 172.21.15.98:0/3956539689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f393015a8f0 msgr2=0x7f3930151980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:44.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.596+0000 7f3934abc640 1 --2- 172.21.15.98:0/3956539689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f393015a8f0 0x7f3930151980 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3918035040 tx=0x7f39180046c0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.596+0000 7f3934abc640 1 -- 172.21.15.98:0/3956539689 shutdown_connections 2024-05-09T10:50:44.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.596+0000 7f3934abc640 1 --2- 172.21.15.98:0/3956539689 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f391006b500 0x7f391006d9c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.596+0000 7f3934abc640 1 --2- 172.21.15.98:0/3956539689 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3930152120 0x7f39300af300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.596+0000 7f3934abc640 1 --2- 172.21.15.98:0/3956539689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f393015a8f0 0x7f3930151980 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.596+0000 7f3934abc640 1 --2- 172.21.15.98:0/3956539689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39300cee80 0x7f39300b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.596+0000 7f3934abc640 1 -- 172.21.15.98:0/3956539689 >> 172.21.15.98:0/3956539689 conn(0x7f39300cf760 msgr2=0x7f39300cc170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:44.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.596+0000 7f3934abc640 1 -- 172.21.15.98:0/3956539689 shutdown_connections 2024-05-09T10:50:44.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.596+0000 7f3934abc640 1 -- 172.21.15.98:0/3956539689 wait complete. 2024-05-09T10:50:44.670 INFO:tasks.cephfs_test_runner:Starting test: test_volumes_module_finisher_thread (tasks.cephfs.test_volumes.TestPerModuleFinsherThread) 2024-05-09T10:50:44.671 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:44.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.782+0000 7f73433f5640 1 Processor -- start 2024-05-09T10:50:44.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.782+0000 7f73433f5640 1 -- start start 2024-05-09T10:50:44.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.783+0000 7f73433f5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f733c15dce0 0x7f733c06bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:44.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.783+0000 7f73433f5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f733c15e0c0 0x7f733c06c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:44.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.783+0000 7f73433f5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f733c0d0c70 con 0x7f733c0d0700 2024-05-09T10:50:44.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.783+0000 7f73433f5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f733c0b4350 con 0x7f733c15e0c0 2024-05-09T10:50:44.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.783+0000 7f73433f5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f733c0d0f50 con 0x7f733c15dce0 2024-05-09T10:50:44.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.783+0000 7f7341191640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f733c15dce0 0x7f733c06bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:44.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.783+0000 7f7340990640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f733c15e0c0 0x7f733c06c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:44.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.783+0000 7f7341191640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f733c15dce0 0x7f733c06bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:43184/0 (socket says 172.21.15.98:43184) 2024-05-09T10:50:44.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.783+0000 7f7341992640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f733c0d0700 0x7f733c161a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.98:46446/0 (socket says 172.21.15.98:46446) 2024-05-09T10:50:44.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.783+0000 7f7341191640 1 -- 172.21.15.98:0/1630978170 learned_addr learned my addr 172.21.15.98:0/1630978170 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:44.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.784+0000 7f7340990640 1 -- 172.21.15.98:0/1630978170 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f733c15dce0 msgr2=0x7f733c06bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:44.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.784+0000 7f7340990640 1 --2- 172.21.15.98:0/1630978170 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f733c15dce0 0x7f733c06bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.784+0000 7f7340990640 1 -- 172.21.15.98:0/1630978170 >> v1:172.21.15.105:6789/0 conn(0x7f733c0d0700 legacy=0x7f733c161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:44.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.784+0000 7f7340990640 1 -- 172.21.15.98:0/1630978170 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733c0b0fd0 con 0x7f733c15e0c0 2024-05-09T10:50:44.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.784+0000 7f7333fff640 1 -- 172.21.15.98:0/1630978170 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276639271 0 0) 0x7f733c0d0c70 con 0x7f733c0d0700 2024-05-09T10:50:44.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.784+0000 7f7341191640 1 --2- 172.21.15.98:0/1630978170 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f733c15dce0 0x7f733c06bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:50:44.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.784+0000 7f7340990640 1 --2- 172.21.15.98:0/1630978170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f733c15e0c0 0x7f733c06c3b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f732c0097a0 tx=0x7f732c02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=437ecf2be6ac70e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:44.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.785+0000 7f7333fff640 1 -- 172.21.15.98:0/1630978170 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c037070 con 0x7f733c15e0c0 2024-05-09T10:50:44.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.785+0000 7f7333fff640 1 -- 172.21.15.98:0/1630978170 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732c032440 con 0x7f733c15e0c0 2024-05-09T10:50:44.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.785+0000 7f7333fff640 1 -- 172.21.15.98:0/1630978170 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c032810 con 0x7f733c15e0c0 2024-05-09T10:50:44.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.785+0000 7f73433f5640 1 -- 172.21.15.98:0/1630978170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f733c15e0c0 msgr2=0x7f733c06c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:44.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.785+0000 7f73433f5640 1 --2- 172.21.15.98:0/1630978170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f733c15e0c0 0x7f733c06c3b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f732c0097a0 tx=0x7f732c02a210 comp rx=0 tx=0).stop 2024-05-09T10:50:44.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.785+0000 7f73433f5640 1 -- 172.21.15.98:0/1630978170 shutdown_connections 2024-05-09T10:50:44.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.785+0000 7f73433f5640 1 --2- 172.21.15.98:0/1630978170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f733c15e0c0 0x7f733c06c3b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.785+0000 7f73433f5640 1 --2- 172.21.15.98:0/1630978170 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f733c15dce0 0x7f733c06bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.785+0000 7f73433f5640 1 -- 172.21.15.98:0/1630978170 >> 172.21.15.98:0/1630978170 conn(0x7f733c0b34e0 msgr2=0x7f733c0b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:44.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.785+0000 7f73433f5640 1 -- 172.21.15.98:0/1630978170 shutdown_connections 2024-05-09T10:50:44.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.785+0000 7f73433f5640 1 -- 172.21.15.98:0/1630978170 wait complete. 2024-05-09T10:50:44.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.786+0000 7f73433f5640 1 Processor -- start 2024-05-09T10:50:44.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.786+0000 7f73433f5640 1 -- start start 2024-05-09T10:50:44.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.787+0000 7f73433f5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f733c0d0700 0x7f733c154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:44.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.787+0000 7f73433f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f733c15dce0 0x7f733c1e91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:44.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.787+0000 7f73433f5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f733c15e0c0 0x7f733c1e96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:44.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.787+0000 7f73433f5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c0b2ac0 con 0x7f733c15e0c0 2024-05-09T10:50:44.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.787+0000 7f73433f5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c0b1590 con 0x7f733c15dce0 2024-05-09T10:50:44.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.787+0000 7f73433f5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f733c0b2c40 con 0x7f733c0d0700 2024-05-09T10:50:44.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.787+0000 7f7341992640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f733c15e0c0 0x7f733c1e96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:44.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.787+0000 7f7341992640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f733c15e0c0 0x7f733c1e96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34052/0 (socket says 172.21.15.98:34052) 2024-05-09T10:50:44.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.787+0000 7f7341992640 1 -- 172.21.15.98:0/648187543 learned_addr learned my addr 172.21.15.98:0/648187543 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:44.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.787+0000 7f7340990640 1 --2- 172.21.15.98:0/648187543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f733c15dce0 0x7f733c1e91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:44.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.787+0000 7f7341191640 1 --2- 172.21.15.98:0/648187543 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f733c0d0700 0x7f733c154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:44.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.788+0000 7f7341992640 1 -- 172.21.15.98:0/648187543 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f733c0d0700 msgr2=0x7f733c154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:44.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.788+0000 7f7341992640 1 --2- 172.21.15.98:0/648187543 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f733c0d0700 0x7f733c154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.788+0000 7f7341992640 1 -- 172.21.15.98:0/648187543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f733c15dce0 msgr2=0x7f733c1e91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:44.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.788+0000 7f7341992640 1 --2- 172.21.15.98:0/648187543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f733c15dce0 0x7f733c1e91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.788+0000 7f7341992640 1 -- 172.21.15.98:0/648187543 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733c0b53e0 con 0x7f733c15e0c0 2024-05-09T10:50:44.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.788+0000 7f7341992640 1 --2- 172.21.15.98:0/648187543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f733c15e0c0 0x7f733c1e96e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f733400c7d0 tx=0x7f733400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:44.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.788+0000 7f7331ffb640 1 -- 172.21.15.98:0/648187543 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7334018070 con 0x7f733c15e0c0 2024-05-09T10:50:44.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.788+0000 7f7331ffb640 1 -- 172.21.15.98:0/648187543 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733400a640 con 0x7f733c15e0c0 2024-05-09T10:50:44.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.788+0000 7f73433f5640 1 -- 172.21.15.98:0/648187543 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f733c0b56c0 con 0x7f733c15e0c0 2024-05-09T10:50:44.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.788+0000 7f7331ffb640 1 -- 172.21.15.98:0/648187543 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733400aa70 con 0x7f733c15e0c0 2024-05-09T10:50:44.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.788+0000 7f73433f5640 1 -- 172.21.15.98:0/648187543 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f733c0b5c80 con 0x7f733c15e0c0 2024-05-09T10:50:44.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.789+0000 7f73433f5640 1 -- 172.21.15.98:0/648187543 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f733c067c30 con 0x7f733c15e0c0 2024-05-09T10:50:44.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.789+0000 7f7331ffb640 1 -- 172.21.15.98:0/648187543 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f733401b070 con 0x7f733c15e0c0 2024-05-09T10:50:44.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.790+0000 7f7331ffb640 1 --2- 172.21.15.98:0/648187543 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f732006b500 0x7f732006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:44.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.790+0000 7f7331ffb640 1 -- 172.21.15.98:0/648187543 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73340897f0 con 0x7f733c15e0c0 2024-05-09T10:50:44.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.790+0000 7f7341191640 1 --2- 172.21.15.98:0/648187543 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f732006b500 0x7f732006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:44.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.790+0000 7f7341191640 1 --2- 172.21.15.98:0/648187543 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f732006b500 0x7f732006d9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7338009a90 tx=0x7f7338008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:44.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.794+0000 7f7331ffb640 1 -- 172.21.15.98:0/648187543 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f733c067c30 con 0x7f733c15e0c0 2024-05-09T10:50:44.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.966+0000 7f73433f5640 1 -- 172.21.15.98:0/648187543 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f733c069cb0 con 0x7f733c15e0c0 2024-05-09T10:50:44.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.967+0000 7f7331ffb640 1 -- 172.21.15.98:0/648187543 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f7334058670 con 0x7f733c15e0c0 2024-05-09T10:50:44.960 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:44.961 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":20,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:21.176419+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:50:17.323444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T10:50:18.482315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:44.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.969+0000 7f73433f5640 1 -- 172.21.15.98:0/648187543 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f732006b500 msgr2=0x7f732006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:44.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.969+0000 7f73433f5640 1 --2- 172.21.15.98:0/648187543 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f732006b500 0x7f732006d9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7338009a90 tx=0x7f7338008040 comp rx=0 tx=0).stop 2024-05-09T10:50:44.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.969+0000 7f73433f5640 1 -- 172.21.15.98:0/648187543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f733c15e0c0 msgr2=0x7f733c1e96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:44.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.969+0000 7f73433f5640 1 --2- 172.21.15.98:0/648187543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f733c15e0c0 0x7f733c1e96e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f733400c7d0 tx=0x7f733400cc90 comp rx=0 tx=0).stop 2024-05-09T10:50:44.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.969+0000 7f73433f5640 1 -- 172.21.15.98:0/648187543 shutdown_connections 2024-05-09T10:50:44.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.969+0000 7f73433f5640 1 --2- 172.21.15.98:0/648187543 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f732006b500 0x7f732006d9c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.969+0000 7f73433f5640 1 --2- 172.21.15.98:0/648187543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f733c15e0c0 0x7f733c1e96e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.969+0000 7f73433f5640 1 --2- 172.21.15.98:0/648187543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f733c15dce0 0x7f733c1e91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.969+0000 7f73433f5640 1 --2- 172.21.15.98:0/648187543 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f733c0d0700 0x7f733c154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:44.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.969+0000 7f73433f5640 1 -- 172.21.15.98:0/648187543 >> 172.21.15.98:0/648187543 conn(0x7f733c0b34e0 msgr2=0x7f733c0b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:44.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.969+0000 7f73433f5640 1 -- 172.21.15.98:0/648187543 shutdown_connections 2024-05-09T10:50:44.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:44.969+0000 7f73433f5640 1 -- 172.21.15.98:0/648187543 wait complete. 2024-05-09T10:50:44.973 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T10:50:07.396423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T10:50:17.323444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T10:50:18.482315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T10:50:44.974 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T10:50:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.083+0000 7f88b70bd640 1 Processor -- start 2024-05-09T10:50:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.084+0000 7f88b70bd640 1 -- start start 2024-05-09T10:50:45.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.084+0000 7f88b70bd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f88b00d0630 0x7f88b00cca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.084+0000 7f88b70bd640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b00cc550 con 0x7f88b00b1d90 2024-05-09T10:50:45.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.084+0000 7f88b70bd640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b00cfa70 con 0x7f88b00b2170 2024-05-09T10:50:45.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.084+0000 7f88b70bd640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b00cea90 con 0x7f88b00d0630 2024-05-09T10:50:45.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.085+0000 7f88b4e59640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f88b00b1d90 0x7f88b00cff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41044/0 (socket says 172.21.15.98:41044) 2024-05-09T10:50:45.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.085+0000 7f88b4e59640 1 -- 172.21.15.98:0/600937991 learned_addr learned my addr 172.21.15.98:0/600937991 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:45.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.085+0000 7f88affff640 1 --2- 172.21.15.98:0/600937991 >> v2:172.21.15.105:3300/0 conn(0x7f88b00d0630 0x7f88b00cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.085+0000 7f88af7fe640 1 -- 172.21.15.98:0/600937991 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3599239389 0 0) 0x7f88a0001110 con 0x7f88b00b1d90 2024-05-09T10:50:45.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.085+0000 7f88af7fe640 1 -- 172.21.15.98:0/600937991 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8898003520 con 0x7f88b00b1d90 2024-05-09T10:50:45.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.085+0000 7f88af7fe640 1 -- 172.21.15.98:0/600937991 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3742046259 0 0) 0x7f8898003520 con 0x7f88b00b1d90 2024-05-09T10:50:45.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.085+0000 7f88af7fe640 1 -- 172.21.15.98:0/600937991 >> v2:172.21.15.105:3300/0 conn(0x7f88b00d0630 msgr2=0x7f88b00cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:45.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.085+0000 7f88af7fe640 1 --2- 172.21.15.98:0/600937991 >> v2:172.21.15.105:3300/0 conn(0x7f88b00d0630 0x7f88b00cca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.085+0000 7f88af7fe640 1 -- 172.21.15.98:0/600937991 >> v1:172.21.15.105:6789/0 conn(0x7f88b00b2170 legacy=0x7f88b00ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:45.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.085+0000 7f88af7fe640 1 -- 172.21.15.98:0/600937991 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b00ced70 con 0x7f88b00b1d90 2024-05-09T10:50:45.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.085+0000 7f88af7fe640 1 -- 172.21.15.98:0/600937991 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f88a0001840 con 0x7f88b00b1d90 2024-05-09T10:50:45.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.085+0000 7f88af7fe640 1 -- 172.21.15.98:0/600937991 >> v1:172.21.15.98:6789/0 conn(0x7f88b00b1d90 legacy=0x7f88b00cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:45.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.085+0000 7f88af7fe640 1 --2- 172.21.15.98:0/600937991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88980039b0 0x7f8898003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.085+0000 7f88af7fe640 1 --2- 172.21.15.98:0/600937991 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8898004800 0x7f8898004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.085+0000 7f88af7fe640 1 --2- 172.21.15.98:0/600937991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88980051c0 0x7f889800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.085+0000 7f88af7fe640 1 -- 172.21.15.98:0/600937991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b00cea90 con 0x7f88980039b0 2024-05-09T10:50:45.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.085+0000 7f88af7fe640 1 -- 172.21.15.98:0/600937991 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889800c060 con 0x7f88980051c0 2024-05-09T10:50:45.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.085+0000 7f88af7fe640 1 -- 172.21.15.98:0/600937991 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889800c260 con 0x7f8898004800 2024-05-09T10:50:45.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.085+0000 7f88b4e59640 1 --2- 172.21.15.98:0/600937991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88980039b0 0x7f8898003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.085+0000 7f88affff640 1 --2- 172.21.15.98:0/600937991 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8898004800 0x7f8898004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.085+0000 7f88b565a640 1 --2- 172.21.15.98:0/600937991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88980051c0 0x7f889800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.086+0000 7f88b4e59640 1 -- 172.21.15.98:0/600937991 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8898004800 msgr2=0x7f8898004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:45.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.086+0000 7f88b4e59640 1 --2- 172.21.15.98:0/600937991 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8898004800 0x7f8898004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.086+0000 7f88b4e59640 1 -- 172.21.15.98:0/600937991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88980051c0 msgr2=0x7f889800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:45.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.086+0000 7f88b4e59640 1 --2- 172.21.15.98:0/600937991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88980051c0 0x7f889800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.086+0000 7f88b4e59640 1 -- 172.21.15.98:0/600937991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8898003520 con 0x7f88980039b0 2024-05-09T10:50:45.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.086+0000 7f88b565a640 1 -- 172.21.15.98:0/600937991 reap_dead start 2024-05-09T10:50:45.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.086+0000 7f88b4e59640 1 --2- 172.21.15.98:0/600937991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88980039b0 0x7f8898003dd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f88a000c4c0 tx=0x7f8898005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4509d2447b7122b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:45.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.086+0000 7f88af7fe640 1 -- 172.21.15.98:0/600937991 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a000c0e0 con 0x7f88980039b0 2024-05-09T10:50:45.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.086+0000 7f88af7fe640 1 -- 172.21.15.98:0/600937991 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a0034070 con 0x7f88980039b0 2024-05-09T10:50:45.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.086+0000 7f88af7fe640 1 -- 172.21.15.98:0/600937991 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0005d70 con 0x7f88980039b0 2024-05-09T10:50:45.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.087+0000 7f88b70bd640 1 -- 172.21.15.98:0/600937991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88980039b0 msgr2=0x7f8898003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:45.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.087+0000 7f88b70bd640 1 --2- 172.21.15.98:0/600937991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88980039b0 0x7f8898003dd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f88a000c4c0 tx=0x7f8898005830 comp rx=0 tx=0).stop 2024-05-09T10:50:45.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.087+0000 7f88b70bd640 1 -- 172.21.15.98:0/600937991 shutdown_connections 2024-05-09T10:50:45.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.087+0000 7f88b70bd640 1 --2- 172.21.15.98:0/600937991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88980039b0 0x7f8898003dd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.087+0000 7f88b70bd640 1 -- 172.21.15.98:0/600937991 >> 172.21.15.98:0/600937991 conn(0x7f88b00b3450 msgr2=0x7f88b00b3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:45.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.087+0000 7f88b70bd640 1 -- 172.21.15.98:0/600937991 shutdown_connections 2024-05-09T10:50:45.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.087+0000 7f88b70bd640 1 -- 172.21.15.98:0/600937991 wait complete. 2024-05-09T10:50:45.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.088+0000 7f88b70bd640 1 Processor -- start 2024-05-09T10:50:45.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.088+0000 7f88b70bd640 1 -- start start 2024-05-09T10:50:45.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.089+0000 7f88b70bd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88980039b0 0x7f88b00c43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.089+0000 7f88b70bd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88b01504e0 0x7f88b01598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.089+0000 7f88b70bd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88b0150940 0x7f88b0159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.089+0000 7f88b70bd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b00b4790 con 0x7f88980039b0 2024-05-09T10:50:45.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.089+0000 7f88b70bd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b00cf720 con 0x7f88b0150940 2024-05-09T10:50:45.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.089+0000 7f88b70bd640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b00b4910 con 0x7f88b01504e0 2024-05-09T10:50:45.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.089+0000 7f88b4e59640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88980039b0 0x7f88b00c43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.089+0000 7f88b4e59640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88980039b0 0x7f88b00c43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34080/0 (socket says 172.21.15.98:34080) 2024-05-09T10:50:45.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.089+0000 7f88b4e59640 1 -- 172.21.15.98:0/1568342673 learned_addr learned my addr 172.21.15.98:0/1568342673 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:45.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.089+0000 7f88b565a640 1 --2- 172.21.15.98:0/1568342673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88b0150940 0x7f88b0159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.089+0000 7f88affff640 1 --2- 172.21.15.98:0/1568342673 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88b01504e0 0x7f88b01598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.089+0000 7f88b4e59640 1 -- 172.21.15.98:0/1568342673 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88b01504e0 msgr2=0x7f88b01598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:45.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.089+0000 7f88b4e59640 1 --2- 172.21.15.98:0/1568342673 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88b01504e0 0x7f88b01598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.089+0000 7f88b4e59640 1 -- 172.21.15.98:0/1568342673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88b0150940 msgr2=0x7f88b0159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:45.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.089+0000 7f88b4e59640 1 --2- 172.21.15.98:0/1568342673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88b0150940 0x7f88b0159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.089+0000 7f88b4e59640 1 -- 172.21.15.98:0/1568342673 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b00b1160 con 0x7f88980039b0 2024-05-09T10:50:45.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.089+0000 7f88b4e59640 1 --2- 172.21.15.98:0/1568342673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88980039b0 0x7f88b00c43a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f88a000c5e0 tx=0x7f88b015c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:45.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.090+0000 7f88ad7fa640 1 -- 172.21.15.98:0/1568342673 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0007480 con 0x7f88980039b0 2024-05-09T10:50:45.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.090+0000 7f88ad7fa640 1 -- 172.21.15.98:0/1568342673 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a003f030 con 0x7f88980039b0 2024-05-09T10:50:45.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.090+0000 7f88ad7fa640 1 -- 172.21.15.98:0/1568342673 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0000ed0 con 0x7f88980039b0 2024-05-09T10:50:45.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.090+0000 7f88b70bd640 1 -- 172.21.15.98:0/1568342673 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b00b1440 con 0x7f88980039b0 2024-05-09T10:50:45.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.090+0000 7f88b70bd640 1 -- 172.21.15.98:0/1568342673 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b00c48e0 con 0x7f88980039b0 2024-05-09T10:50:45.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.091+0000 7f88b70bd640 1 -- 172.21.15.98:0/1568342673 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b00b1610 con 0x7f88980039b0 2024-05-09T10:50:45.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.091+0000 7f88ad7fa640 1 -- 172.21.15.98:0/1568342673 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f88a002f460 con 0x7f88980039b0 2024-05-09T10:50:45.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.092+0000 7f88ad7fa640 1 --2- 172.21.15.98:0/1568342673 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f889c06b6a0 0x7f889c06db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.092+0000 7f88affff640 1 --2- 172.21.15.98:0/1568342673 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f889c06b6a0 0x7f889c06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.092+0000 7f88ad7fa640 1 -- 172.21.15.98:0/1568342673 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88a00aad10 con 0x7f88980039b0 2024-05-09T10:50:45.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.092+0000 7f88affff640 1 --2- 172.21.15.98:0/1568342673 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f889c06b6a0 0x7f889c06db60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f88b01f79d0 tx=0x7f88a400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:45.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.096+0000 7f88ad7fa640 1 -- 172.21.15.98:0/1568342673 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f88b00b1610 con 0x7f88980039b0 2024-05-09T10:50:45.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.267+0000 7f88b70bd640 1 -- 172.21.15.98:0/1568342673 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f88b009fab0 con 0x7f88980039b0 2024-05-09T10:50:45.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.267+0000 7f88ad7fa640 1 -- 172.21.15.98:0/1568342673 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f88a0049030 con 0x7f88980039b0 2024-05-09T10:50:45.260 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-05-09T10:50:45.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.269+0000 7f88b70bd640 1 -- 172.21.15.98:0/1568342673 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f889c06b6a0 msgr2=0x7f889c06db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:45.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.269+0000 7f88b70bd640 1 --2- 172.21.15.98:0/1568342673 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f889c06b6a0 0x7f889c06db60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f88b01f79d0 tx=0x7f88a400b040 comp rx=0 tx=0).stop 2024-05-09T10:50:45.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.269+0000 7f88b70bd640 1 -- 172.21.15.98:0/1568342673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88980039b0 msgr2=0x7f88b00c43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:45.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.269+0000 7f88b70bd640 1 --2- 172.21.15.98:0/1568342673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88980039b0 0x7f88b00c43a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f88a000c5e0 tx=0x7f88b015c710 comp rx=0 tx=0).stop 2024-05-09T10:50:45.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.269+0000 7f88b70bd640 1 -- 172.21.15.98:0/1568342673 shutdown_connections 2024-05-09T10:50:45.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.269+0000 7f88b70bd640 1 --2- 172.21.15.98:0/1568342673 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f889c06b6a0 0x7f889c06db60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.269+0000 7f88b70bd640 1 --2- 172.21.15.98:0/1568342673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88b0150940 0x7f88b0159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.269+0000 7f88b70bd640 1 --2- 172.21.15.98:0/1568342673 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88b01504e0 0x7f88b01598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.269+0000 7f88b70bd640 1 --2- 172.21.15.98:0/1568342673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88980039b0 0x7f88b00c43a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.269+0000 7f88b70bd640 1 -- 172.21.15.98:0/1568342673 >> 172.21.15.98:0/1568342673 conn(0x7f88b00b3450 msgr2=0x7f88b006fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:45.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.269+0000 7f88b70bd640 1 -- 172.21.15.98:0/1568342673 shutdown_connections 2024-05-09T10:50:45.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.269+0000 7f88b70bd640 1 -- 172.21.15.98:0/1568342673 wait complete. 2024-05-09T10:50:45.273 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T10:50:45.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.384+0000 7fcabd076640 1 Processor -- start 2024-05-09T10:50:45.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.384+0000 7fcabd076640 1 -- start start 2024-05-09T10:50:45.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.385+0000 7fcabd076640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcab815bc80 0x7fcab80cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.385+0000 7fcabd076640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcab815c060 0x7fcab80cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.385+0000 7fcabd076640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab80ce380 con 0x7fcab80cd5d0 2024-05-09T10:50:45.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.385+0000 7fcabd076640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab80ce660 con 0x7fcab815bc80 2024-05-09T10:50:45.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.385+0000 7fcabd076640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab80ce940 con 0x7fcab815c060 2024-05-09T10:50:45.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.385+0000 7fcab6d9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcab815bc80 0x7fcab80cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.385+0000 7fcab759e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fcab80cd5d0 0x7fcab815f9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41056/0 (socket says 172.21.15.98:41056) 2024-05-09T10:50:45.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.385+0000 7fcab759e640 1 -- 172.21.15.98:0/1083579180 learned_addr learned my addr 172.21.15.98:0/1083579180 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:45.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.385+0000 7fcaaffff640 1 --2- 172.21.15.98:0/1083579180 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcab815c060 0x7fcab80cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.385+0000 7fcab659c640 1 -- 172.21.15.98:0/1083579180 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3638270989 0 0) 0x7fcab80ce380 con 0x7fcab80cd5d0 2024-05-09T10:50:45.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.385+0000 7fcab659c640 1 -- 172.21.15.98:0/1083579180 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaa4003520 con 0x7fcab80cd5d0 2024-05-09T10:50:45.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.385+0000 7fcab6d9d640 1 -- 172.21.15.98:0/1083579180 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcab815c060 msgr2=0x7fcab80cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:45.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.385+0000 7fcab6d9d640 1 --2- 172.21.15.98:0/1083579180 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcab815c060 0x7fcab80cff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.385+0000 7fcab6d9d640 1 -- 172.21.15.98:0/1083579180 >> v1:172.21.15.98:6789/0 conn(0x7fcab80cd5d0 legacy=0x7fcab815f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:45.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.385+0000 7fcab6d9d640 1 -- 172.21.15.98:0/1083579180 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab80cec20 con 0x7fcab815bc80 2024-05-09T10:50:45.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.386+0000 7fcab6d9d640 1 --2- 172.21.15.98:0/1083579180 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcab815bc80 0x7fcab80cd090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80097a0 tx=0x7fcaa802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147b607b8e096cdc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:45.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.386+0000 7fcab659c640 1 -- 172.21.15.98:0/1083579180 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8037070 con 0x7fcab815bc80 2024-05-09T10:50:45.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.386+0000 7fcab659c640 1 -- 172.21.15.98:0/1083579180 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa8032440 con 0x7fcab815bc80 2024-05-09T10:50:45.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.386+0000 7fcab659c640 1 -- 172.21.15.98:0/1083579180 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa80327f0 con 0x7fcab815bc80 2024-05-09T10:50:45.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.386+0000 7fcabd076640 1 -- 172.21.15.98:0/1083579180 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcab815bc80 msgr2=0x7fcab80cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:45.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.386+0000 7fcabd076640 1 --2- 172.21.15.98:0/1083579180 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcab815bc80 0x7fcab80cd090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80097a0 tx=0x7fcaa802a180 comp rx=0 tx=0).stop 2024-05-09T10:50:45.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.386+0000 7fcabd076640 1 -- 172.21.15.98:0/1083579180 shutdown_connections 2024-05-09T10:50:45.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.386+0000 7fcabd076640 1 --2- 172.21.15.98:0/1083579180 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcab815c060 0x7fcab80cff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.386+0000 7fcabd076640 1 --2- 172.21.15.98:0/1083579180 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcab815bc80 0x7fcab80cd090 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.386+0000 7fcabd076640 1 -- 172.21.15.98:0/1083579180 >> 172.21.15.98:0/1083579180 conn(0x7fcab80d1970 msgr2=0x7fcab80d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:45.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.387+0000 7fcabd076640 1 -- 172.21.15.98:0/1083579180 shutdown_connections 2024-05-09T10:50:45.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.387+0000 7fcabd076640 1 -- 172.21.15.98:0/1083579180 wait complete. 2024-05-09T10:50:45.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.388+0000 7fcabd076640 1 Processor -- start 2024-05-09T10:50:45.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.388+0000 7fcabd076640 1 -- start start 2024-05-09T10:50:45.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcabd076640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcab80cd5d0 0x7fcab80ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcabd076640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcab815bc80 0x7fcab8152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcabd076640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcab815c060 0x7fcab80cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcabd076640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab80ce120 con 0x7fcab815c060 2024-05-09T10:50:45.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcabd076640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab80cdfa0 con 0x7fcab815bc80 2024-05-09T10:50:45.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcabd076640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab80b1ef0 con 0x7fcab80cd5d0 2024-05-09T10:50:45.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcab759e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcab815c060 0x7fcab80cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcab759e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcab815c060 0x7fcab80cb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34106/0 (socket says 172.21.15.98:34106) 2024-05-09T10:50:45.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcab759e640 1 -- 172.21.15.98:0/2444894350 learned_addr learned my addr 172.21.15.98:0/2444894350 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:45.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcab6d9d640 1 --2- 172.21.15.98:0/2444894350 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcab80cd5d0 0x7fcab80ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcaaffff640 1 --2- 172.21.15.98:0/2444894350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcab815bc80 0x7fcab8152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcab759e640 1 -- 172.21.15.98:0/2444894350 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcab80cd5d0 msgr2=0x7fcab80ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:45.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcab759e640 1 --2- 172.21.15.98:0/2444894350 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcab80cd5d0 0x7fcab80ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcab759e640 1 -- 172.21.15.98:0/2444894350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcab815bc80 msgr2=0x7fcab8152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:45.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcab759e640 1 --2- 172.21.15.98:0/2444894350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcab815bc80 0x7fcab8152cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcab759e640 1 -- 172.21.15.98:0/2444894350 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab815eb50 con 0x7fcab815c060 2024-05-09T10:50:45.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcab6d9d640 1 --2- 172.21.15.98:0/2444894350 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcab80cd5d0 0x7fcab80ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:45.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcaaffff640 1 --2- 172.21.15.98:0/2444894350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcab815bc80 0x7fcab8152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:45.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcab759e640 1 --2- 172.21.15.98:0/2444894350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcab815c060 0x7fcab80cb1c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcab000cbe0 tx=0x7fcab000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:45.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcaaf7fe640 1 -- 172.21.15.98:0/2444894350 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0017070 con 0x7fcab815c060 2024-05-09T10:50:45.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcaaf7fe640 1 -- 172.21.15.98:0/2444894350 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab000ad40 con 0x7fcab815c060 2024-05-09T10:50:45.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcabd076640 1 -- 172.21.15.98:0/2444894350 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab80c9c10 con 0x7fcab815c060 2024-05-09T10:50:45.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcaaf7fe640 1 -- 172.21.15.98:0/2444894350 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0012600 con 0x7fcab815c060 2024-05-09T10:50:45.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.389+0000 7fcabd076640 1 -- 172.21.15.98:0/2444894350 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab80ca1d0 con 0x7fcab815c060 2024-05-09T10:50:45.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.390+0000 7fcabd076640 1 -- 172.21.15.98:0/2444894350 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab806ba50 con 0x7fcab815c060 2024-05-09T10:50:45.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.391+0000 7fcaaf7fe640 1 -- 172.21.15.98:0/2444894350 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcab001a070 con 0x7fcab815c060 2024-05-09T10:50:45.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.391+0000 7fcaaf7fe640 1 --2- 172.21.15.98:0/2444894350 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fca9806b4d0 0x7fca9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.391+0000 7fcaaf7fe640 1 -- 172.21.15.98:0/2444894350 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcab0089e50 con 0x7fcab815c060 2024-05-09T10:50:45.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.391+0000 7fcab6d9d640 1 --2- 172.21.15.98:0/2444894350 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fca9806b4d0 0x7fca9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.392+0000 7fcab6d9d640 1 --2- 172.21.15.98:0/2444894350 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fca9806b4d0 0x7fca9806d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8034000 tx=0x7fcaa802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:45.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.395+0000 7fcaaf7fe640 1 -- 172.21.15.98:0/2444894350 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcab815eb50 con 0x7fcab815c060 2024-05-09T10:50:45.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.565+0000 7fcabd076640 1 -- 172.21.15.98:0/2444894350 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcab809fa40 con 0x7fcab815c060 2024-05-09T10:50:45.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.566+0000 7fcaaf7fe640 1 -- 172.21.15.98:0/2444894350 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcab0058680 con 0x7fcab815c060 2024-05-09T10:50:45.559 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2024-05-09T10:50:45.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.568+0000 7fcabd076640 1 -- 172.21.15.98:0/2444894350 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fca9806b4d0 msgr2=0x7fca9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:45.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.568+0000 7fcabd076640 1 --2- 172.21.15.98:0/2444894350 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fca9806b4d0 0x7fca9806d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8034000 tx=0x7fcaa802e000 comp rx=0 tx=0).stop 2024-05-09T10:50:45.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.568+0000 7fcabd076640 1 -- 172.21.15.98:0/2444894350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcab815c060 msgr2=0x7fcab80cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:45.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.568+0000 7fcabd076640 1 --2- 172.21.15.98:0/2444894350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcab815c060 0x7fcab80cb1c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcab000cbe0 tx=0x7fcab000a430 comp rx=0 tx=0).stop 2024-05-09T10:50:45.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.568+0000 7fcabd076640 1 -- 172.21.15.98:0/2444894350 shutdown_connections 2024-05-09T10:50:45.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.568+0000 7fcabd076640 1 --2- 172.21.15.98:0/2444894350 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fca9806b4d0 0x7fca9806d990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.568+0000 7fcabd076640 1 --2- 172.21.15.98:0/2444894350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcab815c060 0x7fcab80cb1c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.568+0000 7fcabd076640 1 --2- 172.21.15.98:0/2444894350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcab815bc80 0x7fcab8152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.568+0000 7fcabd076640 1 --2- 172.21.15.98:0/2444894350 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcab80cd5d0 0x7fcab80ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.568+0000 7fcabd076640 1 -- 172.21.15.98:0/2444894350 >> 172.21.15.98:0/2444894350 conn(0x7fcab80d1970 msgr2=0x7fcab80ae6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:45.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.568+0000 7fcabd076640 1 -- 172.21.15.98:0/2444894350 shutdown_connections 2024-05-09T10:50:45.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.568+0000 7fcabd076640 1 -- 172.21.15.98:0/2444894350 wait complete. 2024-05-09T10:50:45.572 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T10:50:45.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.682+0000 7f0a669ce640 1 Processor -- start 2024-05-09T10:50:45.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.682+0000 7f0a669ce640 1 -- start start 2024-05-09T10:50:45.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.683+0000 7f0a669ce640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0a6015db90 0x7f0a600af720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.683+0000 7f0a669ce640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a600720d0 0x7f0a6006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.683+0000 7f0a669ce640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a60072690 con 0x7f0a6006abe0 2024-05-09T10:50:45.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.683+0000 7f0a669ce640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a600afc60 con 0x7f0a6015db90 2024-05-09T10:50:45.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.683+0000 7f0a669ce640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a600b5d70 con 0x7f0a600720d0 2024-05-09T10:50:45.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.683+0000 7f0a5ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0a6015db90 0x7f0a600af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.683+0000 7f0a5ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0a6015db90 0x7f0a600af720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34116/0 (socket says 172.21.15.98:34116) 2024-05-09T10:50:45.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.683+0000 7f0a5ffff640 1 -- 172.21.15.98:0/3753948453 learned_addr learned my addr 172.21.15.98:0/3753948453 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:45.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.683+0000 7f0a5f7fe640 1 --2- 172.21.15.98:0/3753948453 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a600720d0 0x7f0a6006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.683+0000 7f0a5ffff640 1 -- 172.21.15.98:0/3753948453 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a600720d0 msgr2=0x7f0a6006d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:45.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.683+0000 7f0a5ffff640 1 --2- 172.21.15.98:0/3753948453 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a600720d0 0x7f0a6006d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.683+0000 7f0a5ffff640 1 -- 172.21.15.98:0/3753948453 >> v1:172.21.15.105:6789/0 conn(0x7f0a6006abe0 legacy=0x7f0a6006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:45.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.683+0000 7f0a5ffff640 1 -- 172.21.15.98:0/3753948453 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a600b57b0 con 0x7f0a6015db90 2024-05-09T10:50:45.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.684+0000 7f0a5ffff640 1 --2- 172.21.15.98:0/3753948453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a6015db90 0x7f0a600af720 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0a50009870 tx=0x7f0a5002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94bb85dc73b6eb58 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:45.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.684+0000 7f0a5effd640 1 -- 172.21.15.98:0/3753948453 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a50037070 con 0x7f0a6015db90 2024-05-09T10:50:45.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.684+0000 7f0a5effd640 1 -- 172.21.15.98:0/3753948453 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a50032440 con 0x7f0a6015db90 2024-05-09T10:50:45.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.684+0000 7f0a5effd640 1 -- 172.21.15.98:0/3753948453 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a50032810 con 0x7f0a6015db90 2024-05-09T10:50:45.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.685+0000 7f0a669ce640 1 -- 172.21.15.98:0/3753948453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a6015db90 msgr2=0x7f0a600af720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:45.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.685+0000 7f0a669ce640 1 --2- 172.21.15.98:0/3753948453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a6015db90 0x7f0a600af720 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0a50009870 tx=0x7f0a5002a1c0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.685+0000 7f0a669ce640 1 -- 172.21.15.98:0/3753948453 shutdown_connections 2024-05-09T10:50:45.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.685+0000 7f0a669ce640 1 --2- 172.21.15.98:0/3753948453 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a600720d0 0x7f0a6006d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.685+0000 7f0a669ce640 1 --2- 172.21.15.98:0/3753948453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a6015db90 0x7f0a600af720 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.685+0000 7f0a669ce640 1 -- 172.21.15.98:0/3753948453 >> 172.21.15.98:0/3753948453 conn(0x7f0a60066fd0 msgr2=0x7f0a6006bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:45.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.685+0000 7f0a669ce640 1 -- 172.21.15.98:0/3753948453 shutdown_connections 2024-05-09T10:50:45.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.685+0000 7f0a669ce640 1 -- 172.21.15.98:0/3753948453 wait complete. 2024-05-09T10:50:45.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.686+0000 7f0a669ce640 1 Processor -- start 2024-05-09T10:50:45.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.686+0000 7f0a669ce640 1 -- start start 2024-05-09T10:50:45.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.687+0000 7f0a669ce640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a6006abe0 0x7f0a6014b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.687+0000 7f0a669ce640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a600720d0 0x7f0a60156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.687+0000 7f0a669ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a6015db90 0x7f0a600adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.687+0000 7f0a669ce640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6006d930 con 0x7f0a600720d0 2024-05-09T10:50:45.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.687+0000 7f0a669ce640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a600d0670 con 0x7f0a6015db90 2024-05-09T10:50:45.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.687+0000 7f0a669ce640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6015df70 con 0x7f0a6006abe0 2024-05-09T10:50:45.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.687+0000 7f0a5f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a600720d0 0x7f0a60156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.687+0000 7f0a5f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a600720d0 0x7f0a60156c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34124/0 (socket says 172.21.15.98:34124) 2024-05-09T10:50:45.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.687+0000 7f0a5f7fe640 1 -- 172.21.15.98:0/93145597 learned_addr learned my addr 172.21.15.98:0/93145597 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:45.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.687+0000 7f0a5ffff640 1 --2- 172.21.15.98:0/93145597 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a6006abe0 0x7f0a6014b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.687+0000 7f0a64f6b640 1 --2- 172.21.15.98:0/93145597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a6015db90 0x7f0a600adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.687+0000 7f0a5f7fe640 1 -- 172.21.15.98:0/93145597 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a6006abe0 msgr2=0x7f0a6014b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:45.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.687+0000 7f0a5f7fe640 1 --2- 172.21.15.98:0/93145597 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a6006abe0 0x7f0a6014b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.687+0000 7f0a5f7fe640 1 -- 172.21.15.98:0/93145597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a6015db90 msgr2=0x7f0a600adfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:45.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.687+0000 7f0a5f7fe640 1 --2- 172.21.15.98:0/93145597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a6015db90 0x7f0a600adfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.687+0000 7f0a5f7fe640 1 -- 172.21.15.98:0/93145597 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a60155ef0 con 0x7f0a600720d0 2024-05-09T10:50:45.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.687+0000 7f0a5ffff640 1 --2- 172.21.15.98:0/93145597 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a6006abe0 0x7f0a6014b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:45.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.687+0000 7f0a64f6b640 1 --2- 172.21.15.98:0/93145597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a6015db90 0x7f0a600adfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:45.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.688+0000 7f0a5f7fe640 1 --2- 172.21.15.98:0/93145597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a600720d0 0x7f0a60156c90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400d800 tx=0x7f0a5400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:45.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.688+0000 7f0a5cff9640 1 -- 172.21.15.98:0/93145597 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5400b6c0 con 0x7f0a600720d0 2024-05-09T10:50:45.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.688+0000 7f0a5cff9640 1 -- 172.21.15.98:0/93145597 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a54019070 con 0x7f0a600720d0 2024-05-09T10:50:45.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.688+0000 7f0a669ce640 1 -- 172.21.15.98:0/93145597 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a60160f40 con 0x7f0a600720d0 2024-05-09T10:50:45.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.688+0000 7f0a5cff9640 1 -- 172.21.15.98:0/93145597 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5400ba80 con 0x7f0a600720d0 2024-05-09T10:50:45.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.688+0000 7f0a669ce640 1 -- 172.21.15.98:0/93145597 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a60156490 con 0x7f0a600720d0 2024-05-09T10:50:45.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.689+0000 7f0a669ce640 1 -- 172.21.15.98:0/93145597 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a6006ebe0 con 0x7f0a600720d0 2024-05-09T10:50:45.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.689+0000 7f0a5cff9640 1 -- 172.21.15.98:0/93145597 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a5400bc20 con 0x7f0a600720d0 2024-05-09T10:50:45.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.689+0000 7f0a5cff9640 1 --2- 172.21.15.98:0/93145597 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0a4c06b4d0 0x7f0a4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.690+0000 7f0a5cff9640 1 -- 172.21.15.98:0/93145597 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a5408a6e0 con 0x7f0a600720d0 2024-05-09T10:50:45.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.690+0000 7f0a5ffff640 1 --2- 172.21.15.98:0/93145597 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0a4c06b4d0 0x7f0a4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.690+0000 7f0a5ffff640 1 --2- 172.21.15.98:0/93145597 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0a4c06b4d0 0x7f0a4c06d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0a50034000 tx=0x7f0a5002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:45.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.694+0000 7f0a5cff9640 1 -- 172.21.15.98:0/93145597 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0a6006ebe0 con 0x7f0a600720d0 2024-05-09T10:50:45.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.863+0000 7f0a669ce640 1 -- 172.21.15.98:0/93145597 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0a60072dd0 con 0x7f0a600720d0 2024-05-09T10:50:45.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.864+0000 7f0a5cff9640 1 -- 172.21.15.98:0/93145597 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0a5401c360 con 0x7f0a600720d0 2024-05-09T10:50:45.857 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2024-05-09T10:50:45.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.866+0000 7f0a669ce640 1 -- 172.21.15.98:0/93145597 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0a4c06b4d0 msgr2=0x7f0a4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:45.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.866+0000 7f0a669ce640 1 --2- 172.21.15.98:0/93145597 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0a4c06b4d0 0x7f0a4c06d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0a50034000 tx=0x7f0a5002e000 comp rx=0 tx=0).stop 2024-05-09T10:50:45.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.866+0000 7f0a669ce640 1 -- 172.21.15.98:0/93145597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a600720d0 msgr2=0x7f0a60156c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:45.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.866+0000 7f0a669ce640 1 --2- 172.21.15.98:0/93145597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a600720d0 0x7f0a60156c90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400d800 tx=0x7f0a5400dcc0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.866+0000 7f0a669ce640 1 -- 172.21.15.98:0/93145597 shutdown_connections 2024-05-09T10:50:45.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.866+0000 7f0a669ce640 1 --2- 172.21.15.98:0/93145597 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0a4c06b4d0 0x7f0a4c06d990 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.866+0000 7f0a669ce640 1 --2- 172.21.15.98:0/93145597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a6015db90 0x7f0a600adfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.866+0000 7f0a669ce640 1 --2- 172.21.15.98:0/93145597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a600720d0 0x7f0a60156c90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.866+0000 7f0a669ce640 1 --2- 172.21.15.98:0/93145597 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a6006abe0 0x7f0a6014b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.866+0000 7f0a669ce640 1 -- 172.21.15.98:0/93145597 >> 172.21.15.98:0/93145597 conn(0x7f0a60066fd0 msgr2=0x7f0a60073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:45.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.866+0000 7f0a669ce640 1 -- 172.21.15.98:0/93145597 shutdown_connections 2024-05-09T10:50:45.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.866+0000 7f0a669ce640 1 -- 172.21.15.98:0/93145597 wait complete. 2024-05-09T10:50:45.870 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread' 2024-05-09T10:50:45.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.977+0000 7f88cb37e640 1 Processor -- start 2024-05-09T10:50:45.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.977+0000 7f88cb37e640 1 -- start start 2024-05-09T10:50:45.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.978+0000 7f88cb37e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f88c415db90 0x7f88c406d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.978+0000 7f88cb37e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c40cdb30 con 0x7f88c415df70 2024-05-09T10:50:45.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.978+0000 7f88cb37e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c4071f40 con 0x7f88c406abe0 2024-05-09T10:50:45.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.978+0000 7f88cb37e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c4072220 con 0x7f88c415db90 2024-05-09T10:50:45.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.978+0000 7f88c911a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f88c415db90 0x7f88c406d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.978+0000 7f88c8919640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f88c415df70 0x7f88c40713d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41058/0 (socket says 172.21.15.98:41058) 2024-05-09T10:50:45.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.978+0000 7f88c8919640 1 -- 172.21.15.98:0/996566103 learned_addr learned my addr 172.21.15.98:0/996566103 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:45.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.978+0000 7f88c911a640 1 -- 172.21.15.98:0/996566103 >> v1:172.21.15.105:6789/0 conn(0x7f88c406abe0 legacy=0x7f88c40cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:45.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.978+0000 7f88c3fff640 1 -- 172.21.15.98:0/996566103 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4117715810 0 0) 0x7f88b8001110 con 0x7f88c415df70 2024-05-09T10:50:45.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.978+0000 7f88c911a640 1 -- 172.21.15.98:0/996566103 >> v1:172.21.15.98:6789/0 conn(0x7f88c415df70 legacy=0x7f88c40713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:45.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.978+0000 7f88c911a640 1 -- 172.21.15.98:0/996566103 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c4072500 con 0x7f88c415db90 2024-05-09T10:50:45.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.979+0000 7f88c911a640 1 --2- 172.21.15.98:0/996566103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c415db90 0x7f88c406d430 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f88b40097a0 tx=0x7f88b402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b985e72d26280fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:45.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.979+0000 7f88c3fff640 1 -- 172.21.15.98:0/996566103 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4037070 con 0x7f88c415db90 2024-05-09T10:50:45.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.979+0000 7f88c3fff640 1 -- 172.21.15.98:0/996566103 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b402ae50 con 0x7f88c415db90 2024-05-09T10:50:45.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.979+0000 7f88c3fff640 1 -- 172.21.15.98:0/996566103 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4032660 con 0x7f88c415db90 2024-05-09T10:50:45.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.980+0000 7f88cb37e640 1 -- 172.21.15.98:0/996566103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c415db90 msgr2=0x7f88c406d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:45.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.980+0000 7f88cb37e640 1 --2- 172.21.15.98:0/996566103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c415db90 0x7f88c406d430 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f88b40097a0 tx=0x7f88b402a140 comp rx=0 tx=0).stop 2024-05-09T10:50:45.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.980+0000 7f88cb37e640 1 -- 172.21.15.98:0/996566103 shutdown_connections 2024-05-09T10:50:45.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.980+0000 7f88cb37e640 1 --2- 172.21.15.98:0/996566103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c415db90 0x7f88c406d430 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.980+0000 7f88cb37e640 1 -- 172.21.15.98:0/996566103 >> 172.21.15.98:0/996566103 conn(0x7f88c4066f20 msgr2=0x7f88c406bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:45.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.980+0000 7f88cb37e640 1 -- 172.21.15.98:0/996566103 shutdown_connections 2024-05-09T10:50:45.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.980+0000 7f88cb37e640 1 -- 172.21.15.98:0/996566103 wait complete. 2024-05-09T10:50:45.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.981+0000 7f88cb37e640 1 Processor -- start 2024-05-09T10:50:45.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.981+0000 7f88cb37e640 1 -- start start 2024-05-09T10:50:45.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.982+0000 7f88cb37e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c406abe0 0x7f88c40b4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.982+0000 7f88cb37e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88c415db90 0x7f88c414ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.982+0000 7f88cb37e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88c415df70 0x7f88c40c5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.982+0000 7f88cb37e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c40cefd0 con 0x7f88c406abe0 2024-05-09T10:50:45.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.982+0000 7f88cb37e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c406d970 con 0x7f88c415df70 2024-05-09T10:50:45.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.982+0000 7f88cb37e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c4071ae0 con 0x7f88c415db90 2024-05-09T10:50:45.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.982+0000 7f88c911a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c406abe0 0x7f88c40b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.982+0000 7f88c911a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c406abe0 0x7f88c40b4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34142/0 (socket says 172.21.15.98:34142) 2024-05-09T10:50:45.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.982+0000 7f88c911a640 1 -- 172.21.15.98:0/2173004005 learned_addr learned my addr 172.21.15.98:0/2173004005 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:45.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.982+0000 7f88c991b640 1 --2- 172.21.15.98:0/2173004005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88c415df70 0x7f88c40c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.982+0000 7f88c8919640 1 --2- 172.21.15.98:0/2173004005 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88c415db90 0x7f88c414ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.982+0000 7f88c911a640 1 -- 172.21.15.98:0/2173004005 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88c415db90 msgr2=0x7f88c414ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:45.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.982+0000 7f88c911a640 1 --2- 172.21.15.98:0/2173004005 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88c415db90 0x7f88c414ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.982+0000 7f88c911a640 1 -- 172.21.15.98:0/2173004005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88c415df70 msgr2=0x7f88c40c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:45.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.982+0000 7f88c911a640 1 --2- 172.21.15.98:0/2173004005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88c415df70 0x7f88c40c5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:45.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.982+0000 7f88c911a640 1 -- 172.21.15.98:0/2173004005 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c40b3680 con 0x7f88c406abe0 2024-05-09T10:50:45.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.982+0000 7f88c8919640 1 --2- 172.21.15.98:0/2173004005 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88c415db90 0x7f88c414ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:45.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.983+0000 7f88c911a640 1 --2- 172.21.15.98:0/2173004005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c406abe0 0x7f88c40b4720 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f88b40097a0 tx=0x7f88b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:45.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.983+0000 7f88c1ffb640 1 -- 172.21.15.98:0/2173004005 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4042070 con 0x7f88c406abe0 2024-05-09T10:50:45.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.983+0000 7f88c1ffb640 1 -- 172.21.15.98:0/2173004005 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b403e670 con 0x7f88c406abe0 2024-05-09T10:50:45.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.983+0000 7f88cb37e640 1 -- 172.21.15.98:0/2173004005 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c40b3960 con 0x7f88c406abe0 2024-05-09T10:50:45.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.983+0000 7f88c1ffb640 1 -- 172.21.15.98:0/2173004005 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4037040 con 0x7f88c406abe0 2024-05-09T10:50:45.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.983+0000 7f88cb37e640 1 -- 172.21.15.98:0/2173004005 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c40b3f20 con 0x7f88c406abe0 2024-05-09T10:50:45.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.984+0000 7f88cb37e640 1 -- 172.21.15.98:0/2173004005 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c4069cb0 con 0x7f88c406abe0 2024-05-09T10:50:45.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.984+0000 7f88c1ffb640 1 -- 172.21.15.98:0/2173004005 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f88b4045070 con 0x7f88c406abe0 2024-05-09T10:50:45.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.985+0000 7f88c1ffb640 1 --2- 172.21.15.98:0/2173004005 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f88b006b4d0 0x7f88b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:45.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.985+0000 7f88c1ffb640 1 -- 172.21.15.98:0/2173004005 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88b40aa8e0 con 0x7f88c406abe0 2024-05-09T10:50:45.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.985+0000 7f88c8919640 1 --2- 172.21.15.98:0/2173004005 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f88b006b4d0 0x7f88b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:45.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.985+0000 7f88c8919640 1 --2- 172.21.15.98:0/2173004005 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f88b006b4d0 0x7f88b006d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f88c415d580 tx=0x7f88b8007ca0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:45.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:45.989+0000 7f88c1ffb640 1 -- 172.21.15.98:0/2173004005 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f88b40791a0 con 0x7f88c406abe0 2024-05-09T10:50:46.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.226+0000 7f88cb37e640 1 -- 172.21.15.98:0/2173004005 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread"]} v 0) v1 -- 0x7f88c40d05a0 con 0x7f88c406abe0 2024-05-09T10:50:46.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.307+0000 7f88c1ffb640 1 -- 172.21.15.98:0/2173004005 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread"]}]=0 v51) v1 ==== 170+0+0 (secure 0 0 0) 0x7f88b407d240 con 0x7f88c406abe0 2024-05-09T10:50:46.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.309+0000 7f88cb37e640 1 -- 172.21.15.98:0/2173004005 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f88b006b4d0 msgr2=0x7f88b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:46.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.309+0000 7f88cb37e640 1 --2- 172.21.15.98:0/2173004005 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f88b006b4d0 0x7f88b006d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f88c415d580 tx=0x7f88b8007ca0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.309+0000 7f88cb37e640 1 -- 172.21.15.98:0/2173004005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c406abe0 msgr2=0x7f88c40b4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:46.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.309+0000 7f88cb37e640 1 --2- 172.21.15.98:0/2173004005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c406abe0 0x7f88c40b4720 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f88b40097a0 tx=0x7f88b40046c0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.309+0000 7f88cb37e640 1 -- 172.21.15.98:0/2173004005 shutdown_connections 2024-05-09T10:50:46.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.309+0000 7f88cb37e640 1 --2- 172.21.15.98:0/2173004005 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f88b006b4d0 0x7f88b006d990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.309+0000 7f88cb37e640 1 --2- 172.21.15.98:0/2173004005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88c415df70 0x7f88c40c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.309+0000 7f88cb37e640 1 --2- 172.21.15.98:0/2173004005 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88c415db90 0x7f88c414ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.309+0000 7f88cb37e640 1 --2- 172.21.15.98:0/2173004005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88c406abe0 0x7f88c40b4720 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.309+0000 7f88cb37e640 1 -- 172.21.15.98:0/2173004005 >> 172.21.15.98:0/2173004005 conn(0x7f88c4066f20 msgr2=0x7f88c40b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:46.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.309+0000 7f88cb37e640 1 -- 172.21.15.98:0/2173004005 shutdown_connections 2024-05-09T10:50:46.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.309+0000 7f88cb37e640 1 -- 172.21.15.98:0/2173004005 wait complete. 2024-05-09T10:50:46.313 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-09T10:50:46.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.423+0000 7fefd91c8640 1 Processor -- start 2024-05-09T10:50:46.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.423+0000 7fefd91c8640 1 -- start start 2024-05-09T10:50:46.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.424+0000 7fefd91c8640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fefd415f140 0x7fefd40af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:46.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.424+0000 7fefd91c8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefd406e4b0 0x7fefd415fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:46.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.424+0000 7fefd91c8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fefd4160320 0x7fefd40d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:46.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.424+0000 7fefd91c8640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd415f520 con 0x7fefd415f140 2024-05-09T10:50:46.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.424+0000 7fefd91c8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd406ea70 con 0x7fefd4160320 2024-05-09T10:50:46.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.424+0000 7fefd91c8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd406ed50 con 0x7fefd406e4b0 2024-05-09T10:50:46.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.424+0000 7fefd2d9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fefd415f140 0x7fefd40af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:46.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.424+0000 7fefd2d9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fefd415f140 0x7fefd40af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34154/0 (socket says 172.21.15.98:34154) 2024-05-09T10:50:46.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.424+0000 7fefd2d9d640 1 -- 172.21.15.98:0/1266017775 learned_addr learned my addr 172.21.15.98:0/1266017775 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:46.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.424+0000 7fefcbfff640 1 --2- 172.21.15.98:0/1266017775 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefd406e4b0 0x7fefd415fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:46.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.424+0000 7fefd359e640 1 --2- 172.21.15.98:0/1266017775 >> v2:172.21.15.105:3300/0 conn(0x7fefd4160320 0x7fefd40d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:46.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.424+0000 7fefd2d9d640 1 -- 172.21.15.98:0/1266017775 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefd406e4b0 msgr2=0x7fefd415fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:46.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.424+0000 7fefd2d9d640 1 --2- 172.21.15.98:0/1266017775 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefd406e4b0 0x7fefd415fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.424+0000 7fefd2d9d640 1 -- 172.21.15.98:0/1266017775 >> v2:172.21.15.105:3300/0 conn(0x7fefd4160320 msgr2=0x7fefd40d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:46.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.424+0000 7fefd2d9d640 1 --2- 172.21.15.98:0/1266017775 >> v2:172.21.15.105:3300/0 conn(0x7fefd4160320 0x7fefd40d03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.424+0000 7fefd2d9d640 1 -- 172.21.15.98:0/1266017775 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd406f030 con 0x7fefd415f140 2024-05-09T10:50:46.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.425+0000 7fefd2d9d640 1 --2- 172.21.15.98:0/1266017775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefd415f140 0x7fefd40af9a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fefd40cd3c0 tx=0x7fefc402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e4d2ef3617ee0e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:46.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.425+0000 7fefd259c640 1 -- 172.21.15.98:0/1266017775 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc4037070 con 0x7fefd415f140 2024-05-09T10:50:46.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.425+0000 7fefd259c640 1 -- 172.21.15.98:0/1266017775 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc4032440 con 0x7fefd415f140 2024-05-09T10:50:46.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.425+0000 7fefd259c640 1 -- 172.21.15.98:0/1266017775 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc4032810 con 0x7fefd415f140 2024-05-09T10:50:46.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.425+0000 7fefd91c8640 1 -- 172.21.15.98:0/1266017775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefd415f140 msgr2=0x7fefd40af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:46.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.425+0000 7fefd91c8640 1 --2- 172.21.15.98:0/1266017775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefd415f140 0x7fefd40af9a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fefd40cd3c0 tx=0x7fefc402a1e0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.426+0000 7fefd91c8640 1 -- 172.21.15.98:0/1266017775 shutdown_connections 2024-05-09T10:50:46.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.426+0000 7fefd91c8640 1 --2- 172.21.15.98:0/1266017775 >> v2:172.21.15.105:3300/0 conn(0x7fefd4160320 0x7fefd40d03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.426+0000 7fefd91c8640 1 --2- 172.21.15.98:0/1266017775 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefd406e4b0 0x7fefd415fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.426+0000 7fefd91c8640 1 --2- 172.21.15.98:0/1266017775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefd415f140 0x7fefd40af9a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.426+0000 7fefd91c8640 1 -- 172.21.15.98:0/1266017775 >> 172.21.15.98:0/1266017775 conn(0x7fefd4066fd0 msgr2=0x7fefd406bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:46.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.426+0000 7fefd91c8640 1 -- 172.21.15.98:0/1266017775 shutdown_connections 2024-05-09T10:50:46.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.426+0000 7fefd91c8640 1 -- 172.21.15.98:0/1266017775 wait complete. 2024-05-09T10:50:46.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.427+0000 7fefd91c8640 1 Processor -- start 2024-05-09T10:50:46.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.427+0000 7fefd91c8640 1 -- start start 2024-05-09T10:50:46.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.428+0000 7fefd91c8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefd406e4b0 0x7fefd40b3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:46.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.428+0000 7fefd91c8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefd415f140 0x7fefd41f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:46.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.428+0000 7fefd91c8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefd4160320 0x7fefd40b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:46.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.428+0000 7fefd91c8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd40d0ab0 con 0x7fefd4160320 2024-05-09T10:50:46.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.428+0000 7fefd91c8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd40d0930 con 0x7fefd415f140 2024-05-09T10:50:46.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.428+0000 7fefd91c8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd40d0c30 con 0x7fefd406e4b0 2024-05-09T10:50:46.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.428+0000 7fefd359e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefd4160320 0x7fefd40b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:46.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.428+0000 7fefd359e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefd4160320 0x7fefd40b4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34168/0 (socket says 172.21.15.98:34168) 2024-05-09T10:50:46.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.428+0000 7fefd359e640 1 -- 172.21.15.98:0/3257084407 learned_addr learned my addr 172.21.15.98:0/3257084407 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:46.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.428+0000 7fefcbfff640 1 --2- 172.21.15.98:0/3257084407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefd415f140 0x7fefd41f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:46.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.428+0000 7fefd2d9d640 1 --2- 172.21.15.98:0/3257084407 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefd406e4b0 0x7fefd40b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:46.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.428+0000 7fefd359e640 1 -- 172.21.15.98:0/3257084407 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefd406e4b0 msgr2=0x7fefd40b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:46.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.428+0000 7fefd359e640 1 --2- 172.21.15.98:0/3257084407 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefd406e4b0 0x7fefd40b3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.428+0000 7fefd359e640 1 -- 172.21.15.98:0/3257084407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefd415f140 msgr2=0x7fefd41f3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:46.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.428+0000 7fefd359e640 1 --2- 172.21.15.98:0/3257084407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefd415f140 0x7fefd41f3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.428+0000 7fefd359e640 1 -- 172.21.15.98:0/3257084407 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd40b2890 con 0x7fefd4160320 2024-05-09T10:50:46.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.428+0000 7fefd359e640 1 --2- 172.21.15.98:0/3257084407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefd4160320 0x7fefd40b4850 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fefcc007e40 tx=0x7fefcc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:46.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.429+0000 7fefcb7fe640 1 -- 172.21.15.98:0/3257084407 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc014620 con 0x7fefd4160320 2024-05-09T10:50:46.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.429+0000 7fefcb7fe640 1 -- 172.21.15.98:0/3257084407 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefcc019070 con 0x7fefd4160320 2024-05-09T10:50:46.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.429+0000 7fefd91c8640 1 -- 172.21.15.98:0/3257084407 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd40b2b70 con 0x7fefd4160320 2024-05-09T10:50:46.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.429+0000 7fefcb7fe640 1 -- 172.21.15.98:0/3257084407 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc014a20 con 0x7fefd4160320 2024-05-09T10:50:46.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.429+0000 7fefd91c8640 1 -- 172.21.15.98:0/3257084407 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd40b3130 con 0x7fefd4160320 2024-05-09T10:50:46.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.430+0000 7fefd91c8640 1 -- 172.21.15.98:0/3257084407 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd40b3370 con 0x7fefd4160320 2024-05-09T10:50:46.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.430+0000 7fefcb7fe640 1 -- 172.21.15.98:0/3257084407 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fefcc014bc0 con 0x7fefd4160320 2024-05-09T10:50:46.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.431+0000 7fefcb7fe640 1 --2- 172.21.15.98:0/3257084407 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fefb406b4d0 0x7fefb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:46.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.431+0000 7fefcb7fe640 1 -- 172.21.15.98:0/3257084407 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fefcc08aaa0 con 0x7fefd4160320 2024-05-09T10:50:46.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.431+0000 7fefd2d9d640 1 --2- 172.21.15.98:0/3257084407 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fefb406b4d0 0x7fefb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:46.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.431+0000 7fefd2d9d640 1 --2- 172.21.15.98:0/3257084407 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fefb406b4d0 0x7fefb406d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fefc4034000 tx=0x7fefc402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:46.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.435+0000 7fefcb7fe640 1 -- 172.21.15.98:0/3257084407 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fefd40b3370 con 0x7fefd4160320 2024-05-09T10:50:46.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.604+0000 7fefd91c8640 1 -- 172.21.15.98:0/3257084407 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fefd4072dd0 con 0x7fefd4160320 2024-05-09T10:50:46.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.607+0000 7fefcb7fe640 1 -- 172.21.15.98:0/3257084407 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fefcc059920 con 0x7fefd4160320 2024-05-09T10:50:46.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.609+0000 7fefd91c8640 1 -- 172.21.15.98:0/3257084407 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fefb406b4d0 msgr2=0x7fefb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:46.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.609+0000 7fefd91c8640 1 --2- 172.21.15.98:0/3257084407 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fefb406b4d0 0x7fefb406d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fefc4034000 tx=0x7fefc402e000 comp rx=0 tx=0).stop 2024-05-09T10:50:46.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.609+0000 7fefd91c8640 1 -- 172.21.15.98:0/3257084407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefd4160320 msgr2=0x7fefd40b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:46.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.609+0000 7fefd91c8640 1 --2- 172.21.15.98:0/3257084407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefd4160320 0x7fefd40b4850 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fefcc007e40 tx=0x7fefcc00e4b0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.609+0000 7fefd91c8640 1 -- 172.21.15.98:0/3257084407 shutdown_connections 2024-05-09T10:50:46.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.609+0000 7fefd91c8640 1 --2- 172.21.15.98:0/3257084407 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fefb406b4d0 0x7fefb406d990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.609+0000 7fefd91c8640 1 --2- 172.21.15.98:0/3257084407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefd4160320 0x7fefd40b4850 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.609+0000 7fefd91c8640 1 --2- 172.21.15.98:0/3257084407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefd415f140 0x7fefd41f3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.609+0000 7fefd91c8640 1 --2- 172.21.15.98:0/3257084407 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefd406e4b0 0x7fefd40b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.609+0000 7fefd91c8640 1 -- 172.21.15.98:0/3257084407 >> 172.21.15.98:0/3257084407 conn(0x7fefd4066fd0 msgr2=0x7fefd415a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:46.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.609+0000 7fefd91c8640 1 -- 172.21.15.98:0/3257084407 shutdown_connections 2024-05-09T10:50:46.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.609+0000 7fefd91c8640 1 -- 172.21.15.98:0/3257084407 wait complete. 2024-05-09T10:50:46.613 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-09T10:50:46.614 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T10:50:46.650 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T10:50:46.696 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-09T10:50:46.696 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:50:46.696 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:50:46.712 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:50:46.712 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:50:46.768 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:50:46.768 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:50:46.824 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:46.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.973+0000 7f33e01e2640 1 Processor -- start 2024-05-09T10:50:46.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.973+0000 7f33e01e2640 1 -- start start 2024-05-09T10:50:46.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.974+0000 7f33e01e2640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33d806bbf0 0x7f33d806bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:46.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.974+0000 7f33e01e2640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f33d80ce2c0 0x7f33d80ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:46.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.974+0000 7f33e01e2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f33d8072310 0x7f33d80727a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:46.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.974+0000 7f33e01e2640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33d80cc1c0 con 0x7f33d80ce2c0 2024-05-09T10:50:46.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.974+0000 7f33e01e2640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33d806f2c0 con 0x7f33d8072310 2024-05-09T10:50:46.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.974+0000 7f33e01e2640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d80cbf50 con 0x7f33d806bbf0 2024-05-09T10:50:46.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.974+0000 7f33dd77d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f33d80ce2c0 0x7f33d80ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:46.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.974+0000 7f33dd77d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f33d80ce2c0 0x7f33d80ce720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34172/0 (socket says 172.21.15.98:34172) 2024-05-09T10:50:46.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.974+0000 7f33dd77d640 1 -- 172.21.15.98:0/1156259638 learned_addr learned my addr 172.21.15.98:0/1156259638 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:46.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.974+0000 7f33de77f640 1 --2- 172.21.15.98:0/1156259638 >> v2:172.21.15.105:3300/0 conn(0x7f33d8072310 0x7f33d80727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:46.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.974+0000 7f33ddf7e640 1 --2- 172.21.15.98:0/1156259638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33d806bbf0 0x7f33d806bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:46.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.974+0000 7f33dd77d640 1 -- 172.21.15.98:0/1156259638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33d806bbf0 msgr2=0x7f33d806bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:46.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.974+0000 7f33dd77d640 1 --2- 172.21.15.98:0/1156259638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33d806bbf0 0x7f33d806bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.974+0000 7f33dd77d640 1 -- 172.21.15.98:0/1156259638 >> v2:172.21.15.105:3300/0 conn(0x7f33d8072310 msgr2=0x7f33d80727a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:46.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.974+0000 7f33dd77d640 1 --2- 172.21.15.98:0/1156259638 >> v2:172.21.15.105:3300/0 conn(0x7f33d8072310 0x7f33d80727a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.974+0000 7f33dd77d640 1 -- 172.21.15.98:0/1156259638 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d806c510 con 0x7f33d80ce2c0 2024-05-09T10:50:46.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.974+0000 7f33de77f640 1 --2- 172.21.15.98:0/1156259638 >> v2:172.21.15.105:3300/0 conn(0x7f33d8072310 0x7f33d80727a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:46.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.975+0000 7f33dd77d640 1 --2- 172.21.15.98:0/1156259638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33d80ce2c0 0x7f33d80ce720 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f33d809e0d0 tx=0x7f33c802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51392ceb3c84d87d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:46.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.975+0000 7f33dcf7c640 1 -- 172.21.15.98:0/1156259638 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c8037070 con 0x7f33d80ce2c0 2024-05-09T10:50:46.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.975+0000 7f33dcf7c640 1 -- 172.21.15.98:0/1156259638 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c802ae00 con 0x7f33d80ce2c0 2024-05-09T10:50:46.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.975+0000 7f33dcf7c640 1 -- 172.21.15.98:0/1156259638 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c8032660 con 0x7f33d80ce2c0 2024-05-09T10:50:46.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.976+0000 7f33e01e2640 1 -- 172.21.15.98:0/1156259638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33d80ce2c0 msgr2=0x7f33d80ce720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:46.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.976+0000 7f33e01e2640 1 --2- 172.21.15.98:0/1156259638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33d80ce2c0 0x7f33d80ce720 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f33d809e0d0 tx=0x7f33c802a0b0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.976+0000 7f33e01e2640 1 -- 172.21.15.98:0/1156259638 shutdown_connections 2024-05-09T10:50:46.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.976+0000 7f33e01e2640 1 --2- 172.21.15.98:0/1156259638 >> v2:172.21.15.105:3300/0 conn(0x7f33d8072310 0x7f33d80727a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.976+0000 7f33e01e2640 1 --2- 172.21.15.98:0/1156259638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33d80ce2c0 0x7f33d80ce720 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.976+0000 7f33e01e2640 1 --2- 172.21.15.98:0/1156259638 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33d806bbf0 0x7f33d806bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.976+0000 7f33e01e2640 1 -- 172.21.15.98:0/1156259638 >> 172.21.15.98:0/1156259638 conn(0x7f33d80ccaa0 msgr2=0x7f33d80cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:46.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.976+0000 7f33e01e2640 1 -- 172.21.15.98:0/1156259638 shutdown_connections 2024-05-09T10:50:46.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.976+0000 7f33e01e2640 1 -- 172.21.15.98:0/1156259638 wait complete. 2024-05-09T10:50:46.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.977+0000 7f33e01e2640 1 Processor -- start 2024-05-09T10:50:46.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.977+0000 7f33e01e2640 1 -- start start 2024-05-09T10:50:46.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.978+0000 7f33e01e2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33d806bbf0 0x7f33d81e6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:46.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.978+0000 7f33e01e2640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33d8072310 0x7f33d809d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:46.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.978+0000 7f33e01e2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33d80ce2c0 0x7f33d809d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:46.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.978+0000 7f33e01e2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d80c37d0 con 0x7f33d806bbf0 2024-05-09T10:50:46.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.978+0000 7f33e01e2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d80cb820 con 0x7f33d80ce2c0 2024-05-09T10:50:46.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.978+0000 7f33e01e2640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d80c3950 con 0x7f33d8072310 2024-05-09T10:50:46.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.978+0000 7f33ddf7e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33d806bbf0 0x7f33d81e6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:46.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.978+0000 7f33ddf7e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33d806bbf0 0x7f33d81e6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34188/0 (socket says 172.21.15.98:34188) 2024-05-09T10:50:46.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.978+0000 7f33ddf7e640 1 -- 172.21.15.98:0/587443528 learned_addr learned my addr 172.21.15.98:0/587443528 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:46.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.978+0000 7f33ddf7e640 1 -- 172.21.15.98:0/587443528 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33d8072310 msgr2=0x7f33d809d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:46.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.978+0000 7f33ddf7e640 1 --2- 172.21.15.98:0/587443528 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33d8072310 0x7f33d809d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.978+0000 7f33ddf7e640 1 -- 172.21.15.98:0/587443528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33d80ce2c0 msgr2=0x7f33d809d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:46.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.978+0000 7f33de77f640 1 --2- 172.21.15.98:0/587443528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33d80ce2c0 0x7f33d809d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:46.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.978+0000 7f33ddf7e640 1 --2- 172.21.15.98:0/587443528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33d80ce2c0 0x7f33d809d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:46.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.978+0000 7f33ddf7e640 1 -- 172.21.15.98:0/587443528 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d809c190 con 0x7f33d806bbf0 2024-05-09T10:50:46.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.978+0000 7f33ddf7e640 1 --2- 172.21.15.98:0/587443528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33d806bbf0 0x7f33d81e6e20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f33d400d800 tx=0x7f33d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:46.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.979+0000 7f33ce7fc640 1 -- 172.21.15.98:0/587443528 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d4018020 con 0x7f33d806bbf0 2024-05-09T10:50:46.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.979+0000 7f33ce7fc640 1 -- 172.21.15.98:0/587443528 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d40096a0 con 0x7f33d806bbf0 2024-05-09T10:50:46.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.979+0000 7f33ce7fc640 1 -- 172.21.15.98:0/587443528 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d4009a70 con 0x7f33d806bbf0 2024-05-09T10:50:46.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.979+0000 7f33e01e2640 1 -- 172.21.15.98:0/587443528 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d809c470 con 0x7f33d806bbf0 2024-05-09T10:50:46.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.979+0000 7f33e01e2640 1 -- 172.21.15.98:0/587443528 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d809ca30 con 0x7f33d806bbf0 2024-05-09T10:50:46.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.980+0000 7f33e01e2640 1 -- 172.21.15.98:0/587443528 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d8069cb0 con 0x7f33d806bbf0 2024-05-09T10:50:46.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.980+0000 7f33ce7fc640 1 -- 172.21.15.98:0/587443528 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f33d401b020 con 0x7f33d806bbf0 2024-05-09T10:50:46.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.981+0000 7f33ce7fc640 1 --2- 172.21.15.98:0/587443528 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f33bc06b4d0 0x7f33bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:46.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.981+0000 7f33dd77d640 1 --2- 172.21.15.98:0/587443528 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f33bc06b4d0 0x7f33bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:46.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.981+0000 7f33ce7fc640 1 -- 172.21.15.98:0/587443528 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33d408a3e0 con 0x7f33d806bbf0 2024-05-09T10:50:46.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.981+0000 7f33dd77d640 1 --2- 172.21.15.98:0/587443528 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f33bc06b4d0 0x7f33bc06d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f33c8034000 tx=0x7f33c802c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:46.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:46.985+0000 7f33ce7fc640 1 -- 172.21.15.98:0/587443528 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f33d4022030 con 0x7f33d806bbf0 2024-05-09T10:50:47.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.159+0000 7f33e01e2640 1 -- 172.21.15.98:0/587443528 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33d80c31d0 con 0x7f33d806bbf0 2024-05-09T10:50:47.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.160+0000 7f33ce7fc640 1 -- 172.21.15.98:0/587443528 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f33d40592f0 con 0x7f33d806bbf0 2024-05-09T10:50:47.154 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:47.154 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":20,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:21.176419+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:50:17.323444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T10:50:18.482315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:47.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.162+0000 7f33e01e2640 1 -- 172.21.15.98:0/587443528 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f33bc06b4d0 msgr2=0x7f33bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:47.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.162+0000 7f33e01e2640 1 --2- 172.21.15.98:0/587443528 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f33bc06b4d0 0x7f33bc06d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f33c8034000 tx=0x7f33c802c000 comp rx=0 tx=0).stop 2024-05-09T10:50:47.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.162+0000 7f33e01e2640 1 -- 172.21.15.98:0/587443528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33d806bbf0 msgr2=0x7f33d81e6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:47.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.162+0000 7f33e01e2640 1 --2- 172.21.15.98:0/587443528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33d806bbf0 0x7f33d81e6e20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f33d400d800 tx=0x7f33d400dcc0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.162+0000 7f33e01e2640 1 -- 172.21.15.98:0/587443528 shutdown_connections 2024-05-09T10:50:47.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.162+0000 7f33e01e2640 1 --2- 172.21.15.98:0/587443528 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f33bc06b4d0 0x7f33bc06d990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.162+0000 7f33e01e2640 1 --2- 172.21.15.98:0/587443528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33d80ce2c0 0x7f33d809d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.162+0000 7f33e01e2640 1 --2- 172.21.15.98:0/587443528 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33d8072310 0x7f33d809d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.162+0000 7f33e01e2640 1 --2- 172.21.15.98:0/587443528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33d806bbf0 0x7f33d81e6e20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.162+0000 7f33e01e2640 1 -- 172.21.15.98:0/587443528 >> 172.21.15.98:0/587443528 conn(0x7f33d80ccaa0 msgr2=0x7f33d809e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:47.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.162+0000 7f33e01e2640 1 -- 172.21.15.98:0/587443528 shutdown_connections 2024-05-09T10:50:47.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.163+0000 7f33e01e2640 1 -- 172.21.15.98:0/587443528 wait complete. 2024-05-09T10:50:47.167 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T10:50:07.396423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T10:50:17.323444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T10:50:18.482315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T10:50:47.168 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T10:50:47.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.277+0000 7f952f8cb640 1 Processor -- start 2024-05-09T10:50:47.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.277+0000 7f952f8cb640 1 -- start start 2024-05-09T10:50:47.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.278+0000 7f952f8cb640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f95280b2d40 0x7f952806c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:47.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.278+0000 7f952f8cb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9528162240 0x7f952815d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:47.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.278+0000 7f952f8cb640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95280d0090 con 0x7f95280cfbb0 2024-05-09T10:50:47.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.278+0000 7f952f8cb640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95280d0370 con 0x7f95280b2d40 2024-05-09T10:50:47.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.278+0000 7f952f8cb640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95280d0650 con 0x7f9528162240 2024-05-09T10:50:47.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.278+0000 7f952d667640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f95280b2d40 0x7f952806c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:47.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.278+0000 7f952ce66640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f95280cfbb0 0x7f9528161b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41064/0 (socket says 172.21.15.98:41064) 2024-05-09T10:50:47.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.278+0000 7f952ce66640 1 -- 172.21.15.98:0/3815090049 learned_addr learned my addr 172.21.15.98:0/3815090049 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:47.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.279+0000 7f952d667640 1 -- 172.21.15.98:0/3815090049 >> v2:172.21.15.105:3300/0 conn(0x7f9528162240 msgr2=0x7f952815d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:47.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.279+0000 7f952d667640 1 --2- 172.21.15.98:0/3815090049 >> v2:172.21.15.105:3300/0 conn(0x7f9528162240 0x7f952815d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.279+0000 7f952d667640 1 -- 172.21.15.98:0/3815090049 >> v1:172.21.15.98:6789/0 conn(0x7f95280cfbb0 legacy=0x7f9528161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:47.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.279+0000 7f951ffff640 1 -- 172.21.15.98:0/3815090049 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 916012660 0 0) 0x7f95280d0090 con 0x7f95280cfbb0 2024-05-09T10:50:47.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.279+0000 7f952d667640 1 -- 172.21.15.98:0/3815090049 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95280d0930 con 0x7f95280b2d40 2024-05-09T10:50:47.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.279+0000 7f952d667640 1 --2- 172.21.15.98:0/3815090049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95280b2d40 0x7f952806c910 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f95280d2950 tx=0x7f952402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=968e05a531f3b851 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:47.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.279+0000 7f951ffff640 1 -- 172.21.15.98:0/3815090049 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524037070 con 0x7f95280b2d40 2024-05-09T10:50:47.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.279+0000 7f951ffff640 1 -- 172.21.15.98:0/3815090049 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9524032440 con 0x7f95280b2d40 2024-05-09T10:50:47.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.279+0000 7f951ffff640 1 -- 172.21.15.98:0/3815090049 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952402ae60 con 0x7f95280b2d40 2024-05-09T10:50:47.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.280+0000 7f952f8cb640 1 -- 172.21.15.98:0/3815090049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95280b2d40 msgr2=0x7f952806c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:47.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.280+0000 7f952f8cb640 1 --2- 172.21.15.98:0/3815090049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95280b2d40 0x7f952806c910 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f95280d2950 tx=0x7f952402a110 comp rx=0 tx=0).stop 2024-05-09T10:50:47.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.280+0000 7f952f8cb640 1 -- 172.21.15.98:0/3815090049 shutdown_connections 2024-05-09T10:50:47.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.280+0000 7f952f8cb640 1 --2- 172.21.15.98:0/3815090049 >> v2:172.21.15.105:3300/0 conn(0x7f9528162240 0x7f952815d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.280+0000 7f952f8cb640 1 --2- 172.21.15.98:0/3815090049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95280b2d40 0x7f952806c910 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.280+0000 7f952f8cb640 1 -- 172.21.15.98:0/3815090049 >> 172.21.15.98:0/3815090049 conn(0x7f95280d1970 msgr2=0x7f95280d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:47.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.280+0000 7f952f8cb640 1 -- 172.21.15.98:0/3815090049 shutdown_connections 2024-05-09T10:50:47.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.280+0000 7f952f8cb640 1 -- 172.21.15.98:0/3815090049 wait complete. 2024-05-09T10:50:47.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.281+0000 7f952f8cb640 1 Processor -- start 2024-05-09T10:50:47.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.281+0000 7f952f8cb640 1 -- start start 2024-05-09T10:50:47.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.282+0000 7f952f8cb640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f95280b2d40 0x7f95281f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:47.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.282+0000 7f952f8cb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95280cfbb0 0x7f95280d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:47.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.282+0000 7f952f8cb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9528162240 0x7f9528148500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:47.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.282+0000 7f952f8cb640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95280b32a0 con 0x7f95280cfbb0 2024-05-09T10:50:47.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.282+0000 7f952f8cb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95280b3120 con 0x7f9528162240 2024-05-09T10:50:47.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.282+0000 7f952f8cb640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952806ce50 con 0x7f95280b2d40 2024-05-09T10:50:47.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.282+0000 7f952ce66640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95280cfbb0 0x7f95280d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:47.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.282+0000 7f952ce66640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95280cfbb0 0x7f95280d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34212/0 (socket says 172.21.15.98:34212) 2024-05-09T10:50:47.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.282+0000 7f952ce66640 1 -- 172.21.15.98:0/261256167 learned_addr learned my addr 172.21.15.98:0/261256167 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:47.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.282+0000 7f952ce66640 1 -- 172.21.15.98:0/261256167 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f95280b2d40 msgr2=0x7f95281f7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:47.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.282+0000 7f952ce66640 1 --2- 172.21.15.98:0/261256167 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f95280b2d40 0x7f95281f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.282+0000 7f952ce66640 1 -- 172.21.15.98:0/261256167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9528162240 msgr2=0x7f9528148500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:47.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.282+0000 7f952ce66640 1 --2- 172.21.15.98:0/261256167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9528162240 0x7f9528148500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.282+0000 7f952ce66640 1 -- 172.21.15.98:0/261256167 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95281f84e0 con 0x7f95280cfbb0 2024-05-09T10:50:47.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.282+0000 7f952ce66640 1 --2- 172.21.15.98:0/261256167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95280cfbb0 0x7f95280d1fc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f951800c430 tx=0x7f951800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:47.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.283+0000 7f951dffb640 1 -- 172.21.15.98:0/261256167 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95180074e0 con 0x7f95280cfbb0 2024-05-09T10:50:47.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.283+0000 7f951dffb640 1 -- 172.21.15.98:0/261256167 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95180047e0 con 0x7f95280cfbb0 2024-05-09T10:50:47.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.283+0000 7f951dffb640 1 -- 172.21.15.98:0/261256167 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9518004c10 con 0x7f95280cfbb0 2024-05-09T10:50:47.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.283+0000 7f952f8cb640 1 -- 172.21.15.98:0/261256167 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9528148f30 con 0x7f95280cfbb0 2024-05-09T10:50:47.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.283+0000 7f952f8cb640 1 -- 172.21.15.98:0/261256167 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95281f77a0 con 0x7f95280cfbb0 2024-05-09T10:50:47.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.284+0000 7f952f8cb640 1 -- 172.21.15.98:0/261256167 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9528149450 con 0x7f95280cfbb0 2024-05-09T10:50:47.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.284+0000 7f951dffb640 1 -- 172.21.15.98:0/261256167 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9518011070 con 0x7f95280cfbb0 2024-05-09T10:50:47.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.284+0000 7f951dffb640 1 --2- 172.21.15.98:0/261256167 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f950c06b4d0 0x7f950c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:47.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.284+0000 7f951dffb640 1 -- 172.21.15.98:0/261256167 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9518089d50 con 0x7f95280cfbb0 2024-05-09T10:50:47.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.284+0000 7f952d667640 1 --2- 172.21.15.98:0/261256167 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f950c06b4d0 0x7f950c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:47.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.285+0000 7f952d667640 1 --2- 172.21.15.98:0/261256167 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f950c06b4d0 0x7f950c06d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9524034000 tx=0x7f952402c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:47.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.288+0000 7f951dffb640 1 -- 172.21.15.98:0/261256167 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9528149450 con 0x7f95280cfbb0 2024-05-09T10:50:47.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.461+0000 7f952f8cb640 1 -- 172.21.15.98:0/261256167 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f952804f000 con 0x7f95280cfbb0 2024-05-09T10:50:47.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.461+0000 7f951dffb640 1 -- 172.21.15.98:0/261256167 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f951808d030 con 0x7f95280cfbb0 2024-05-09T10:50:47.455 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-05-09T10:50:47.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.463+0000 7f952f8cb640 1 -- 172.21.15.98:0/261256167 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f950c06b4d0 msgr2=0x7f950c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:47.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.463+0000 7f952f8cb640 1 --2- 172.21.15.98:0/261256167 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f950c06b4d0 0x7f950c06d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9524034000 tx=0x7f952402c000 comp rx=0 tx=0).stop 2024-05-09T10:50:47.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.463+0000 7f952f8cb640 1 -- 172.21.15.98:0/261256167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95280cfbb0 msgr2=0x7f95280d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:47.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.463+0000 7f952f8cb640 1 --2- 172.21.15.98:0/261256167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95280cfbb0 0x7f95280d1fc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f951800c430 tx=0x7f951800c8f0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.463+0000 7f952f8cb640 1 -- 172.21.15.98:0/261256167 shutdown_connections 2024-05-09T10:50:47.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.463+0000 7f952f8cb640 1 --2- 172.21.15.98:0/261256167 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f950c06b4d0 0x7f950c06d990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.463+0000 7f952f8cb640 1 --2- 172.21.15.98:0/261256167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9528162240 0x7f9528148500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.463+0000 7f952f8cb640 1 --2- 172.21.15.98:0/261256167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95280cfbb0 0x7f95280d1fc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.463+0000 7f952f8cb640 1 --2- 172.21.15.98:0/261256167 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f95280b2d40 0x7f95281f7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.463+0000 7f952f8cb640 1 -- 172.21.15.98:0/261256167 >> 172.21.15.98:0/261256167 conn(0x7f95280d1970 msgr2=0x7f95280c7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:47.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.463+0000 7f952f8cb640 1 -- 172.21.15.98:0/261256167 shutdown_connections 2024-05-09T10:50:47.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.463+0000 7f952f8cb640 1 -- 172.21.15.98:0/261256167 wait complete. 2024-05-09T10:50:47.467 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T10:50:47.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.577+0000 7f83156b3640 1 Processor -- start 2024-05-09T10:50:47.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.577+0000 7f83156b3640 1 -- start start 2024-05-09T10:50:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.578+0000 7f83156b3640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f831015dd60 0x7f83100d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.578+0000 7f83156b3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f83100d24f0 0x7f83100cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.578+0000 7f83156b3640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83100cf8e0 con 0x7f831015e140 2024-05-09T10:50:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.578+0000 7f83156b3640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83100cf6b0 con 0x7f831015dd60 2024-05-09T10:50:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.578+0000 7f83156b3640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83100b2f50 con 0x7f83100d24f0 2024-05-09T10:50:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.578+0000 7f830effd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f831015dd60 0x7f83100d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.578+0000 7f830effd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f831015dd60 0x7f83100d1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34226/0 (socket says 172.21.15.98:34226) 2024-05-09T10:50:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.578+0000 7f830effd640 1 -- 172.21.15.98:0/2262760823 learned_addr learned my addr 172.21.15.98:0/2262760823 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.578+0000 7f830effd640 1 -- 172.21.15.98:0/2262760823 >> v2:172.21.15.105:3300/0 conn(0x7f83100d24f0 msgr2=0x7f83100cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.578+0000 7f830effd640 1 --2- 172.21.15.98:0/2262760823 >> v2:172.21.15.105:3300/0 conn(0x7f83100d24f0 0x7f83100cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.578+0000 7f830effd640 1 -- 172.21.15.98:0/2262760823 >> v1:172.21.15.105:6789/0 conn(0x7f831015e140 legacy=0x7f83100d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.578+0000 7f830effd640 1 -- 172.21.15.98:0/2262760823 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83100b31a0 con 0x7f831015dd60 2024-05-09T10:50:47.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.579+0000 7f830effd640 1 --2- 172.21.15.98:0/2262760823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f831015dd60 0x7f83100d1790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f83100b30d0 tx=0x7f830002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47372b89d13b41e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:47.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.579+0000 7f830dffb640 1 -- 172.21.15.98:0/2262760823 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8300036070 con 0x7f831015dd60 2024-05-09T10:50:47.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.579+0000 7f830dffb640 1 -- 172.21.15.98:0/2262760823 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8300031600 con 0x7f831015dd60 2024-05-09T10:50:47.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.579+0000 7f830dffb640 1 -- 172.21.15.98:0/2262760823 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83000319d0 con 0x7f831015dd60 2024-05-09T10:50:47.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.580+0000 7f83156b3640 1 -- 172.21.15.98:0/2262760823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f831015dd60 msgr2=0x7f83100d1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:47.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.580+0000 7f83156b3640 1 --2- 172.21.15.98:0/2262760823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f831015dd60 0x7f83100d1790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f83100b30d0 tx=0x7f830002a6d0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.580+0000 7f83156b3640 1 -- 172.21.15.98:0/2262760823 shutdown_connections 2024-05-09T10:50:47.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.580+0000 7f83156b3640 1 --2- 172.21.15.98:0/2262760823 >> v2:172.21.15.105:3300/0 conn(0x7f83100d24f0 0x7f83100cf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.580+0000 7f83156b3640 1 --2- 172.21.15.98:0/2262760823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f831015dd60 0x7f83100d1790 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.580+0000 7f83156b3640 1 -- 172.21.15.98:0/2262760823 >> 172.21.15.98:0/2262760823 conn(0x7f83100d3b50 msgr2=0x7f83100d3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:47.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.580+0000 7f83156b3640 1 -- 172.21.15.98:0/2262760823 shutdown_connections 2024-05-09T10:50:47.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.580+0000 7f83156b3640 1 -- 172.21.15.98:0/2262760823 wait complete. 2024-05-09T10:50:47.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.581+0000 7f83156b3640 1 Processor -- start 2024-05-09T10:50:47.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.581+0000 7f83156b3640 1 -- start start 2024-05-09T10:50:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.582+0000 7f83156b3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83100d24f0 0x7f8310150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.582+0000 7f83156b3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f831015dd60 0x7f8310160c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.582+0000 7f83156b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8310161150 0x7f83100b0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.582+0000 7f83156b3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83100b3c80 con 0x7f831015dd60 2024-05-09T10:50:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.582+0000 7f83156b3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83100b3760 con 0x7f8310161150 2024-05-09T10:50:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.582+0000 7f83156b3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83100b3e00 con 0x7f83100d24f0 2024-05-09T10:50:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.582+0000 7f830e7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f831015dd60 0x7f8310160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.582+0000 7f830e7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f831015dd60 0x7f8310160c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34240/0 (socket says 172.21.15.98:34240) 2024-05-09T10:50:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.582+0000 7f830e7fc640 1 -- 172.21.15.98:0/3106115164 learned_addr learned my addr 172.21.15.98:0/3106115164 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.582+0000 7f830f7fe640 1 --2- 172.21.15.98:0/3106115164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8310161150 0x7f83100b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.582+0000 7f830e7fc640 1 -- 172.21.15.98:0/3106115164 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83100d24f0 msgr2=0x7f8310150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.582+0000 7f830e7fc640 1 --2- 172.21.15.98:0/3106115164 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83100d24f0 0x7f8310150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.582+0000 7f830e7fc640 1 -- 172.21.15.98:0/3106115164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8310161150 msgr2=0x7f83100b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.582+0000 7f830e7fc640 1 --2- 172.21.15.98:0/3106115164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8310161150 0x7f83100b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.582+0000 7f830e7fc640 1 -- 172.21.15.98:0/3106115164 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8310151500 con 0x7f831015dd60 2024-05-09T10:50:47.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.582+0000 7f830e7fc640 1 --2- 172.21.15.98:0/3106115164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f831015dd60 0x7f8310160c10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f830400b3e0 tx=0x7f830400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:47.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.582+0000 7f82f77fe640 1 -- 172.21.15.98:0/3106115164 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8304015020 con 0x7f831015dd60 2024-05-09T10:50:47.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.582+0000 7f82f77fe640 1 -- 172.21.15.98:0/3106115164 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8304011850 con 0x7f831015dd60 2024-05-09T10:50:47.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.582+0000 7f83156b3640 1 -- 172.21.15.98:0/3106115164 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83100b8930 con 0x7f831015dd60 2024-05-09T10:50:47.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.582+0000 7f82f77fe640 1 -- 172.21.15.98:0/3106115164 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8304011c40 con 0x7f831015dd60 2024-05-09T10:50:47.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.582+0000 7f83156b3640 1 -- 172.21.15.98:0/3106115164 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83100b8ef0 con 0x7f831015dd60 2024-05-09T10:50:47.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.583+0000 7f83156b3640 1 -- 172.21.15.98:0/3106115164 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83100a1b60 con 0x7f831015dd60 2024-05-09T10:50:47.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.584+0000 7f82f77fe640 1 -- 172.21.15.98:0/3106115164 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8304018020 con 0x7f831015dd60 2024-05-09T10:50:47.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.584+0000 7f82f77fe640 1 --2- 172.21.15.98:0/3106115164 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f82fc06b4d0 0x7f82fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:47.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.584+0000 7f82f77fe640 1 -- 172.21.15.98:0/3106115164 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f830408a100 con 0x7f831015dd60 2024-05-09T10:50:47.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.584+0000 7f830effd640 1 --2- 172.21.15.98:0/3106115164 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f82fc06b4d0 0x7f82fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:47.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.585+0000 7f830effd640 1 --2- 172.21.15.98:0/3106115164 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f82fc06b4d0 0x7f82fc06d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8300033000 tx=0x7f830002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:47.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.588+0000 7f82f77fe640 1 -- 172.21.15.98:0/3106115164 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f830401f020 con 0x7f831015dd60 2024-05-09T10:50:47.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.758+0000 7f83156b3640 1 -- 172.21.15.98:0/3106115164 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f831004f000 con 0x7f831015dd60 2024-05-09T10:50:47.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.759+0000 7f82f77fe640 1 -- 172.21.15.98:0/3106115164 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8310151500 con 0x7f831015dd60 2024-05-09T10:50:47.752 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2024-05-09T10:50:47.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.761+0000 7f83156b3640 1 -- 172.21.15.98:0/3106115164 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f82fc06b4d0 msgr2=0x7f82fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:47.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.761+0000 7f83156b3640 1 --2- 172.21.15.98:0/3106115164 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f82fc06b4d0 0x7f82fc06d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8300033000 tx=0x7f830002e000 comp rx=0 tx=0).stop 2024-05-09T10:50:47.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.761+0000 7f83156b3640 1 -- 172.21.15.98:0/3106115164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f831015dd60 msgr2=0x7f8310160c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:47.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.761+0000 7f83156b3640 1 --2- 172.21.15.98:0/3106115164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f831015dd60 0x7f8310160c10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f830400b3e0 tx=0x7f830400b8a0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.761+0000 7f83156b3640 1 -- 172.21.15.98:0/3106115164 shutdown_connections 2024-05-09T10:50:47.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.761+0000 7f83156b3640 1 --2- 172.21.15.98:0/3106115164 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f82fc06b4d0 0x7f82fc06d990 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.761+0000 7f83156b3640 1 --2- 172.21.15.98:0/3106115164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8310161150 0x7f83100b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.761+0000 7f83156b3640 1 --2- 172.21.15.98:0/3106115164 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f831015dd60 0x7f8310160c10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.761+0000 7f83156b3640 1 --2- 172.21.15.98:0/3106115164 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83100d24f0 0x7f8310150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.761+0000 7f83156b3640 1 -- 172.21.15.98:0/3106115164 >> 172.21.15.98:0/3106115164 conn(0x7f83100d3b50 msgr2=0x7f83100d0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:47.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.761+0000 7f83156b3640 1 -- 172.21.15.98:0/3106115164 shutdown_connections 2024-05-09T10:50:47.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.761+0000 7f83156b3640 1 -- 172.21.15.98:0/3106115164 wait complete. 2024-05-09T10:50:47.765 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T10:50:47.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.875+0000 7fcffbae2640 1 Processor -- start 2024-05-09T10:50:47.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.875+0000 7fcffbae2640 1 -- start start 2024-05-09T10:50:47.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.876+0000 7fcffbae2640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcff415db90 0x7fcff40af720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:47.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.876+0000 7fcffbae2640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcff40720d0 0x7fcff406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:47.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.876+0000 7fcffbae2640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff4072690 con 0x7fcff406abe0 2024-05-09T10:50:47.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.876+0000 7fcffbae2640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcff40afc60 con 0x7fcff415db90 2024-05-09T10:50:47.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.876+0000 7fcffbae2640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff40b5d70 con 0x7fcff40720d0 2024-05-09T10:50:47.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.876+0000 7fcff987e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcff415db90 0x7fcff40af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:47.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.876+0000 7fcffa07f640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fcff406abe0 0x7fcff406aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41072/0 (socket says 172.21.15.98:41072) 2024-05-09T10:50:47.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.876+0000 7fcffa07f640 1 -- 172.21.15.98:0/1000355764 learned_addr learned my addr 172.21.15.98:0/1000355764 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:47.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.876+0000 7fcff907d640 1 --2- 172.21.15.98:0/1000355764 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcff40720d0 0x7fcff406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:47.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.876+0000 7fcff987e640 1 -- 172.21.15.98:0/1000355764 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcff40720d0 msgr2=0x7fcff406d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:47.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.876+0000 7fcff987e640 1 --2- 172.21.15.98:0/1000355764 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcff40720d0 0x7fcff406d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.876+0000 7fcff987e640 1 -- 172.21.15.98:0/1000355764 >> v1:172.21.15.98:6789/0 conn(0x7fcff406abe0 legacy=0x7fcff406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:47.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.876+0000 7fcff987e640 1 -- 172.21.15.98:0/1000355764 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff40b57b0 con 0x7fcff415db90 2024-05-09T10:50:47.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.876+0000 7fcff887c640 1 -- 172.21.15.98:0/1000355764 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 665394554 0 0) 0x7fcff4072690 con 0x7fcff406abe0 2024-05-09T10:50:47.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.876+0000 7fcff907d640 1 --2- 172.21.15.98:0/1000355764 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcff40720d0 0x7fcff406d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:47.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.877+0000 7fcff987e640 1 --2- 172.21.15.98:0/1000355764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff415db90 0x7fcff40af720 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcff00097a0 tx=0x7fcff002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7b486ef1b781cb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:47.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.877+0000 7fcff887c640 1 -- 172.21.15.98:0/1000355764 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0037070 con 0x7fcff415db90 2024-05-09T10:50:47.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.877+0000 7fcff887c640 1 -- 172.21.15.98:0/1000355764 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff002ae50 con 0x7fcff415db90 2024-05-09T10:50:47.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.877+0000 7fcff887c640 1 -- 172.21.15.98:0/1000355764 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0032660 con 0x7fcff415db90 2024-05-09T10:50:47.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.878+0000 7fcffbae2640 1 -- 172.21.15.98:0/1000355764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff415db90 msgr2=0x7fcff40af720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:47.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.878+0000 7fcffbae2640 1 --2- 172.21.15.98:0/1000355764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff415db90 0x7fcff40af720 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcff00097a0 tx=0x7fcff002a100 comp rx=0 tx=0).stop 2024-05-09T10:50:47.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.878+0000 7fcffbae2640 1 -- 172.21.15.98:0/1000355764 shutdown_connections 2024-05-09T10:50:47.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.878+0000 7fcffbae2640 1 --2- 172.21.15.98:0/1000355764 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcff40720d0 0x7fcff406d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.878+0000 7fcffbae2640 1 --2- 172.21.15.98:0/1000355764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff415db90 0x7fcff40af720 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.878+0000 7fcffbae2640 1 -- 172.21.15.98:0/1000355764 >> 172.21.15.98:0/1000355764 conn(0x7fcff4066fd0 msgr2=0x7fcff406bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:47.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.878+0000 7fcffbae2640 1 -- 172.21.15.98:0/1000355764 shutdown_connections 2024-05-09T10:50:47.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.878+0000 7fcffbae2640 1 -- 172.21.15.98:0/1000355764 wait complete. 2024-05-09T10:50:47.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.879+0000 7fcffbae2640 1 Processor -- start 2024-05-09T10:50:47.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.879+0000 7fcffbae2640 1 -- start start 2024-05-09T10:50:47.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.880+0000 7fcffbae2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff406abe0 0x7fcff4157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:47.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.880+0000 7fcffbae2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcff40720d0 0x7fcff40b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:47.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.880+0000 7fcffbae2640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcff415db90 0x7fcff40b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:47.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.880+0000 7fcffbae2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff406d930 con 0x7fcff406abe0 2024-05-09T10:50:47.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.880+0000 7fcffbae2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff40d0670 con 0x7fcff40720d0 2024-05-09T10:50:47.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.880+0000 7fcffbae2640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff415df70 con 0x7fcff415db90 2024-05-09T10:50:47.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.880+0000 7fcff987e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff406abe0 0x7fcff4157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:47.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.880+0000 7fcff987e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff406abe0 0x7fcff4157c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34252/0 (socket says 172.21.15.98:34252) 2024-05-09T10:50:47.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.880+0000 7fcff987e640 1 -- 172.21.15.98:0/1837532989 learned_addr learned my addr 172.21.15.98:0/1837532989 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:47.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.880+0000 7fcff907d640 1 --2- 172.21.15.98:0/1837532989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcff40720d0 0x7fcff40b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:47.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.880+0000 7fcff987e640 1 -- 172.21.15.98:0/1837532989 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcff415db90 msgr2=0x7fcff40b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:47.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.880+0000 7fcff987e640 1 --2- 172.21.15.98:0/1837532989 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcff415db90 0x7fcff40b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.880+0000 7fcff987e640 1 -- 172.21.15.98:0/1837532989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcff40720d0 msgr2=0x7fcff40b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:47.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.880+0000 7fcff987e640 1 --2- 172.21.15.98:0/1837532989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcff40720d0 0x7fcff40b2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:47.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.880+0000 7fcff987e640 1 -- 172.21.15.98:0/1837532989 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff4156160 con 0x7fcff406abe0 2024-05-09T10:50:47.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.880+0000 7fcff987e640 1 --2- 172.21.15.98:0/1837532989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff406abe0 0x7fcff4157c80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcff00097a0 tx=0x7fcff00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:47.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.880+0000 7fcfea7fc640 1 -- 172.21.15.98:0/1837532989 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0037040 con 0x7fcff406abe0 2024-05-09T10:50:47.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.880+0000 7fcfea7fc640 1 -- 172.21.15.98:0/1837532989 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff0004250 con 0x7fcff406abe0 2024-05-09T10:50:47.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.880+0000 7fcfea7fc640 1 -- 172.21.15.98:0/1837532989 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0003610 con 0x7fcff406abe0 2024-05-09T10:50:47.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.880+0000 7fcffbae2640 1 -- 172.21.15.98:0/1837532989 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff4156440 con 0x7fcff406abe0 2024-05-09T10:50:47.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.880+0000 7fcffbae2640 1 -- 172.21.15.98:0/1837532989 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff4156a00 con 0x7fcff406abe0 2024-05-09T10:50:47.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.881+0000 7fcffbae2640 1 -- 172.21.15.98:0/1837532989 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff4072dd0 con 0x7fcff406abe0 2024-05-09T10:50:47.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.882+0000 7fcfea7fc640 1 -- 172.21.15.98:0/1837532989 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcff00037b0 con 0x7fcff406abe0 2024-05-09T10:50:47.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.882+0000 7fcfea7fc640 1 --2- 172.21.15.98:0/1837532989 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fcfd806b4d0 0x7fcfd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:47.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.882+0000 7fcfea7fc640 1 -- 172.21.15.98:0/1837532989 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcff00ab160 con 0x7fcff406abe0 2024-05-09T10:50:47.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.882+0000 7fcff907d640 1 --2- 172.21.15.98:0/1837532989 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fcfd806b4d0 0x7fcfd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:47.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.883+0000 7fcff907d640 1 --2- 172.21.15.98:0/1837532989 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fcfd806b4d0 0x7fcfd806d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4008850 tx=0x7fcfe4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:47.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:47.886+0000 7fcfea7fc640 1 -- 172.21.15.98:0/1837532989 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcff007a190 con 0x7fcff406abe0 2024-05-09T10:50:48.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.057+0000 7fcffbae2640 1 -- 172.21.15.98:0/1837532989 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcff4156c00 con 0x7fcff406abe0 2024-05-09T10:50:48.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.057+0000 7fcfea7fc640 1 -- 172.21.15.98:0/1837532989 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcff0079a90 con 0x7fcff406abe0 2024-05-09T10:50:48.051 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2024-05-09T10:50:48.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.059+0000 7fcffbae2640 1 -- 172.21.15.98:0/1837532989 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fcfd806b4d0 msgr2=0x7fcfd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:48.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.059+0000 7fcffbae2640 1 --2- 172.21.15.98:0/1837532989 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fcfd806b4d0 0x7fcfd806d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4008850 tx=0x7fcfe4008040 comp rx=0 tx=0).stop 2024-05-09T10:50:48.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.059+0000 7fcffbae2640 1 -- 172.21.15.98:0/1837532989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff406abe0 msgr2=0x7fcff4157c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:48.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.059+0000 7fcffbae2640 1 --2- 172.21.15.98:0/1837532989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff406abe0 0x7fcff4157c80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcff00097a0 tx=0x7fcff00046c0 comp rx=0 tx=0).stop 2024-05-09T10:50:48.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.059+0000 7fcffbae2640 1 -- 172.21.15.98:0/1837532989 shutdown_connections 2024-05-09T10:50:48.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.059+0000 7fcffbae2640 1 --2- 172.21.15.98:0/1837532989 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fcfd806b4d0 0x7fcfd806d990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:48.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.060+0000 7fcffbae2640 1 --2- 172.21.15.98:0/1837532989 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcff415db90 0x7fcff40b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:48.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.060+0000 7fcffbae2640 1 --2- 172.21.15.98:0/1837532989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcff40720d0 0x7fcff40b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:48.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.060+0000 7fcffbae2640 1 --2- 172.21.15.98:0/1837532989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff406abe0 0x7fcff4157c80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:48.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.060+0000 7fcffbae2640 1 -- 172.21.15.98:0/1837532989 >> 172.21.15.98:0/1837532989 conn(0x7fcff4066fd0 msgr2=0x7fcff40ada90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:48.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.060+0000 7fcffbae2640 1 -- 172.21.15.98:0/1837532989 shutdown_connections 2024-05-09T10:50:48.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.060+0000 7fcffbae2640 1 -- 172.21.15.98:0/1837532989 wait complete. 2024-05-09T10:50:48.064 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T10:50:48.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.170+0000 7ff99a4fc640 1 Processor -- start 2024-05-09T10:50:48.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.170+0000 7ff99a4fc640 1 -- start start 2024-05-09T10:50:48.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.171+0000 7ff99a4fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff994143740 0x7ff9940720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:48.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.171+0000 7ff99a4fc640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff994143b20 con 0x7ff994143d70 2024-05-09T10:50:48.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.171+0000 7ff99a4fc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff99415f460 con 0x7ff9941443e0 2024-05-09T10:50:48.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.171+0000 7ff99a4fc640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9940b28c0 con 0x7ff994143740 2024-05-09T10:50:48.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.171+0000 7ff993fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff994143740 0x7ff9940720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:48.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.171+0000 7ff998a99640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff994143d70 0x7ff9940b30c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41074/0 (socket says 172.21.15.98:41074) 2024-05-09T10:50:48.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.171+0000 7ff998a99640 1 -- 172.21.15.98:0/2559787674 learned_addr learned my addr 172.21.15.98:0/2559787674 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:48.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.171+0000 7ff993fff640 1 -- 172.21.15.98:0/2559787674 >> v1:172.21.15.105:6789/0 conn(0x7ff9941443e0 legacy=0x7ff99415e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:48.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.171+0000 7ff993fff640 1 -- 172.21.15.98:0/2559787674 >> v1:172.21.15.98:6789/0 conn(0x7ff994143d70 legacy=0x7ff9940b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:48.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.171+0000 7ff993fff640 1 -- 172.21.15.98:0/2559787674 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9940b2ba0 con 0x7ff994143740 2024-05-09T10:50:48.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.172+0000 7ff993fff640 1 --2- 172.21.15.98:0/2559787674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff994143740 0x7ff9940720a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff9840097a0 tx=0x7ff98402a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411a77f85d0bdf10 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:48.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.172+0000 7ff992ffd640 1 -- 172.21.15.98:0/2559787674 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff984037070 con 0x7ff994143740 2024-05-09T10:50:48.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.172+0000 7ff992ffd640 1 -- 172.21.15.98:0/2559787674 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98402ad60 con 0x7ff994143740 2024-05-09T10:50:48.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.172+0000 7ff992ffd640 1 -- 172.21.15.98:0/2559787674 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff984032660 con 0x7ff994143740 2024-05-09T10:50:48.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.173+0000 7ff99a4fc640 1 -- 172.21.15.98:0/2559787674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff994143740 msgr2=0x7ff9940720a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:48.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.173+0000 7ff99a4fc640 1 --2- 172.21.15.98:0/2559787674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff994143740 0x7ff9940720a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff9840097a0 tx=0x7ff98402a0d0 comp rx=0 tx=0).stop 2024-05-09T10:50:48.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.173+0000 7ff99a4fc640 1 -- 172.21.15.98:0/2559787674 shutdown_connections 2024-05-09T10:50:48.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.173+0000 7ff99a4fc640 1 --2- 172.21.15.98:0/2559787674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff994143740 0x7ff9940720a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:48.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.173+0000 7ff99a4fc640 1 -- 172.21.15.98:0/2559787674 >> 172.21.15.98:0/2559787674 conn(0x7ff99406abe0 msgr2=0x7ff99406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:48.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.173+0000 7ff99a4fc640 1 -- 172.21.15.98:0/2559787674 shutdown_connections 2024-05-09T10:50:48.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.173+0000 7ff99a4fc640 1 -- 172.21.15.98:0/2559787674 wait complete. 2024-05-09T10:50:48.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.174+0000 7ff99a4fc640 1 Processor -- start 2024-05-09T10:50:48.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.174+0000 7ff99a4fc640 1 -- start start 2024-05-09T10:50:48.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.175+0000 7ff99a4fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff994143740 0x7ff99415a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:48.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.175+0000 7ff99a4fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff994143d70 0x7ff99415c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:48.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.175+0000 7ff99a4fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9941443e0 0x7ff9940cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:48.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.175+0000 7ff99a4fc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99415f1c0 con 0x7ff994143d70 2024-05-09T10:50:48.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.175+0000 7ff99a4fc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99415f040 con 0x7ff9941443e0 2024-05-09T10:50:48.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.175+0000 7ff99a4fc640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9940b37d0 con 0x7ff994143740 2024-05-09T10:50:48.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.175+0000 7ff9937fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff994143d70 0x7ff99415c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:48.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.175+0000 7ff9937fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff994143d70 0x7ff99415c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34280/0 (socket says 172.21.15.98:34280) 2024-05-09T10:50:48.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.175+0000 7ff9937fe640 1 -- 172.21.15.98:0/1877775650 learned_addr learned my addr 172.21.15.98:0/1877775650 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:48.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.175+0000 7ff993fff640 1 --2- 172.21.15.98:0/1877775650 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff994143740 0x7ff99415a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:48.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.175+0000 7ff998a99640 1 --2- 172.21.15.98:0/1877775650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9941443e0 0x7ff9940cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:48.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.175+0000 7ff9937fe640 1 -- 172.21.15.98:0/1877775650 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff994143740 msgr2=0x7ff99415a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:48.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.175+0000 7ff9937fe640 1 --2- 172.21.15.98:0/1877775650 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff994143740 0x7ff99415a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:48.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.175+0000 7ff9937fe640 1 -- 172.21.15.98:0/1877775650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9941443e0 msgr2=0x7ff9940cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:48.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.175+0000 7ff9937fe640 1 --2- 172.21.15.98:0/1877775650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9941443e0 0x7ff9940cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:48.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.175+0000 7ff9937fe640 1 -- 172.21.15.98:0/1877775650 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99415aa30 con 0x7ff994143d70 2024-05-09T10:50:48.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.175+0000 7ff993fff640 1 --2- 172.21.15.98:0/1877775650 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff994143740 0x7ff99415a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:48.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.175+0000 7ff998a99640 1 --2- 172.21.15.98:0/1877775650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9941443e0 0x7ff9940cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:48.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.175+0000 7ff9937fe640 1 --2- 172.21.15.98:0/1877775650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff994143d70 0x7ff99415c6a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff98800b3f0 tx=0x7ff98800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:48.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.176+0000 7ff990ff9640 1 -- 172.21.15.98:0/1877775650 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff988015020 con 0x7ff994143d70 2024-05-09T10:50:48.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.176+0000 7ff990ff9640 1 -- 172.21.15.98:0/1877775650 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff988011830 con 0x7ff994143d70 2024-05-09T10:50:48.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.176+0000 7ff990ff9640 1 -- 172.21.15.98:0/1877775650 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff988011c20 con 0x7ff994143d70 2024-05-09T10:50:48.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.176+0000 7ff99a4fc640 1 -- 172.21.15.98:0/1877775650 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff99415df90 con 0x7ff994143d70 2024-05-09T10:50:48.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.176+0000 7ff99a4fc640 1 -- 172.21.15.98:0/1877775650 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99415d1d0 con 0x7ff994143d70 2024-05-09T10:50:48.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.177+0000 7ff99a4fc640 1 -- 172.21.15.98:0/1877775650 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff994069cb0 con 0x7ff994143d70 2024-05-09T10:50:48.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.177+0000 7ff990ff9640 1 -- 172.21.15.98:0/1877775650 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff988018020 con 0x7ff994143d70 2024-05-09T10:50:48.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.177+0000 7ff990ff9640 1 --2- 172.21.15.98:0/1877775650 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff98006b4d0 0x7ff98006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:48.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.177+0000 7ff993fff640 1 --2- 172.21.15.98:0/1877775650 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff98006b4d0 0x7ff98006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:48.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.177+0000 7ff990ff9640 1 -- 172.21.15.98:0/1877775650 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff988089e00 con 0x7ff994143d70 2024-05-09T10:50:48.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.178+0000 7ff993fff640 1 --2- 172.21.15.98:0/1877775650 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff98006b4d0 0x7ff98006d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff984034000 tx=0x7ff98402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:48.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.181+0000 7ff990ff9640 1 -- 172.21.15.98:0/1877775650 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff98801f030 con 0x7ff994143d70 2024-05-09T10:50:48.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.363+0000 7ff99a4fc640 1 -- 172.21.15.98:0/1877775650 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff99415e450 con 0x7ff994143d70 2024-05-09T10:50:48.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.364+0000 7ff990ff9640 1 -- 172.21.15.98:0/1877775650 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff99415aa30 con 0x7ff994143d70 2024-05-09T10:50:48.357 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-05-09T10:50:48.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.366+0000 7ff99a4fc640 1 -- 172.21.15.98:0/1877775650 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff98006b4d0 msgr2=0x7ff98006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:48.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.366+0000 7ff99a4fc640 1 --2- 172.21.15.98:0/1877775650 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff98006b4d0 0x7ff98006d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff984034000 tx=0x7ff98402e000 comp rx=0 tx=0).stop 2024-05-09T10:50:48.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.366+0000 7ff99a4fc640 1 -- 172.21.15.98:0/1877775650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff994143d70 msgr2=0x7ff99415c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:48.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.366+0000 7ff99a4fc640 1 --2- 172.21.15.98:0/1877775650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff994143d70 0x7ff99415c6a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff98800b3f0 tx=0x7ff98800b8b0 comp rx=0 tx=0).stop 2024-05-09T10:50:48.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.366+0000 7ff99a4fc640 1 -- 172.21.15.98:0/1877775650 shutdown_connections 2024-05-09T10:50:48.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.366+0000 7ff99a4fc640 1 --2- 172.21.15.98:0/1877775650 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff98006b4d0 0x7ff98006d990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:48.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.366+0000 7ff99a4fc640 1 --2- 172.21.15.98:0/1877775650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9941443e0 0x7ff9940cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:48.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.366+0000 7ff99a4fc640 1 --2- 172.21.15.98:0/1877775650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff994143d70 0x7ff99415c6a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:48.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.366+0000 7ff99a4fc640 1 --2- 172.21.15.98:0/1877775650 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff994143740 0x7ff99415a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:48.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.366+0000 7ff99a4fc640 1 -- 172.21.15.98:0/1877775650 >> 172.21.15.98:0/1877775650 conn(0x7ff99406abe0 msgr2=0x7ff99406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:48.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.366+0000 7ff99a4fc640 1 -- 172.21.15.98:0/1877775650 shutdown_connections 2024-05-09T10:50:48.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.366+0000 7ff99a4fc640 1 -- 172.21.15.98:0/1877775650 wait complete. 2024-05-09T10:50:48.370 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-09T10:50:48.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:48.370 DEBUG:teuthology.orchestra.run.smithi098:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-05-09T10:50:48.396 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-05-09T10:50:48.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:48.396 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T10:50:48.474 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:50:48.474 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:50:48.495 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:50:48.495 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:50:48.554 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:50:48.554 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:50:48.612 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:48.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.717+0000 7f0a13370640 1 Processor -- start 2024-05-09T10:50:48.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.718+0000 7f0a13370640 1 -- start start 2024-05-09T10:50:48.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.718+0000 7f0a13370640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a0c0d0b80 0x7f0a0c0b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:48.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.718+0000 7f0a13370640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0c0d2c30 con 0x7f0a0c0d27a0 2024-05-09T10:50:48.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.718+0000 7f0a13370640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0c15e220 con 0x7f0a0c15de40 2024-05-09T10:50:48.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.718+0000 7f0a13370640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0d1790 con 0x7f0a0c0d0b80 2024-05-09T10:50:48.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.719+0000 7f0a1090b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f0a0c0d27a0 0x7f0a0c0d0470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41086/0 (socket says 172.21.15.98:41086) 2024-05-09T10:50:48.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.719+0000 7f0a1090b640 1 -- 172.21.15.98:0/2204971467 learned_addr learned my addr 172.21.15.98:0/2204971467 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:48.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.719+0000 7f0a1190d640 1 --2- 172.21.15.98:0/2204971467 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a0c0d0b80 0x7f0a0c0b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:48.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.719+0000 7f0a0bfff640 1 -- 172.21.15.98:0/2204971467 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725610002 0 0) 0x7f0a0c0d2c30 con 0x7f0a0c0d27a0 2024-05-09T10:50:48.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.719+0000 7f0a0bfff640 1 -- 172.21.15.98:0/2204971467 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09f4003520 con 0x7f0a0c0d27a0 2024-05-09T10:50:48.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.719+0000 7f0a0bfff640 1 -- 172.21.15.98:0/2204971467 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2722601266 0 0) 0x7f0a0c15e220 con 0x7f0a0c15de40 2024-05-09T10:50:48.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.719+0000 7f0a0bfff640 1 -- 172.21.15.98:0/2204971467 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a0c0d2c30 con 0x7f0a0c15de40 2024-05-09T10:50:48.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.719+0000 7f0a0bfff640 1 -- 172.21.15.98:0/2204971467 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1395252456 0 0) 0x7f09f4003520 con 0x7f0a0c0d27a0 2024-05-09T10:50:48.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.719+0000 7f0a0bfff640 1 -- 172.21.15.98:0/2204971467 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a0c0d0b80 msgr2=0x7f0a0c0b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:48.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.719+0000 7f0a0bfff640 1 --2- 172.21.15.98:0/2204971467 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a0c0d0b80 0x7f0a0c0b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:48.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.719+0000 7f0a0bfff640 1 -- 172.21.15.98:0/2204971467 >> v1:172.21.15.105:6789/0 conn(0x7f0a0c15de40 legacy=0x7f0a0c0cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:48.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.719+0000 7f0a0bfff640 1 -- 172.21.15.98:0/2204971467 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c0d19e0 con 0x7f0a0c0d27a0 2024-05-09T10:50:48.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.719+0000 7f0a0bfff640 1 -- 172.21.15.98:0/2204971467 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f0a00001670 con 0x7f0a0c0d27a0 2024-05-09T10:50:48.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.719+0000 7f0a0bfff640 1 -- 172.21.15.98:0/2204971467 >> v1:172.21.15.98:6789/0 conn(0x7f0a0c0d27a0 legacy=0x7f0a0c0d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:48.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.719+0000 7f0a0bfff640 1 --2- 172.21.15.98:0/2204971467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09f4003b70 0x7f09f4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:48.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.719+0000 7f0a0bfff640 1 --2- 172.21.15.98:0/2204971467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09f4004800 0x7f09f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:48.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.719+0000 7f0a0bfff640 1 --2- 172.21.15.98:0/2204971467 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a0c0d0b80 0x7f09f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:48.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.719+0000 7f0a0bfff640 1 -- 172.21.15.98:0/2204971467 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0d1790 con 0x7f09f4004800 2024-05-09T10:50:48.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.719+0000 7f0a0bfff640 1 -- 172.21.15.98:0/2204971467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f400baa0 con 0x7f09f4003b70 2024-05-09T10:50:48.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.719+0000 7f0a0bfff640 1 -- 172.21.15.98:0/2204971467 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f400bca0 con 0x7f0a0c0d0b80 2024-05-09T10:50:48.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.719+0000 7f0a1090b640 1 --2- 172.21.15.98:0/2204971467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09f4004800 0x7f09f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:48.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.719+0000 7f0a1110c640 1 --2- 172.21.15.98:0/2204971467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09f4003b70 0x7f09f4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:48.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.719+0000 7f0a1190d640 1 --2- 172.21.15.98:0/2204971467 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a0c0d0b80 0x7f09f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:48.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.720+0000 7f0a1090b640 1 -- 172.21.15.98:0/2204971467 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a0c0d0b80 msgr2=0x7f09f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:48.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.720+0000 7f0a1090b640 1 --2- 172.21.15.98:0/2204971467 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a0c0d0b80 0x7f09f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:48.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.720+0000 7f0a1090b640 1 -- 172.21.15.98:0/2204971467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09f4003b70 msgr2=0x7f09f4003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:48.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.720+0000 7f0a1090b640 1 --2- 172.21.15.98:0/2204971467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09f4003b70 0x7f09f4003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:48.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.720+0000 7f0a1090b640 1 -- 172.21.15.98:0/2204971467 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f4003520 con 0x7f09f4004800 2024-05-09T10:50:48.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.720+0000 7f0a1110c640 1 --2- 172.21.15.98:0/2204971467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09f4003b70 0x7f09f4003f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:48.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.720+0000 7f0a1090b640 1 --2- 172.21.15.98:0/2204971467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09f4004800 0x7f09f4004c80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000c0e0 tx=0x7f09f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f86ba9c2e707924 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:48.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.720+0000 7f0a0bfff640 1 -- 172.21.15.98:0/2204971467 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a00007c40 con 0x7f09f4004800 2024-05-09T10:50:48.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.720+0000 7f0a0bfff640 1 -- 172.21.15.98:0/2204971467 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a00005ad0 con 0x7f09f4004800 2024-05-09T10:50:48.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.720+0000 7f0a0bfff640 1 -- 172.21.15.98:0/2204971467 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a00007070 con 0x7f09f4004800 2024-05-09T10:50:48.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.721+0000 7f0a13370640 1 -- 172.21.15.98:0/2204971467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09f4004800 msgr2=0x7f09f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:48.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.721+0000 7f0a13370640 1 --2- 172.21.15.98:0/2204971467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09f4004800 0x7f09f4004c80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000c0e0 tx=0x7f09f4005830 comp rx=0 tx=0).stop 2024-05-09T10:50:48.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.721+0000 7f0a1190d640 1 -- 172.21.15.98:0/2204971467 reap_dead start 2024-05-09T10:50:48.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.721+0000 7f0a13370640 1 -- 172.21.15.98:0/2204971467 shutdown_connections 2024-05-09T10:50:48.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.721+0000 7f0a13370640 1 -- 172.21.15.98:0/2204971467 >> 172.21.15.98:0/2204971467 conn(0x7f0a0c09ce90 msgr2=0x7f0a0c0d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:48.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.721+0000 7f0a13370640 1 -- 172.21.15.98:0/2204971467 shutdown_connections 2024-05-09T10:50:48.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.721+0000 7f0a13370640 1 -- 172.21.15.98:0/2204971467 wait complete. 2024-05-09T10:50:48.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.722+0000 7f0a13370640 1 Processor -- start 2024-05-09T10:50:48.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.722+0000 7f0a13370640 1 -- start start 2024-05-09T10:50:48.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.723+0000 7f0a13370640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f09f4004800 0x7f0a0c1f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:48.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.723+0000 7f0a13370640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a0c0c7ea0 0x7f0a0c154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:48.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.723+0000 7f0a13370640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a0c154e80 0x7f0a0c0cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:48.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.723+0000 7f0a13370640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0d2280 con 0x7f0a0c0c7ea0 2024-05-09T10:50:48.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.723+0000 7f0a13370640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0d1fa0 con 0x7f0a0c154e80 2024-05-09T10:50:48.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.723+0000 7f0a13370640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0cf960 con 0x7f09f4004800 2024-05-09T10:50:48.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.723+0000 7f0a1090b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a0c0c7ea0 0x7f0a0c154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:48.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.723+0000 7f0a1090b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a0c0c7ea0 0x7f0a0c154940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34296/0 (socket says 172.21.15.98:34296) 2024-05-09T10:50:48.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.723+0000 7f0a1090b640 1 -- 172.21.15.98:0/2619410665 learned_addr learned my addr 172.21.15.98:0/2619410665 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:48.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.723+0000 7f0a1190d640 1 --2- 172.21.15.98:0/2619410665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a0c154e80 0x7f0a0c0cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:48.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.723+0000 7f0a1110c640 1 --2- 172.21.15.98:0/2619410665 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f09f4004800 0x7f0a0c1f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:48.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.723+0000 7f0a1090b640 1 -- 172.21.15.98:0/2619410665 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f09f4004800 msgr2=0x7f0a0c1f6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:48.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.723+0000 7f0a1090b640 1 --2- 172.21.15.98:0/2619410665 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f09f4004800 0x7f0a0c1f6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:48.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.723+0000 7f0a1090b640 1 -- 172.21.15.98:0/2619410665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a0c154e80 msgr2=0x7f0a0c0cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:48.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.723+0000 7f0a1090b640 1 --2- 172.21.15.98:0/2619410665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a0c154e80 0x7f0a0c0cbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:48.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.723+0000 7f0a1090b640 1 -- 172.21.15.98:0/2619410665 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c0ccfd0 con 0x7f0a0c0c7ea0 2024-05-09T10:50:48.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.723+0000 7f0a1090b640 1 --2- 172.21.15.98:0/2619410665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a0c0c7ea0 0x7f0a0c154940 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000c0e0 tx=0x7f0a00001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:48.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.724+0000 7f0a09ffb640 1 -- 172.21.15.98:0/2619410665 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a00041020 con 0x7f0a0c0c7ea0 2024-05-09T10:50:48.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.724+0000 7f0a09ffb640 1 -- 172.21.15.98:0/2619410665 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a00001470 con 0x7f0a0c0c7ea0 2024-05-09T10:50:48.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.724+0000 7f0a13370640 1 -- 172.21.15.98:0/2619410665 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0c0cd2b0 con 0x7f0a0c0c7ea0 2024-05-09T10:50:48.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.724+0000 7f0a09ffb640 1 -- 172.21.15.98:0/2619410665 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a00007070 con 0x7f0a0c0c7ea0 2024-05-09T10:50:48.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.724+0000 7f0a13370640 1 -- 172.21.15.98:0/2619410665 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0c155610 con 0x7f0a0c0c7ea0 2024-05-09T10:50:48.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.725+0000 7f0a13370640 1 -- 172.21.15.98:0/2619410665 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a0c069cb0 con 0x7f0a0c0c7ea0 2024-05-09T10:50:48.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.725+0000 7f0a09ffb640 1 -- 172.21.15.98:0/2619410665 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a00044020 con 0x7f0a0c0c7ea0 2024-05-09T10:50:48.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.726+0000 7f0a09ffb640 1 --2- 172.21.15.98:0/2619410665 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f09f806b4d0 0x7f09f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:48.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.726+0000 7f0a09ffb640 1 -- 172.21.15.98:0/2619410665 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a000aa6b0 con 0x7f0a0c0c7ea0 2024-05-09T10:50:48.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.726+0000 7f0a1110c640 1 --2- 172.21.15.98:0/2619410665 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f09f806b4d0 0x7f09f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:48.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.726+0000 7f0a1110c640 1 --2- 172.21.15.98:0/2619410665 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f09f806b4d0 0x7f09f806d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0a04007fd0 tx=0x7f0a04009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:48.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.730+0000 7f0a09ffb640 1 -- 172.21.15.98:0/2619410665 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0a00005aa0 con 0x7f0a0c0c7ea0 2024-05-09T10:50:48.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.898+0000 7f0a13370640 1 -- 172.21.15.98:0/2619410665 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a0c0d42d0 con 0x7f0a0c0c7ea0 2024-05-09T10:50:48.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.899+0000 7f0a09ffb640 1 -- 172.21.15.98:0/2619410665 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f0a00079530 con 0x7f0a0c0c7ea0 2024-05-09T10:50:48.893 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:48.894 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":20,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:21.176419+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:50:17.323444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T10:50:18.482315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:48.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.901+0000 7f0a13370640 1 -- 172.21.15.98:0/2619410665 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f09f806b4d0 msgr2=0x7f09f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:48.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.901+0000 7f0a13370640 1 --2- 172.21.15.98:0/2619410665 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f09f806b4d0 0x7f09f806d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0a04007fd0 tx=0x7f0a04009040 comp rx=0 tx=0).stop 2024-05-09T10:50:48.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.901+0000 7f0a13370640 1 -- 172.21.15.98:0/2619410665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a0c0c7ea0 msgr2=0x7f0a0c154940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:48.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.901+0000 7f0a13370640 1 --2- 172.21.15.98:0/2619410665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a0c0c7ea0 0x7f0a0c154940 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000c0e0 tx=0x7f0a00001970 comp rx=0 tx=0).stop 2024-05-09T10:50:48.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.901+0000 7f0a13370640 1 -- 172.21.15.98:0/2619410665 shutdown_connections 2024-05-09T10:50:48.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.901+0000 7f0a13370640 1 --2- 172.21.15.98:0/2619410665 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f09f806b4d0 0x7f09f806d990 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:48.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.901+0000 7f0a13370640 1 --2- 172.21.15.98:0/2619410665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a0c154e80 0x7f0a0c0cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:48.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.901+0000 7f0a13370640 1 --2- 172.21.15.98:0/2619410665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a0c0c7ea0 0x7f0a0c154940 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:48.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.901+0000 7f0a13370640 1 --2- 172.21.15.98:0/2619410665 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f09f4004800 0x7f0a0c1f6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:48.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.901+0000 7f0a13370640 1 -- 172.21.15.98:0/2619410665 >> 172.21.15.98:0/2619410665 conn(0x7f0a0c09ce90 msgr2=0x7f0a0c15d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:48.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.901+0000 7f0a13370640 1 -- 172.21.15.98:0/2619410665 shutdown_connections 2024-05-09T10:50:48.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:48.901+0000 7f0a13370640 1 -- 172.21.15.98:0/2619410665 wait complete. 2024-05-09T10:50:48.907 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T10:50:07.396423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T10:50:17.323444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T10:50:18.482315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T10:50:48.907 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T10:50:49.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.015+0000 7f30e2c83640 1 Processor -- start 2024-05-09T10:50:49.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.016+0000 7f30e2c83640 1 -- start start 2024-05-09T10:50:49.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.016+0000 7f30e2c83640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f30dc0d1be0 0x7f30dc0d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.016+0000 7f30e2c83640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30dc0cf950 0x7f30dc0ae830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.016+0000 7f30e2c83640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30dc0d1710 con 0x7f30dc0d1fc0 2024-05-09T10:50:49.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.016+0000 7f30e2c83640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30dc0b2bb0 con 0x7f30dc0d1be0 2024-05-09T10:50:49.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.016+0000 7f30e2c83640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc0b2e90 con 0x7f30dc0cf950 2024-05-09T10:50:49.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.016+0000 7f30e0a1f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f30dc0d1be0 0x7f30dc0d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.017+0000 7f30e1220640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f30dc0d1fc0 0x7f30dc0aed70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41102/0 (socket says 172.21.15.98:41102) 2024-05-09T10:50:49.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.017+0000 7f30e1220640 1 -- 172.21.15.98:0/605416845 learned_addr learned my addr 172.21.15.98:0/605416845 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:49.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.017+0000 7f30d3fff640 1 --2- 172.21.15.98:0/605416845 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30dc0cf950 0x7f30dc0ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.017+0000 7f30e0a1f640 1 -- 172.21.15.98:0/605416845 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30dc0cf950 msgr2=0x7f30dc0ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:49.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.017+0000 7f30e0a1f640 1 --2- 172.21.15.98:0/605416845 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30dc0cf950 0x7f30dc0ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.017+0000 7f30e0a1f640 1 -- 172.21.15.98:0/605416845 >> v1:172.21.15.98:6789/0 conn(0x7f30dc0d1fc0 legacy=0x7f30dc0aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:49.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.017+0000 7f30e0a1f640 1 -- 172.21.15.98:0/605416845 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30dc0b3170 con 0x7f30dc0d1be0 2024-05-09T10:50:49.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.017+0000 7f30d37fe640 1 -- 172.21.15.98:0/605416845 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1980316157 0 0) 0x7f30d4001110 con 0x7f30dc0d1fc0 2024-05-09T10:50:49.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.017+0000 7f30d3fff640 1 --2- 172.21.15.98:0/605416845 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30dc0cf950 0x7f30dc0ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:49.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.018+0000 7f30e0a1f640 1 --2- 172.21.15.98:0/605416845 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30dc0d1be0 0x7f30dc0d11d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f30d8009d90 tx=0x7f30d802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9174ed8de15eb5cb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:49.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.018+0000 7f30d37fe640 1 -- 172.21.15.98:0/605416845 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d8007740 con 0x7f30dc0d1be0 2024-05-09T10:50:49.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.018+0000 7f30d37fe640 1 -- 172.21.15.98:0/605416845 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d80078e0 con 0x7f30dc0d1be0 2024-05-09T10:50:49.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.018+0000 7f30d37fe640 1 -- 172.21.15.98:0/605416845 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d8007cb0 con 0x7f30dc0d1be0 2024-05-09T10:50:49.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.018+0000 7f30e2c83640 1 -- 172.21.15.98:0/605416845 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30dc0d1be0 msgr2=0x7f30dc0d11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:49.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.018+0000 7f30e2c83640 1 --2- 172.21.15.98:0/605416845 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30dc0d1be0 0x7f30dc0d11d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f30d8009d90 tx=0x7f30d802a660 comp rx=0 tx=0).stop 2024-05-09T10:50:49.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.018+0000 7f30e2c83640 1 -- 172.21.15.98:0/605416845 shutdown_connections 2024-05-09T10:50:49.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.018+0000 7f30e2c83640 1 --2- 172.21.15.98:0/605416845 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30dc0cf950 0x7f30dc0ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.018+0000 7f30e2c83640 1 --2- 172.21.15.98:0/605416845 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30dc0d1be0 0x7f30dc0d11d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.018+0000 7f30e2c83640 1 -- 172.21.15.98:0/605416845 >> 172.21.15.98:0/605416845 conn(0x7f30dc0b6db0 msgr2=0x7f30dc0b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:49.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.018+0000 7f30e2c83640 1 -- 172.21.15.98:0/605416845 shutdown_connections 2024-05-09T10:50:49.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.018+0000 7f30e2c83640 1 -- 172.21.15.98:0/605416845 wait complete. 2024-05-09T10:50:49.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.019+0000 7f30e2c83640 1 Processor -- start 2024-05-09T10:50:49.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.019+0000 7f30e2c83640 1 -- start start 2024-05-09T10:50:49.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.020+0000 7f30e2c83640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30dc0cf950 0x7f30dc142500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.020+0000 7f30e2c83640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30dc0d1be0 0x7f30dc1556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.020+0000 7f30e2c83640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30dc0d1fc0 0x7f30dc155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.020+0000 7f30e2c83640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc0b26b0 con 0x7f30dc0d1fc0 2024-05-09T10:50:49.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.020+0000 7f30e2c83640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc0b2190 con 0x7f30dc0cf950 2024-05-09T10:50:49.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.020+0000 7f30e2c83640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc15d5c0 con 0x7f30dc0d1be0 2024-05-09T10:50:49.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.020+0000 7f30e1220640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30dc0d1fc0 0x7f30dc155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.020+0000 7f30e1220640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30dc0d1fc0 0x7f30dc155c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34310/0 (socket says 172.21.15.98:34310) 2024-05-09T10:50:49.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.020+0000 7f30e1220640 1 -- 172.21.15.98:0/3649451939 learned_addr learned my addr 172.21.15.98:0/3649451939 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:49.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.021+0000 7f30e0a1f640 1 --2- 172.21.15.98:0/3649451939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30dc0cf950 0x7f30dc142500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.021+0000 7f30d3fff640 1 --2- 172.21.15.98:0/3649451939 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30dc0d1be0 0x7f30dc1556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.021+0000 7f30e1220640 1 -- 172.21.15.98:0/3649451939 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30dc0d1be0 msgr2=0x7f30dc1556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:49.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.021+0000 7f30e1220640 1 --2- 172.21.15.98:0/3649451939 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30dc0d1be0 0x7f30dc1556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.021+0000 7f30e1220640 1 -- 172.21.15.98:0/3649451939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30dc0cf950 msgr2=0x7f30dc142500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:49.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.021+0000 7f30e1220640 1 --2- 172.21.15.98:0/3649451939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30dc0cf950 0x7f30dc142500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.021+0000 7f30e1220640 1 -- 172.21.15.98:0/3649451939 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30dc156230 con 0x7f30dc0d1fc0 2024-05-09T10:50:49.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.021+0000 7f30e1220640 1 --2- 172.21.15.98:0/3649451939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30dc0d1fc0 0x7f30dc155c20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f30d400c780 tx=0x7f30d400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:49.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.021+0000 7f30d17fa640 1 -- 172.21.15.98:0/3649451939 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d4018070 con 0x7f30dc0d1fc0 2024-05-09T10:50:49.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.021+0000 7f30d17fa640 1 -- 172.21.15.98:0/3649451939 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d40070d0 con 0x7f30dc0d1fc0 2024-05-09T10:50:49.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.021+0000 7f30e2c83640 1 -- 172.21.15.98:0/3649451939 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30dc15cc90 con 0x7f30dc0d1fc0 2024-05-09T10:50:49.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.021+0000 7f30d17fa640 1 -- 172.21.15.98:0/3649451939 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d4007500 con 0x7f30dc0d1fc0 2024-05-09T10:50:49.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.021+0000 7f30e2c83640 1 -- 172.21.15.98:0/3649451939 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30dc15d250 con 0x7f30dc0d1fc0 2024-05-09T10:50:49.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.022+0000 7f30e2c83640 1 -- 172.21.15.98:0/3649451939 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30dc142b60 con 0x7f30dc0d1fc0 2024-05-09T10:50:49.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.023+0000 7f30d17fa640 1 -- 172.21.15.98:0/3649451939 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f30d401b070 con 0x7f30dc0d1fc0 2024-05-09T10:50:49.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.023+0000 7f30d17fa640 1 --2- 172.21.15.98:0/3649451939 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f30c806b500 0x7f30c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.023+0000 7f30d17fa640 1 -- 172.21.15.98:0/3649451939 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30d4089390 con 0x7f30dc0d1fc0 2024-05-09T10:50:49.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.023+0000 7f30e0a1f640 1 --2- 172.21.15.98:0/3649451939 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f30c806b500 0x7f30c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.024+0000 7f30e0a1f640 1 --2- 172.21.15.98:0/3649451939 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f30c806b500 0x7f30c806d9c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f30d8034000 tx=0x7f30d802f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:49.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.027+0000 7f30d17fa640 1 -- 172.21.15.98:0/3649451939 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f30dc0d1710 con 0x7f30dc0d1fc0 2024-05-09T10:50:49.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.196+0000 7f30e2c83640 1 -- 172.21.15.98:0/3649451939 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f30dc09fbb0 con 0x7f30dc0d1fc0 2024-05-09T10:50:49.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.196+0000 7f30d17fa640 1 -- 172.21.15.98:0/3649451939 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f30d4057bc0 con 0x7f30dc0d1fc0 2024-05-09T10:50:49.191 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-05-09T10:50:49.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.198+0000 7f30e2c83640 1 -- 172.21.15.98:0/3649451939 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f30c806b500 msgr2=0x7f30c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:49.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.198+0000 7f30e2c83640 1 --2- 172.21.15.98:0/3649451939 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f30c806b500 0x7f30c806d9c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f30d8034000 tx=0x7f30d802f000 comp rx=0 tx=0).stop 2024-05-09T10:50:49.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.198+0000 7f30e2c83640 1 -- 172.21.15.98:0/3649451939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30dc0d1fc0 msgr2=0x7f30dc155c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:49.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.198+0000 7f30e2c83640 1 --2- 172.21.15.98:0/3649451939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30dc0d1fc0 0x7f30dc155c20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f30d400c780 tx=0x7f30d400cc40 comp rx=0 tx=0).stop 2024-05-09T10:50:49.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.198+0000 7f30e2c83640 1 -- 172.21.15.98:0/3649451939 shutdown_connections 2024-05-09T10:50:49.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.198+0000 7f30e2c83640 1 --2- 172.21.15.98:0/3649451939 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f30c806b500 0x7f30c806d9c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.198+0000 7f30e2c83640 1 --2- 172.21.15.98:0/3649451939 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30dc0d1fc0 0x7f30dc155c20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.198+0000 7f30e2c83640 1 --2- 172.21.15.98:0/3649451939 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30dc0d1be0 0x7f30dc1556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.198+0000 7f30e2c83640 1 --2- 172.21.15.98:0/3649451939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30dc0cf950 0x7f30dc142500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.198+0000 7f30e2c83640 1 -- 172.21.15.98:0/3649451939 >> 172.21.15.98:0/3649451939 conn(0x7f30dc0b6db0 msgr2=0x7f30dc0b0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:49.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.198+0000 7f30e2c83640 1 -- 172.21.15.98:0/3649451939 shutdown_connections 2024-05-09T10:50:49.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.198+0000 7f30e2c83640 1 -- 172.21.15.98:0/3649451939 wait complete. 2024-05-09T10:50:49.203 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T10:50:49.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.312+0000 7f4021af5640 1 Processor -- start 2024-05-09T10:50:49.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.312+0000 7f4021af5640 1 -- start start 2024-05-09T10:50:49.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.313+0000 7f4021af5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f401c15f140 0x7f401c0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.313+0000 7f4021af5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f401c0d03f0 0x7f401c0d0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.313+0000 7f4021af5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f401c06ea70 con 0x7f401c06e4b0 2024-05-09T10:50:49.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.313+0000 7f4021af5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f401c15f520 con 0x7f401c15f140 2024-05-09T10:50:49.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.313+0000 7f4021af5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401c06ed50 con 0x7f401c0d03f0 2024-05-09T10:50:49.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.313+0000 7f4012ffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f401c06e4b0 0x7f401c15fde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41116/0 (socket says 172.21.15.98:41116) 2024-05-09T10:50:49.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.313+0000 7f4012ffd640 1 -- 172.21.15.98:0/2080327601 learned_addr learned my addr 172.21.15.98:0/2080327601 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:49.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.313+0000 7f401bfff640 1 --2- 172.21.15.98:0/2080327601 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f401c0d03f0 0x7f401c0d0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.313+0000 7f401b7fe640 1 --2- 172.21.15.98:0/2080327601 >> v2:172.21.15.105:3300/0 conn(0x7f401c15f140 0x7f401c0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.313+0000 7f401bfff640 1 -- 172.21.15.98:0/2080327601 >> v2:172.21.15.105:3300/0 conn(0x7f401c15f140 msgr2=0x7f401c0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:49.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.313+0000 7f401bfff640 1 --2- 172.21.15.98:0/2080327601 >> v2:172.21.15.105:3300/0 conn(0x7f401c15f140 0x7f401c0af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.313+0000 7f401bfff640 1 -- 172.21.15.98:0/2080327601 >> v1:172.21.15.98:6789/0 conn(0x7f401c06e4b0 legacy=0x7f401c15fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:49.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.313+0000 7f401bfff640 1 -- 172.21.15.98:0/2080327601 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c06f030 con 0x7f401c0d03f0 2024-05-09T10:50:49.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.314+0000 7f401bfff640 1 --2- 172.21.15.98:0/2080327601 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f401c0d03f0 0x7f401c0d0840 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4014009d90 tx=0x7f401402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=708d1140b89cb9d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:49.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.317+0000 7f401affd640 1 -- 172.21.15.98:0/2080327601 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4014031600 con 0x7f401c0d03f0 2024-05-09T10:50:49.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.317+0000 7f401affd640 1 -- 172.21.15.98:0/2080327601 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40140317a0 con 0x7f401c0d03f0 2024-05-09T10:50:49.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.317+0000 7f401affd640 1 -- 172.21.15.98:0/2080327601 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4014031b70 con 0x7f401c0d03f0 2024-05-09T10:50:49.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.317+0000 7f4021af5640 1 -- 172.21.15.98:0/2080327601 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f401c0d03f0 msgr2=0x7f401c0d0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:49.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.317+0000 7f4021af5640 1 --2- 172.21.15.98:0/2080327601 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f401c0d03f0 0x7f401c0d0840 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4014009d90 tx=0x7f401402a5d0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.317+0000 7f4021af5640 1 -- 172.21.15.98:0/2080327601 shutdown_connections 2024-05-09T10:50:49.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.317+0000 7f4021af5640 1 --2- 172.21.15.98:0/2080327601 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f401c0d03f0 0x7f401c0d0840 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.317+0000 7f4021af5640 1 --2- 172.21.15.98:0/2080327601 >> v2:172.21.15.105:3300/0 conn(0x7f401c15f140 0x7f401c0af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.317+0000 7f4021af5640 1 -- 172.21.15.98:0/2080327601 >> 172.21.15.98:0/2080327601 conn(0x7f401c066fd0 msgr2=0x7f401c06bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:49.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.317+0000 7f4021af5640 1 -- 172.21.15.98:0/2080327601 shutdown_connections 2024-05-09T10:50:49.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.317+0000 7f4021af5640 1 -- 172.21.15.98:0/2080327601 wait complete. 2024-05-09T10:50:49.313 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T10:50:49.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.318+0000 7f4021af5640 1 Processor -- start 2024-05-09T10:50:49.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.319+0000 7f4021af5640 1 -- start start 2024-05-09T10:50:49.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.319+0000 7f4021af5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f401c06e4b0 0x7f401c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.319+0000 7f4021af5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f401c0d03f0 0x7f401c1f4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.319+0000 7f4021af5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f401c15f140 0x7f401c14a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.319+0000 7f4021af5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c160670 con 0x7f401c15f140 2024-05-09T10:50:49.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.319+0000 7f4021af5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c1604f0 con 0x7f401c0d03f0 2024-05-09T10:50:49.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.319+0000 7f4021af5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401c06abe0 con 0x7f401c06e4b0 2024-05-09T10:50:49.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.320+0000 7f401bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f401c15f140 0x7f401c14a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.320+0000 7f401bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f401c15f140 0x7f401c14a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34320/0 (socket says 172.21.15.98:34320) 2024-05-09T10:50:49.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.320+0000 7f401bfff640 1 -- 172.21.15.98:0/3535511897 learned_addr learned my addr 172.21.15.98:0/3535511897 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:49.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.320+0000 7f401b7fe640 1 --2- 172.21.15.98:0/3535511897 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f401c06e4b0 0x7f401c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.320+0000 7f4012ffd640 1 --2- 172.21.15.98:0/3535511897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f401c0d03f0 0x7f401c1f4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.316 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T10:50:49.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.320+0000 7f401bfff640 1 -- 172.21.15.98:0/3535511897 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f401c06e4b0 msgr2=0x7f401c0b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:49.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.320+0000 7f401bfff640 1 --2- 172.21.15.98:0/3535511897 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f401c06e4b0 0x7f401c0b4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.320+0000 7f401bfff640 1 -- 172.21.15.98:0/3535511897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f401c0d03f0 msgr2=0x7f401c1f4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:49.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.320+0000 7f401bfff640 1 --2- 172.21.15.98:0/3535511897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f401c0d03f0 0x7f401c1f4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.320+0000 7f401bfff640 1 -- 172.21.15.98:0/3535511897 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c1f52e0 con 0x7f401c15f140 2024-05-09T10:50:49.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.320+0000 7f401bfff640 1 --2- 172.21.15.98:0/3535511897 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f401c15f140 0x7f401c14a2b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f401402abc0 tx=0x7f4014004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:49.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.320+0000 7f4018ff9640 1 -- 172.21.15.98:0/3535511897 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4014040070 con 0x7f401c15f140 2024-05-09T10:50:49.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.320+0000 7f4018ff9640 1 -- 172.21.15.98:0/3535511897 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4014030b60 con 0x7f401c15f140 2024-05-09T10:50:49.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.320+0000 7f4021af5640 1 -- 172.21.15.98:0/3535511897 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f401c0b3db0 con 0x7f401c15f140 2024-05-09T10:50:49.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.320+0000 7f4018ff9640 1 -- 172.21.15.98:0/3535511897 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4014009450 con 0x7f401c15f140 2024-05-09T10:50:49.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.320+0000 7f4021af5640 1 -- 172.21.15.98:0/3535511897 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f401c0b4370 con 0x7f401c15f140 2024-05-09T10:50:49.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.321+0000 7f4021af5640 1 -- 172.21.15.98:0/3535511897 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f401c072dd0 con 0x7f401c15f140 2024-05-09T10:50:49.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.322+0000 7f4018ff9640 1 -- 172.21.15.98:0/3535511897 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4014039040 con 0x7f401c15f140 2024-05-09T10:50:49.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.322+0000 7f4018ff9640 1 --2- 172.21.15.98:0/3535511897 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3ffc06b4d0 0x7f3ffc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.323+0000 7f401b7fe640 1 --2- 172.21.15.98:0/3535511897 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3ffc06b4d0 0x7f3ffc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.323+0000 7f4018ff9640 1 -- 172.21.15.98:0/3535511897 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40140ab7f0 con 0x7f401c15f140 2024-05-09T10:50:49.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.323+0000 7f401b7fe640 1 --2- 172.21.15.98:0/3535511897 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3ffc06b4d0 0x7f3ffc06d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f400c009aa0 tx=0x7f400c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:49.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.327+0000 7f4018ff9640 1 -- 172.21.15.98:0/3535511897 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f401407a140 con 0x7f401c15f140 2024-05-09T10:50:49.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.502+0000 7f4021af5640 1 -- 172.21.15.98:0/3535511897 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f401c04f000 con 0x7f401c15f140 2024-05-09T10:50:49.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.502+0000 7f4018ff9640 1 -- 172.21.15.98:0/3535511897 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f401407e1e0 con 0x7f401c15f140 2024-05-09T10:50:49.496 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2024-05-09T10:50:49.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.504+0000 7f4021af5640 1 -- 172.21.15.98:0/3535511897 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3ffc06b4d0 msgr2=0x7f3ffc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:49.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.504+0000 7f4021af5640 1 --2- 172.21.15.98:0/3535511897 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3ffc06b4d0 0x7f3ffc06d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f400c009aa0 tx=0x7f400c009190 comp rx=0 tx=0).stop 2024-05-09T10:50:49.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.504+0000 7f4021af5640 1 -- 172.21.15.98:0/3535511897 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f401c15f140 msgr2=0x7f401c14a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:49.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.504+0000 7f4021af5640 1 --2- 172.21.15.98:0/3535511897 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f401c15f140 0x7f401c14a2b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f401402abc0 tx=0x7f4014004a40 comp rx=0 tx=0).stop 2024-05-09T10:50:49.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.504+0000 7f4021af5640 1 -- 172.21.15.98:0/3535511897 shutdown_connections 2024-05-09T10:50:49.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.504+0000 7f4021af5640 1 --2- 172.21.15.98:0/3535511897 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3ffc06b4d0 0x7f3ffc06d990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.504+0000 7f4021af5640 1 --2- 172.21.15.98:0/3535511897 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f401c15f140 0x7f401c14a2b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.504+0000 7f4021af5640 1 --2- 172.21.15.98:0/3535511897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f401c0d03f0 0x7f401c1f4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.504+0000 7f4021af5640 1 --2- 172.21.15.98:0/3535511897 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f401c06e4b0 0x7f401c0b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.504+0000 7f4021af5640 1 -- 172.21.15.98:0/3535511897 >> 172.21.15.98:0/3535511897 conn(0x7f401c066fd0 msgr2=0x7f401c06da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:49.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.504+0000 7f4021af5640 1 -- 172.21.15.98:0/3535511897 shutdown_connections 2024-05-09T10:50:49.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.504+0000 7f4021af5640 1 -- 172.21.15.98:0/3535511897 wait complete. 2024-05-09T10:50:49.509 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T10:50:49.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.616+0000 7f74fda15640 1 Processor -- start 2024-05-09T10:50:49.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.617+0000 7f74fda15640 1 -- start start 2024-05-09T10:50:49.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.617+0000 7f74fda15640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74f815a100 0x7f74f806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.617+0000 7f74fda15640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f74f815a4e0 0x7f74f80c4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.617+0000 7f74fda15640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f80cd660 con 0x7f74f80c4b60 2024-05-09T10:50:49.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.617+0000 7f74fda15640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74f80cd920 con 0x7f74f815a4e0 2024-05-09T10:50:49.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.617+0000 7f74fda15640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f80cdc00 con 0x7f74f815a100 2024-05-09T10:50:49.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.618+0000 7f74ee7fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f74f815a4e0 0x7f74f80c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.618+0000 7f74ee7fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f74f815a4e0 0x7f74f80c4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34328/0 (socket says 172.21.15.98:34328) 2024-05-09T10:50:49.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.618+0000 7f74ee7fc640 1 -- 172.21.15.98:0/2134091870 learned_addr learned my addr 172.21.15.98:0/2134091870 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:49.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.618+0000 7f74f6ffd640 1 --2- 172.21.15.98:0/2134091870 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74f815a100 0x7f74f806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.618+0000 7f74ee7fc640 1 -- 172.21.15.98:0/2134091870 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74f815a100 msgr2=0x7f74f806d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:49.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.618+0000 7f74ee7fc640 1 --2- 172.21.15.98:0/2134091870 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74f815a100 0x7f74f806d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.618+0000 7f74ee7fc640 1 -- 172.21.15.98:0/2134091870 >> v1:172.21.15.105:6789/0 conn(0x7f74f80c4b60 legacy=0x7f74f80ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:49.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.618+0000 7f74ee7fc640 1 -- 172.21.15.98:0/2134091870 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f80cdee0 con 0x7f74f815a4e0 2024-05-09T10:50:49.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.619+0000 7f74ee7fc640 1 --2- 172.21.15.98:0/2134091870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74f815a4e0 0x7f74f80c4620 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f74e000bb00 tx=0x7f74e002c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7c0f9ca3db3dcc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:49.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.619+0000 7f74f67fc640 1 -- 172.21.15.98:0/2134091870 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e003a070 con 0x7f74f815a4e0 2024-05-09T10:50:49.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.619+0000 7f74f67fc640 1 -- 172.21.15.98:0/2134091870 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e0032480 con 0x7f74f815a4e0 2024-05-09T10:50:49.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.619+0000 7f74f67fc640 1 -- 172.21.15.98:0/2134091870 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e0032850 con 0x7f74f815a4e0 2024-05-09T10:50:49.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.619+0000 7f74fda15640 1 -- 172.21.15.98:0/2134091870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74f815a4e0 msgr2=0x7f74f80c4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:49.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.619+0000 7f74fda15640 1 --2- 172.21.15.98:0/2134091870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74f815a4e0 0x7f74f80c4620 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f74e000bb00 tx=0x7f74e002c770 comp rx=0 tx=0).stop 2024-05-09T10:50:49.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.619+0000 7f74fda15640 1 -- 172.21.15.98:0/2134091870 shutdown_connections 2024-05-09T10:50:49.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.619+0000 7f74fda15640 1 --2- 172.21.15.98:0/2134091870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74f815a4e0 0x7f74f80c4620 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.619+0000 7f74fda15640 1 --2- 172.21.15.98:0/2134091870 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74f815a100 0x7f74f806d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.619+0000 7f74fda15640 1 -- 172.21.15.98:0/2134091870 >> 172.21.15.98:0/2134091870 conn(0x7f74f80cf760 msgr2=0x7f74f80cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:49.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.619+0000 7f74fda15640 1 -- 172.21.15.98:0/2134091870 shutdown_connections 2024-05-09T10:50:49.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.619+0000 7f74fda15640 1 -- 172.21.15.98:0/2134091870 wait complete. 2024-05-09T10:50:49.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.621+0000 7f74fda15640 1 Processor -- start 2024-05-09T10:50:49.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.621+0000 7f74fda15640 1 -- start start 2024-05-09T10:50:49.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.621+0000 7f74fda15640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74f80c4b60 0x7f74f81570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.621+0000 7f74fda15640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74f815a100 0x7f74f8156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.621+0000 7f74fda15640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74f815a4e0 0x7f74f80c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.621+0000 7f74fda15640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f80cedf0 con 0x7f74f80c4b60 2024-05-09T10:50:49.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.621+0000 7f74fda15640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f806e460 con 0x7f74f815a4e0 2024-05-09T10:50:49.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.621+0000 7f74fda15640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f80cef70 con 0x7f74f815a100 2024-05-09T10:50:49.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.622+0000 7f74f6ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74f80c4b60 0x7f74f81570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.622+0000 7f74f6ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74f80c4b60 0x7f74f81570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34344/0 (socket says 172.21.15.98:34344) 2024-05-09T10:50:49.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.622+0000 7f74f6ffd640 1 -- 172.21.15.98:0/1103678442 learned_addr learned my addr 172.21.15.98:0/1103678442 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:49.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.622+0000 7f74ee7fc640 1 --2- 172.21.15.98:0/1103678442 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74f815a100 0x7f74f8156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.622+0000 7f74f77fe640 1 --2- 172.21.15.98:0/1103678442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74f815a4e0 0x7f74f80c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.622+0000 7f74f6ffd640 1 -- 172.21.15.98:0/1103678442 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74f815a100 msgr2=0x7f74f8156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:49.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.622+0000 7f74f6ffd640 1 --2- 172.21.15.98:0/1103678442 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74f815a100 0x7f74f8156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.622+0000 7f74f6ffd640 1 -- 172.21.15.98:0/1103678442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74f815a4e0 msgr2=0x7f74f80c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:49.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.622+0000 7f74f6ffd640 1 --2- 172.21.15.98:0/1103678442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74f815a4e0 0x7f74f80c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.622+0000 7f74f6ffd640 1 -- 172.21.15.98:0/1103678442 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f80b4df0 con 0x7f74f80c4b60 2024-05-09T10:50:49.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.622+0000 7f74f6ffd640 1 --2- 172.21.15.98:0/1103678442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74f80c4b60 0x7f74f81570e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f74e800e9b0 tx=0x7f74e800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:49.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.622+0000 7f74f4ff9640 1 -- 172.21.15.98:0/1103678442 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e8017070 con 0x7f74f80c4b60 2024-05-09T10:50:49.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.622+0000 7f74f4ff9640 1 -- 172.21.15.98:0/1103678442 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e800c9a0 con 0x7f74f80c4b60 2024-05-09T10:50:49.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.622+0000 7f74fda15640 1 -- 172.21.15.98:0/1103678442 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f80b5070 con 0x7f74f80c4b60 2024-05-09T10:50:49.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.622+0000 7f74f4ff9640 1 -- 172.21.15.98:0/1103678442 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e800cd90 con 0x7f74f80c4b60 2024-05-09T10:50:49.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.622+0000 7f74fda15640 1 -- 172.21.15.98:0/1103678442 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f80b5630 con 0x7f74f80c4b60 2024-05-09T10:50:49.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.623+0000 7f74fda15640 1 -- 172.21.15.98:0/1103678442 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f8072c80 con 0x7f74f80c4b60 2024-05-09T10:50:49.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.624+0000 7f74f4ff9640 1 -- 172.21.15.98:0/1103678442 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f74e801a070 con 0x7f74f80c4b60 2024-05-09T10:50:49.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.624+0000 7f74f4ff9640 1 --2- 172.21.15.98:0/1103678442 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f74d806b4d0 0x7f74d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.624+0000 7f74f4ff9640 1 -- 172.21.15.98:0/1103678442 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74e808a100 con 0x7f74f80c4b60 2024-05-09T10:50:49.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.624+0000 7f74ee7fc640 1 --2- 172.21.15.98:0/1103678442 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f74d806b4d0 0x7f74d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.625+0000 7f74ee7fc640 1 --2- 172.21.15.98:0/1103678442 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f74d806b4d0 0x7f74d806d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f74e0037000 tx=0x7f74e0032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:49.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.629+0000 7f74f4ff9640 1 -- 172.21.15.98:0/1103678442 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f74f80b4df0 con 0x7f74f80c4b60 2024-05-09T10:50:49.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.800+0000 7f74fda15640 1 -- 172.21.15.98:0/1103678442 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f74f80ae4d0 con 0x7f74f80c4b60 2024-05-09T10:50:49.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.801+0000 7f74f4ff9640 1 -- 172.21.15.98:0/1103678442 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f74e80589c0 con 0x7f74f80c4b60 2024-05-09T10:50:49.795 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2024-05-09T10:50:49.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.803+0000 7f74fda15640 1 -- 172.21.15.98:0/1103678442 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f74d806b4d0 msgr2=0x7f74d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:49.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.803+0000 7f74fda15640 1 --2- 172.21.15.98:0/1103678442 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f74d806b4d0 0x7f74d806d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f74e0037000 tx=0x7f74e0032000 comp rx=0 tx=0).stop 2024-05-09T10:50:49.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.803+0000 7f74fda15640 1 -- 172.21.15.98:0/1103678442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74f80c4b60 msgr2=0x7f74f81570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:49.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.803+0000 7f74fda15640 1 --2- 172.21.15.98:0/1103678442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74f80c4b60 0x7f74f81570e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f74e800e9b0 tx=0x7f74e800ee70 comp rx=0 tx=0).stop 2024-05-09T10:50:49.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.803+0000 7f74fda15640 1 -- 172.21.15.98:0/1103678442 shutdown_connections 2024-05-09T10:50:49.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.803+0000 7f74fda15640 1 --2- 172.21.15.98:0/1103678442 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f74d806b4d0 0x7f74d806d990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.803+0000 7f74fda15640 1 --2- 172.21.15.98:0/1103678442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74f815a4e0 0x7f74f80c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.803+0000 7f74fda15640 1 --2- 172.21.15.98:0/1103678442 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74f815a100 0x7f74f8156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.803+0000 7f74fda15640 1 --2- 172.21.15.98:0/1103678442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74f80c4b60 0x7f74f81570e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.803+0000 7f74fda15640 1 -- 172.21.15.98:0/1103678442 >> 172.21.15.98:0/1103678442 conn(0x7f74f80cf760 msgr2=0x7f74f80af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:49.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.803+0000 7f74fda15640 1 -- 172.21.15.98:0/1103678442 shutdown_connections 2024-05-09T10:50:49.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.803+0000 7f74fda15640 1 -- 172.21.15.98:0/1103678442 wait complete. 2024-05-09T10:50:49.809 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T10:50:49.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.915+0000 7ffaf3fd5640 1 Processor -- start 2024-05-09T10:50:49.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.916+0000 7ffaf3fd5640 1 -- start start 2024-05-09T10:50:49.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.916+0000 7ffaf3fd5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffaec06eb30 0x7ffaec0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.916+0000 7ffaf3fd5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ffaec15ddc0 0x7ffaec15e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.916+0000 7ffaf3fd5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaec0d07a0 con 0x7ffaec06d570 2024-05-09T10:50:49.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.916+0000 7ffaf3fd5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaec0d0db0 con 0x7ffaec15ddc0 2024-05-09T10:50:49.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.916+0000 7ffaf3fd5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec06f9b0 con 0x7ffaec06eb30 2024-05-09T10:50:49.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.917+0000 7ffaf1570640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ffaec06d570 0x7ffaec06ef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41128/0 (socket says 172.21.15.98:41128) 2024-05-09T10:50:49.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.917+0000 7ffaf1570640 1 -- 172.21.15.98:0/2667555296 learned_addr learned my addr 172.21.15.98:0/2667555296 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:49.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.917+0000 7ffaf2572640 1 --2- 172.21.15.98:0/2667555296 >> v2:172.21.15.105:3300/0 conn(0x7ffaec15ddc0 0x7ffaec15e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.917+0000 7ffaf1d71640 1 --2- 172.21.15.98:0/2667555296 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffaec06eb30 0x7ffaec0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.917+0000 7ffaf0d6f640 1 -- 172.21.15.98:0/2667555296 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3850290876 0 0) 0x7ffaec0d07a0 con 0x7ffaec06d570 2024-05-09T10:50:49.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.917+0000 7ffaf0d6f640 1 -- 172.21.15.98:0/2667555296 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffad4003520 con 0x7ffaec06d570 2024-05-09T10:50:49.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.917+0000 7ffaf0d6f640 1 -- 172.21.15.98:0/2667555296 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3263092369 0 0) 0x7ffad4003520 con 0x7ffaec06d570 2024-05-09T10:50:49.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.917+0000 7ffaf0d6f640 1 -- 172.21.15.98:0/2667555296 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffaec06eb30 msgr2=0x7ffaec0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:49.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.917+0000 7ffaf0d6f640 1 --2- 172.21.15.98:0/2667555296 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffaec06eb30 0x7ffaec0b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.917+0000 7ffaf0d6f640 1 -- 172.21.15.98:0/2667555296 >> v2:172.21.15.105:3300/0 conn(0x7ffaec15ddc0 msgr2=0x7ffaec15e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:49.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.917+0000 7ffaf0d6f640 1 --2- 172.21.15.98:0/2667555296 >> v2:172.21.15.105:3300/0 conn(0x7ffaec15ddc0 0x7ffaec15e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.917+0000 7ffaf0d6f640 1 -- 172.21.15.98:0/2667555296 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaec06f6d0 con 0x7ffaec06d570 2024-05-09T10:50:49.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.917+0000 7ffaf0d6f640 1 -- 172.21.15.98:0/2667555296 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7ffae0001670 con 0x7ffaec06d570 2024-05-09T10:50:49.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.917+0000 7ffaf0d6f640 1 -- 172.21.15.98:0/2667555296 >> v1:172.21.15.98:6789/0 conn(0x7ffaec06d570 legacy=0x7ffaec06ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:49.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.917+0000 7ffaf0d6f640 1 --2- 172.21.15.98:0/2667555296 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffaec06eb30 0x7ffad40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.917+0000 7ffaf0d6f640 1 --2- 172.21.15.98:0/2667555296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffad4003f30 0x7ffad4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.917+0000 7ffaf0d6f640 1 --2- 172.21.15.98:0/2667555296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffad4004d40 0x7ffad40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.917+0000 7ffaf0d6f640 1 -- 172.21.15.98:0/2667555296 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec0d0db0 con 0x7ffad4004d40 2024-05-09T10:50:49.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.917+0000 7ffaf0d6f640 1 -- 172.21.15.98:0/2667555296 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec06f9b0 con 0x7ffad4003f30 2024-05-09T10:50:49.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.917+0000 7ffaf0d6f640 1 -- 172.21.15.98:0/2667555296 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad400ba10 con 0x7ffaec06eb30 2024-05-09T10:50:49.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.917+0000 7ffaf2572640 1 --2- 172.21.15.98:0/2667555296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffad4004d40 0x7ffad40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.917+0000 7ffaf1d71640 1 --2- 172.21.15.98:0/2667555296 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffaec06eb30 0x7ffad40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.917+0000 7ffaf1570640 1 --2- 172.21.15.98:0/2667555296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffad4003f30 0x7ffad4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.918+0000 7ffaf2572640 1 -- 172.21.15.98:0/2667555296 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffaec06eb30 msgr2=0x7ffad40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:49.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.918+0000 7ffaf2572640 1 --2- 172.21.15.98:0/2667555296 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffaec06eb30 0x7ffad40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.918+0000 7ffaf2572640 1 -- 172.21.15.98:0/2667555296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffad4003f30 msgr2=0x7ffad4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:49.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.918+0000 7ffaf2572640 1 --2- 172.21.15.98:0/2667555296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffad4003f30 0x7ffad4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.918+0000 7ffaf2572640 1 -- 172.21.15.98:0/2667555296 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad4003520 con 0x7ffad4004d40 2024-05-09T10:50:49.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.918+0000 7ffaf2572640 1 --2- 172.21.15.98:0/2667555296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffad4004d40 0x7ffad40051c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ffadc00e170 tx=0x7ffad4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db76a46d26863d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:49.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.918+0000 7ffaf0d6f640 1 -- 172.21.15.98:0/2667555296 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc034020 con 0x7ffad4004d40 2024-05-09T10:50:49.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.918+0000 7ffaf0d6f640 1 -- 172.21.15.98:0/2667555296 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffadc030430 con 0x7ffad4004d40 2024-05-09T10:50:49.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.918+0000 7ffaf0d6f640 1 -- 172.21.15.98:0/2667555296 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc009c10 con 0x7ffad4004d40 2024-05-09T10:50:49.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.919+0000 7ffaf3fd5640 1 -- 172.21.15.98:0/2667555296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffad4004d40 msgr2=0x7ffad40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:49.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.919+0000 7ffaf3fd5640 1 --2- 172.21.15.98:0/2667555296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffad4004d40 0x7ffad40051c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ffadc00e170 tx=0x7ffad4005830 comp rx=0 tx=0).stop 2024-05-09T10:50:49.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.919+0000 7ffaf2572640 1 -- 172.21.15.98:0/2667555296 reap_dead start 2024-05-09T10:50:49.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.919+0000 7ffaf3fd5640 1 -- 172.21.15.98:0/2667555296 shutdown_connections 2024-05-09T10:50:49.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.919+0000 7ffaf3fd5640 1 -- 172.21.15.98:0/2667555296 >> 172.21.15.98:0/2667555296 conn(0x7ffaec0b34e0 msgr2=0x7ffaec0b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:49.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.919+0000 7ffaf3fd5640 1 -- 172.21.15.98:0/2667555296 shutdown_connections 2024-05-09T10:50:49.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.919+0000 7ffaf3fd5640 1 -- 172.21.15.98:0/2667555296 wait complete. 2024-05-09T10:50:49.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.920+0000 7ffaf3fd5640 1 Processor -- start 2024-05-09T10:50:49.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.920+0000 7ffaf3fd5640 1 -- start start 2024-05-09T10:50:49.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.921+0000 7ffaf3fd5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaec155b80 0x7ffaec0b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.921+0000 7ffaf3fd5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffaec0b4780 0x7ffaec0b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.921+0000 7ffaf3fd5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaec0c6fc0 0x7ffaec0c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.921+0000 7ffaf3fd5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec0b4000 con 0x7ffaec155b80 2024-05-09T10:50:49.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.921+0000 7ffaf3fd5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec15e790 con 0x7ffaec0c6fc0 2024-05-09T10:50:49.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.921+0000 7ffaf3fd5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec0b4180 con 0x7ffaec0b4780 2024-05-09T10:50:49.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.921+0000 7ffaf1d71640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaec155b80 0x7ffaec0b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.921+0000 7ffaf1d71640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaec155b80 0x7ffaec0b6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34358/0 (socket says 172.21.15.98:34358) 2024-05-09T10:50:49.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.921+0000 7ffaf1d71640 1 -- 172.21.15.98:0/1230652879 learned_addr learned my addr 172.21.15.98:0/1230652879 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:49.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.921+0000 7ffaf1d71640 1 -- 172.21.15.98:0/1230652879 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffaec0b4780 msgr2=0x7ffaec0b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:49.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.921+0000 7ffaf1d71640 1 --2- 172.21.15.98:0/1230652879 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffaec0b4780 0x7ffaec0b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.921+0000 7ffaf1d71640 1 -- 172.21.15.98:0/1230652879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaec0c6fc0 msgr2=0x7ffaec0c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:49.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.921+0000 7ffaf2572640 1 --2- 172.21.15.98:0/1230652879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaec0c6fc0 0x7ffaec0c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.921+0000 7ffaf1d71640 1 --2- 172.21.15.98:0/1230652879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaec0c6fc0 0x7ffaec0c7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:49.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.921+0000 7ffaf1d71640 1 -- 172.21.15.98:0/1230652879 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaec0c7ba0 con 0x7ffaec155b80 2024-05-09T10:50:49.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.922+0000 7ffaf1d71640 1 --2- 172.21.15.98:0/1230652879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaec155b80 0x7ffaec0b6c40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ffae40119d0 tx=0x7ffae4011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:49.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.922+0000 7ffaea7fc640 1 -- 172.21.15.98:0/1230652879 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae400faf0 con 0x7ffaec155b80 2024-05-09T10:50:49.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.922+0000 7ffaea7fc640 1 -- 172.21.15.98:0/1230652879 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae400fc90 con 0x7ffaec155b80 2024-05-09T10:50:49.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.922+0000 7ffaea7fc640 1 -- 172.21.15.98:0/1230652879 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae4014440 con 0x7ffaec155b80 2024-05-09T10:50:49.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.922+0000 7ffaf3fd5640 1 -- 172.21.15.98:0/1230652879 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaec0c7e80 con 0x7ffaec155b80 2024-05-09T10:50:49.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.922+0000 7ffaf3fd5640 1 -- 172.21.15.98:0/1230652879 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaec15bf30 con 0x7ffaec155b80 2024-05-09T10:50:49.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.923+0000 7ffaf3fd5640 1 -- 172.21.15.98:0/1230652879 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaec0b0b60 con 0x7ffaec155b80 2024-05-09T10:50:49.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.923+0000 7ffaea7fc640 1 -- 172.21.15.98:0/1230652879 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffae4012040 con 0x7ffaec155b80 2024-05-09T10:50:49.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.923+0000 7ffaea7fc640 1 --2- 172.21.15.98:0/1230652879 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ffad806b4d0 0x7ffad806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:49.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.924+0000 7ffaea7fc640 1 -- 172.21.15.98:0/1230652879 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffae408a870 con 0x7ffaec155b80 2024-05-09T10:50:49.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.924+0000 7ffaf1570640 1 --2- 172.21.15.98:0/1230652879 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ffad806b4d0 0x7ffad806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:49.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.924+0000 7ffaf1570640 1 --2- 172.21.15.98:0/1230652879 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ffad806b4d0 0x7ffad806d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffae0001c00 tx=0x7ffae0001b50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:49.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:49.928+0000 7ffaea7fc640 1 -- 172.21.15.98:0/1230652879 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffae4019030 con 0x7ffaec155b80 2024-05-09T10:50:50.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.111+0000 7ffaf3fd5640 1 -- 172.21.15.98:0/1230652879 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffaec15c310 con 0x7ffaec155b80 2024-05-09T10:50:50.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.111+0000 7ffaea7fc640 1 -- 172.21.15.98:0/1230652879 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7ffae40596f0 con 0x7ffaec155b80 2024-05-09T10:50:50.106 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-05-09T10:50:50.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.113+0000 7ffaf3fd5640 1 -- 172.21.15.98:0/1230652879 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ffad806b4d0 msgr2=0x7ffad806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:50.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.113+0000 7ffaf3fd5640 1 --2- 172.21.15.98:0/1230652879 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ffad806b4d0 0x7ffad806d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffae0001c00 tx=0x7ffae0001b50 comp rx=0 tx=0).stop 2024-05-09T10:50:50.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.113+0000 7ffaf3fd5640 1 -- 172.21.15.98:0/1230652879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaec155b80 msgr2=0x7ffaec0b6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:50.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.113+0000 7ffaf3fd5640 1 --2- 172.21.15.98:0/1230652879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaec155b80 0x7ffaec0b6c40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ffae40119d0 tx=0x7ffae4011e90 comp rx=0 tx=0).stop 2024-05-09T10:50:50.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.113+0000 7ffaf3fd5640 1 -- 172.21.15.98:0/1230652879 shutdown_connections 2024-05-09T10:50:50.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.113+0000 7ffaf3fd5640 1 --2- 172.21.15.98:0/1230652879 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ffad806b4d0 0x7ffad806d990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.113+0000 7ffaf3fd5640 1 --2- 172.21.15.98:0/1230652879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffaec0c6fc0 0x7ffaec0c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.113+0000 7ffaf3fd5640 1 --2- 172.21.15.98:0/1230652879 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffaec0b4780 0x7ffaec0b7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.113+0000 7ffaf3fd5640 1 --2- 172.21.15.98:0/1230652879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaec155b80 0x7ffaec0b6c40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.113+0000 7ffaf3fd5640 1 -- 172.21.15.98:0/1230652879 >> 172.21.15.98:0/1230652879 conn(0x7ffaec0b34e0 msgr2=0x7ffaec0b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:50.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.113+0000 7ffaf3fd5640 1 -- 172.21.15.98:0/1230652879 shutdown_connections 2024-05-09T10:50:50.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.113+0000 7ffaf3fd5640 1 -- 172.21.15.98:0/1230652879 wait complete. 2024-05-09T10:50:50.119 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-09T10:50:50.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:50.119 DEBUG:teuthology.orchestra.run.smithi105:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-05-09T10:50:50.152 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2024-05-09T10:50:50.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:50.152 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-09T10:50:50.213 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:50:50.213 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:50:50.229 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:50:50.229 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:50:50.284 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:50:50.284 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:50:50.340 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:50.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.485+0000 7f4510c4a640 1 Processor -- start 2024-05-09T10:50:50.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.485+0000 7f4510c4a640 1 -- start start 2024-05-09T10:50:50.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.486+0000 7f4510c4a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f450c143740 0x7f450c0720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:50.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.486+0000 7f4510c4a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f450c1443e0 0x7f450c15e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:50.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.486+0000 7f4510c4a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f450c143b20 con 0x7f450c143d70 2024-05-09T10:50:50.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.486+0000 7f4510c4a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f450c15f460 con 0x7f450c1443e0 2024-05-09T10:50:50.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.486+0000 7f4510c4a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450c0b28c0 con 0x7f450c143740 2024-05-09T10:50:50.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.486+0000 7f450a59c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f450c143740 0x7f450c0720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:50.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.486+0000 7f4501d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f450c1443e0 0x7f450c15e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:50.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.486+0000 7f450ad9d640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f450c143d70 0x7f450c15ec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.98:46518/0 (socket says 172.21.15.98:46518) 2024-05-09T10:50:50.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.486+0000 7f450ad9d640 1 -- 172.21.15.98:0/3301073386 learned_addr learned my addr 172.21.15.98:0/3301073386 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:50.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.486+0000 7f4501d9b640 1 -- 172.21.15.98:0/3301073386 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f450c143740 msgr2=0x7f450c0720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:50.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.486+0000 7f4501d9b640 1 --2- 172.21.15.98:0/3301073386 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f450c143740 0x7f450c0720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.486+0000 7f4501d9b640 1 -- 172.21.15.98:0/3301073386 >> v1:172.21.15.105:6789/0 conn(0x7f450c143d70 legacy=0x7f450c15ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:50.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.486+0000 7f4501d9b640 1 -- 172.21.15.98:0/3301073386 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450c0b2ba0 con 0x7f450c1443e0 2024-05-09T10:50:50.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.486+0000 7f450a59c640 1 --2- 172.21.15.98:0/3301073386 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f450c143740 0x7f450c0720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:50:50.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.487+0000 7f4501d9b640 1 --2- 172.21.15.98:0/3301073386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f450c1443e0 0x7f450c15e6e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f44f40097a0 tx=0x7f44f402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=826c8b363e9f30ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:50.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.487+0000 7f4509d9b640 1 -- 172.21.15.98:0/3301073386 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4037070 con 0x7f450c1443e0 2024-05-09T10:50:50.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.487+0000 7f4509d9b640 1 -- 172.21.15.98:0/3301073386 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f402ae50 con 0x7f450c1443e0 2024-05-09T10:50:50.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.487+0000 7f4509d9b640 1 -- 172.21.15.98:0/3301073386 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4032660 con 0x7f450c1443e0 2024-05-09T10:50:50.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.488+0000 7f4510c4a640 1 -- 172.21.15.98:0/3301073386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f450c1443e0 msgr2=0x7f450c15e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:50.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.488+0000 7f4510c4a640 1 --2- 172.21.15.98:0/3301073386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f450c1443e0 0x7f450c15e6e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f44f40097a0 tx=0x7f44f402a100 comp rx=0 tx=0).stop 2024-05-09T10:50:50.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.488+0000 7f4510c4a640 1 -- 172.21.15.98:0/3301073386 shutdown_connections 2024-05-09T10:50:50.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.488+0000 7f4510c4a640 1 --2- 172.21.15.98:0/3301073386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f450c1443e0 0x7f450c15e6e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.488+0000 7f4510c4a640 1 --2- 172.21.15.98:0/3301073386 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f450c143740 0x7f450c0720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.488+0000 7f4510c4a640 1 -- 172.21.15.98:0/3301073386 >> 172.21.15.98:0/3301073386 conn(0x7f450c06abe0 msgr2=0x7f450c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:50.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.488+0000 7f4510c4a640 1 -- 172.21.15.98:0/3301073386 shutdown_connections 2024-05-09T10:50:50.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.488+0000 7f4510c4a640 1 -- 172.21.15.98:0/3301073386 wait complete. 2024-05-09T10:50:50.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.489+0000 7f4510c4a640 1 Processor -- start 2024-05-09T10:50:50.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.489+0000 7f4510c4a640 1 -- start start 2024-05-09T10:50:50.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.490+0000 7f4510c4a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f450c143740 0x7f450c139220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:50.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.490+0000 7f4510c4a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f450c143d70 0x7f450c0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:50.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.490+0000 7f4510c4a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f450c1443e0 0x7f450c15bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:50.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.490+0000 7f4510c4a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450c0b34d0 con 0x7f450c143d70 2024-05-09T10:50:50.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.490+0000 7f4510c4a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450c0b3350 con 0x7f450c143740 2024-05-09T10:50:50.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.490+0000 7f4510c4a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450c0b3650 con 0x7f450c1443e0 2024-05-09T10:50:50.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.490+0000 7f4501d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f450c143d70 0x7f450c0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:50.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.490+0000 7f4501d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f450c143d70 0x7f450c0cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34366/0 (socket says 172.21.15.98:34366) 2024-05-09T10:50:50.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.490+0000 7f4501d9b640 1 -- 172.21.15.98:0/2996937991 learned_addr learned my addr 172.21.15.98:0/2996937991 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:50.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.490+0000 7f450a59c640 1 --2- 172.21.15.98:0/2996937991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f450c143740 0x7f450c139220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:50.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.490+0000 7f450ad9d640 1 --2- 172.21.15.98:0/2996937991 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f450c1443e0 0x7f450c15bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:50.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.490+0000 7f4501d9b640 1 -- 172.21.15.98:0/2996937991 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f450c1443e0 msgr2=0x7f450c15bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:50.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.491+0000 7f4501d9b640 1 --2- 172.21.15.98:0/2996937991 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f450c1443e0 0x7f450c15bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.491+0000 7f4501d9b640 1 -- 172.21.15.98:0/2996937991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f450c143740 msgr2=0x7f450c139220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:50.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.491+0000 7f4501d9b640 1 --2- 172.21.15.98:0/2996937991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f450c143740 0x7f450c139220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.491+0000 7f4501d9b640 1 -- 172.21.15.98:0/2996937991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450c15c000 con 0x7f450c143d70 2024-05-09T10:50:50.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.491+0000 7f450ad9d640 1 --2- 172.21.15.98:0/2996937991 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f450c1443e0 0x7f450c15bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:50.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.491+0000 7f4501d9b640 1 --2- 172.21.15.98:0/2996937991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f450c143d70 0x7f450c0cfb20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f44f402a140 tx=0x7f44f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:50.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.491+0000 7f45037fe640 1 -- 172.21.15.98:0/2996937991 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4041070 con 0x7f450c143d70 2024-05-09T10:50:50.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.491+0000 7f45037fe640 1 -- 172.21.15.98:0/2996937991 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f403c670 con 0x7f450c143d70 2024-05-09T10:50:50.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.491+0000 7f4510c4a640 1 -- 172.21.15.98:0/2996937991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f450c15d740 con 0x7f450c143d70 2024-05-09T10:50:50.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.491+0000 7f45037fe640 1 -- 172.21.15.98:0/2996937991 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4037040 con 0x7f450c143d70 2024-05-09T10:50:50.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.491+0000 7f4510c4a640 1 -- 172.21.15.98:0/2996937991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f450c15dce0 con 0x7f450c143d70 2024-05-09T10:50:50.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.492+0000 7f4510c4a640 1 -- 172.21.15.98:0/2996937991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f450c069cb0 con 0x7f450c143d70 2024-05-09T10:50:50.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.492+0000 7f45037fe640 1 -- 172.21.15.98:0/2996937991 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44f4044070 con 0x7f450c143d70 2024-05-09T10:50:50.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.493+0000 7f45037fe640 1 --2- 172.21.15.98:0/2996937991 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f44ec06b500 0x7f44ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:50.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.493+0000 7f450a59c640 1 --2- 172.21.15.98:0/2996937991 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f44ec06b500 0x7f44ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:50.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.493+0000 7f45037fe640 1 -- 172.21.15.98:0/2996937991 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44f40aa7e0 con 0x7f450c143d70 2024-05-09T10:50:50.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.494+0000 7f450a59c640 1 --2- 172.21.15.98:0/2996937991 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f44ec06b500 0x7f44ec06d9c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009560 tx=0x7f44fc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:50.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.497+0000 7f45037fe640 1 -- 172.21.15.98:0/2996937991 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44f40796f0 con 0x7f450c143d70 2024-05-09T10:50:50.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.668+0000 7f4510c4a640 1 -- 172.21.15.98:0/2996937991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f450c15e450 con 0x7f450c143d70 2024-05-09T10:50:50.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.669+0000 7f45037fe640 1 -- 172.21.15.98:0/2996937991 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f44f4078ff0 con 0x7f450c143d70 2024-05-09T10:50:50.664 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:50.664 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":20,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:21.176419+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:50:17.323444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T10:50:18.482315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:50.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.671+0000 7f4510c4a640 1 -- 172.21.15.98:0/2996937991 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f44ec06b500 msgr2=0x7f44ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:50.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.671+0000 7f4510c4a640 1 --2- 172.21.15.98:0/2996937991 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f44ec06b500 0x7f44ec06d9c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009560 tx=0x7f44fc008040 comp rx=0 tx=0).stop 2024-05-09T10:50:50.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.671+0000 7f4510c4a640 1 -- 172.21.15.98:0/2996937991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f450c143d70 msgr2=0x7f450c0cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:50.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.671+0000 7f4510c4a640 1 --2- 172.21.15.98:0/2996937991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f450c143d70 0x7f450c0cfb20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f44f402a140 tx=0x7f44f40046c0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.671+0000 7f4510c4a640 1 -- 172.21.15.98:0/2996937991 shutdown_connections 2024-05-09T10:50:50.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.671+0000 7f4510c4a640 1 --2- 172.21.15.98:0/2996937991 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f44ec06b500 0x7f44ec06d9c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.671+0000 7f4510c4a640 1 --2- 172.21.15.98:0/2996937991 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f450c1443e0 0x7f450c15bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.671+0000 7f4510c4a640 1 --2- 172.21.15.98:0/2996937991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f450c143d70 0x7f450c0cfb20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.671+0000 7f4510c4a640 1 --2- 172.21.15.98:0/2996937991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f450c143740 0x7f450c139220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.671+0000 7f4510c4a640 1 -- 172.21.15.98:0/2996937991 >> 172.21.15.98:0/2996937991 conn(0x7f450c06abe0 msgr2=0x7f450c15e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:50.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.671+0000 7f4510c4a640 1 -- 172.21.15.98:0/2996937991 shutdown_connections 2024-05-09T10:50:50.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.671+0000 7f4510c4a640 1 -- 172.21.15.98:0/2996937991 wait complete. 2024-05-09T10:50:50.677 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T10:50:07.396423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T10:50:17.323444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T10:50:18.482315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T10:50:50.677 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T10:50:50.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.786+0000 7f307c646640 1 Processor -- start 2024-05-09T10:50:50.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.786+0000 7f307c646640 1 -- start start 2024-05-09T10:50:50.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.787+0000 7f307c646640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f30740b5080 0x7f30740adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:50.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.787+0000 7f307c646640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30740afa70 con 0x7f30740ad880 2024-05-09T10:50:50.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.787+0000 7f307c646640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f307415e200 con 0x7f30740ae1a0 2024-05-09T10:50:50.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.787+0000 7f307c646640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307415e4c0 con 0x7f30740b5080 2024-05-09T10:50:50.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.787+0000 7f307a3e2640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f30740ad880 0x7f30740b2fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41134/0 (socket says 172.21.15.98:41134) 2024-05-09T10:50:50.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.787+0000 7f307a3e2640 1 -- 172.21.15.98:0/533852141 learned_addr learned my addr 172.21.15.98:0/533852141 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:50.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.787+0000 7f3079be1640 1 --2- 172.21.15.98:0/533852141 >> v2:172.21.15.105:3300/0 conn(0x7f30740b5080 0x7f30740adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:50.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.787+0000 7f30793e0640 1 -- 172.21.15.98:0/533852141 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940575974 0 0) 0x7f3070001110 con 0x7f30740ad880 2024-05-09T10:50:50.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.787+0000 7f30793e0640 1 -- 172.21.15.98:0/533852141 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3060003520 con 0x7f30740ad880 2024-05-09T10:50:50.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.787+0000 7f30793e0640 1 -- 172.21.15.98:0/533852141 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1012286880 0 0) 0x7f3060003520 con 0x7f30740ad880 2024-05-09T10:50:50.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.787+0000 7f30793e0640 1 -- 172.21.15.98:0/533852141 >> v2:172.21.15.105:3300/0 conn(0x7f30740b5080 msgr2=0x7f30740adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:50.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.787+0000 7f30793e0640 1 --2- 172.21.15.98:0/533852141 >> v2:172.21.15.105:3300/0 conn(0x7f30740b5080 0x7f30740adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.787+0000 7f30793e0640 1 -- 172.21.15.98:0/533852141 >> v1:172.21.15.105:6789/0 conn(0x7f30740ae1a0 legacy=0x7f3074167820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:50.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.787+0000 7f30793e0640 1 -- 172.21.15.98:0/533852141 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307415e7a0 con 0x7f30740ad880 2024-05-09T10:50:50.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.787+0000 7f30793e0640 1 -- 172.21.15.98:0/533852141 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f3070001840 con 0x7f30740ad880 2024-05-09T10:50:50.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.787+0000 7f30793e0640 1 -- 172.21.15.98:0/533852141 >> v1:172.21.15.98:6789/0 conn(0x7f30740ad880 legacy=0x7f30740b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:50.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.787+0000 7f30793e0640 1 --2- 172.21.15.98:0/533852141 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3060003a30 0x7f3060003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:50.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.787+0000 7f30793e0640 1 --2- 172.21.15.98:0/533852141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3060004800 0x7f3060004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:50.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.787+0000 7f30793e0640 1 --2- 172.21.15.98:0/533852141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30600052e0 0x7f306000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:50.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.787+0000 7f30793e0640 1 -- 172.21.15.98:0/533852141 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307415e4c0 con 0x7f30600052e0 2024-05-09T10:50:50.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.787+0000 7f30793e0640 1 -- 172.21.15.98:0/533852141 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306000c230 con 0x7f3060004800 2024-05-09T10:50:50.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.787+0000 7f30793e0640 1 -- 172.21.15.98:0/533852141 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306000c430 con 0x7f3060003a30 2024-05-09T10:50:50.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.788+0000 7f307abe3640 1 --2- 172.21.15.98:0/533852141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30600052e0 0x7f306000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:50.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.788+0000 7f3079be1640 1 --2- 172.21.15.98:0/533852141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3060004800 0x7f3060004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:50.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.788+0000 7f307a3e2640 1 --2- 172.21.15.98:0/533852141 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3060003a30 0x7f3060003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:50.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.788+0000 7f307abe3640 1 -- 172.21.15.98:0/533852141 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3060003a30 msgr2=0x7f3060003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:50.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.788+0000 7f307abe3640 1 --2- 172.21.15.98:0/533852141 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3060003a30 0x7f3060003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.788+0000 7f307abe3640 1 -- 172.21.15.98:0/533852141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3060004800 msgr2=0x7f3060004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:50.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.788+0000 7f307abe3640 1 --2- 172.21.15.98:0/533852141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3060004800 0x7f3060004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.788+0000 7f307abe3640 1 -- 172.21.15.98:0/533852141 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3060003520 con 0x7f30600052e0 2024-05-09T10:50:50.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.788+0000 7f307abe3640 1 -- 172.21.15.98:0/533852141 reap_dead start 2024-05-09T10:50:50.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.789+0000 7f307abe3640 1 --2- 172.21.15.98:0/533852141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30600052e0 0x7f306000b9c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f306c001e20 tx=0x7f3060005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6959994c4d59e846 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:50.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.789+0000 7f30793e0640 1 -- 172.21.15.98:0/533852141 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c001a70 con 0x7f30600052e0 2024-05-09T10:50:50.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.789+0000 7f30793e0640 1 -- 172.21.15.98:0/533852141 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306c02ae40 con 0x7f30600052e0 2024-05-09T10:50:50.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.789+0000 7f30793e0640 1 -- 172.21.15.98:0/533852141 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c003bb0 con 0x7f30600052e0 2024-05-09T10:50:50.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.789+0000 7f307c646640 1 -- 172.21.15.98:0/533852141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30600052e0 msgr2=0x7f306000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:50.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.789+0000 7f307c646640 1 --2- 172.21.15.98:0/533852141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30600052e0 0x7f306000b9c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f306c001e20 tx=0x7f3060005830 comp rx=0 tx=0).stop 2024-05-09T10:50:50.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.789+0000 7f307c646640 1 -- 172.21.15.98:0/533852141 shutdown_connections 2024-05-09T10:50:50.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.789+0000 7f307c646640 1 --2- 172.21.15.98:0/533852141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30600052e0 0x7f306000b9c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.789+0000 7f307c646640 1 -- 172.21.15.98:0/533852141 >> 172.21.15.98:0/533852141 conn(0x7f307406cbf0 msgr2=0x7f307406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:50.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.789+0000 7f307c646640 1 -- 172.21.15.98:0/533852141 shutdown_connections 2024-05-09T10:50:50.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.789+0000 7f307c646640 1 -- 172.21.15.98:0/533852141 wait complete. 2024-05-09T10:50:50.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.791+0000 7f307c646640 1 Processor -- start 2024-05-09T10:50:50.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.791+0000 7f307c646640 1 -- start start 2024-05-09T10:50:50.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.791+0000 7f307c646640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30600052e0 0x7f30740c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:50.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.791+0000 7f307c646640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30740cbb50 0x7f30740c4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:50.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.791+0000 7f307c646640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3074070b20 0x7f3074070fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:50.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.791+0000 7f307c646640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30740ae630 con 0x7f3074070b20 2024-05-09T10:50:50.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.791+0000 7f307c646640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307415ed60 con 0x7f30600052e0 2024-05-09T10:50:50.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.791+0000 7f307c646640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30740adc60 con 0x7f30740cbb50 2024-05-09T10:50:50.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.791+0000 7f307abe3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3074070b20 0x7f3074070fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:50.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.791+0000 7f307abe3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3074070b20 0x7f3074070fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34380/0 (socket says 172.21.15.98:34380) 2024-05-09T10:50:50.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.791+0000 7f307abe3640 1 -- 172.21.15.98:0/1060003653 learned_addr learned my addr 172.21.15.98:0/1060003653 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:50.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.791+0000 7f307a3e2640 1 --2- 172.21.15.98:0/1060003653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30600052e0 0x7f30740c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:50.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.791+0000 7f3079be1640 1 --2- 172.21.15.98:0/1060003653 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30740cbb50 0x7f30740c4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:50.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.792+0000 7f307abe3640 1 -- 172.21.15.98:0/1060003653 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30740cbb50 msgr2=0x7f30740c4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:50.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.792+0000 7f307abe3640 1 --2- 172.21.15.98:0/1060003653 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30740cbb50 0x7f30740c4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.792+0000 7f307abe3640 1 -- 172.21.15.98:0/1060003653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30600052e0 msgr2=0x7f30740c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:50.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.792+0000 7f307abe3640 1 --2- 172.21.15.98:0/1060003653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30600052e0 0x7f30740c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.792+0000 7f307abe3640 1 -- 172.21.15.98:0/1060003653 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30740c5880 con 0x7f3074070b20 2024-05-09T10:50:50.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.792+0000 7f307abe3640 1 --2- 172.21.15.98:0/1060003653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3074070b20 0x7f3074070fd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f306c005040 tx=0x7f306c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:50.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.792+0000 7f306affd640 1 -- 172.21.15.98:0/1060003653 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c007db0 con 0x7f3074070b20 2024-05-09T10:50:50.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.792+0000 7f306affd640 1 -- 172.21.15.98:0/1060003653 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306c041070 con 0x7f3074070b20 2024-05-09T10:50:50.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.792+0000 7f307c646640 1 -- 172.21.15.98:0/1060003653 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30740c5b60 con 0x7f3074070b20 2024-05-09T10:50:50.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.792+0000 7f306affd640 1 -- 172.21.15.98:0/1060003653 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c001a70 con 0x7f3074070b20 2024-05-09T10:50:50.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.792+0000 7f307c646640 1 -- 172.21.15.98:0/1060003653 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30740c6810 con 0x7f3074070b20 2024-05-09T10:50:50.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.793+0000 7f307c646640 1 -- 172.21.15.98:0/1060003653 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30740af100 con 0x7f3074070b20 2024-05-09T10:50:50.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.793+0000 7f306affd640 1 -- 172.21.15.98:0/1060003653 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f306c032660 con 0x7f3074070b20 2024-05-09T10:50:50.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.794+0000 7f306affd640 1 --2- 172.21.15.98:0/1060003653 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f305806b4d0 0x7f305806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:50.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.794+0000 7f307a3e2640 1 --2- 172.21.15.98:0/1060003653 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f305806b4d0 0x7f305806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:50.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.794+0000 7f306affd640 1 -- 172.21.15.98:0/1060003653 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f306c0a9fd0 con 0x7f3074070b20 2024-05-09T10:50:50.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.794+0000 7f307a3e2640 1 --2- 172.21.15.98:0/1060003653 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f305806b4d0 0x7f305806d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3070001c80 tx=0x7f3070001c10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:50.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.798+0000 7f306affd640 1 -- 172.21.15.98:0/1060003653 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f30740af100 con 0x7f3074070b20 2024-05-09T10:50:50.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.967+0000 7f307c646640 1 -- 172.21.15.98:0/1060003653 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f307406dd40 con 0x7f3074070b20 2024-05-09T10:50:50.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.968+0000 7f306affd640 1 -- 172.21.15.98:0/1060003653 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f306c039070 con 0x7f3074070b20 2024-05-09T10:50:50.962 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-05-09T10:50:50.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.970+0000 7f307c646640 1 -- 172.21.15.98:0/1060003653 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f305806b4d0 msgr2=0x7f305806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:50.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.970+0000 7f307c646640 1 --2- 172.21.15.98:0/1060003653 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f305806b4d0 0x7f305806d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3070001c80 tx=0x7f3070001c10 comp rx=0 tx=0).stop 2024-05-09T10:50:50.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.970+0000 7f307c646640 1 -- 172.21.15.98:0/1060003653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3074070b20 msgr2=0x7f3074070fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:50.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.970+0000 7f307c646640 1 --2- 172.21.15.98:0/1060003653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3074070b20 0x7f3074070fd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f306c005040 tx=0x7f306c0014a0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.970+0000 7f307c646640 1 -- 172.21.15.98:0/1060003653 shutdown_connections 2024-05-09T10:50:50.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.970+0000 7f307c646640 1 --2- 172.21.15.98:0/1060003653 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f305806b4d0 0x7f305806d990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.970+0000 7f307c646640 1 --2- 172.21.15.98:0/1060003653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3074070b20 0x7f3074070fd0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.970+0000 7f307c646640 1 --2- 172.21.15.98:0/1060003653 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30740cbb50 0x7f30740c4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.970+0000 7f307c646640 1 --2- 172.21.15.98:0/1060003653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30600052e0 0x7f30740c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:50.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.970+0000 7f307c646640 1 -- 172.21.15.98:0/1060003653 >> 172.21.15.98:0/1060003653 conn(0x7f307406cbf0 msgr2=0x7f307415f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:50.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.970+0000 7f307c646640 1 -- 172.21.15.98:0/1060003653 shutdown_connections 2024-05-09T10:50:50.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:50.970+0000 7f307c646640 1 -- 172.21.15.98:0/1060003653 wait complete. 2024-05-09T10:50:50.976 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T10:50:51.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.084+0000 7fa1b12cb640 1 Processor -- start 2024-05-09T10:50:51.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.084+0000 7fa1b12cb640 1 -- start start 2024-05-09T10:50:51.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1b12cb640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1ac161450 0x7fa1ac0d4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:51.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1b12cb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa1ac0d2820 0x7fa1ac0b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:51.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1b12cb640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1ac161830 con 0x7fa1ac0b2a30 2024-05-09T10:50:51.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1b12cb640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1ac0b0760 con 0x7fa1ac0d2820 2024-05-09T10:50:51.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1b12cb640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac0cf2f0 con 0x7fa1ac161450 2024-05-09T10:50:51.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1a27fc640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa1ac0b2a30 0x7fa1ac0d2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41150/0 (socket says 172.21.15.98:41150) 2024-05-09T10:50:51.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1a27fc640 1 -- 172.21.15.98:0/1168844491 learned_addr learned my addr 172.21.15.98:0/1168844491 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:51.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1aa7fc640 1 -- 172.21.15.98:0/1168844491 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2745307093 0 0) 0x7fa194001110 con 0x7fa1ac0b2a30 2024-05-09T10:50:51.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1ab7fe640 1 --2- 172.21.15.98:0/1168844491 >> v2:172.21.15.105:3300/0 conn(0x7fa1ac0d2820 0x7fa1ac0b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:51.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1aaffd640 1 --2- 172.21.15.98:0/1168844491 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1ac161450 0x7fa1ac0d4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:51.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1aa7fc640 1 -- 172.21.15.98:0/1168844491 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa198003520 con 0x7fa1ac0b2a30 2024-05-09T10:50:51.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1aa7fc640 1 -- 172.21.15.98:0/1168844491 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2580435648 0 0) 0x7fa198003520 con 0x7fa1ac0b2a30 2024-05-09T10:50:51.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1aa7fc640 1 -- 172.21.15.98:0/1168844491 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1ac161450 msgr2=0x7fa1ac0d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:51.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1aa7fc640 1 --2- 172.21.15.98:0/1168844491 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1ac161450 0x7fa1ac0d4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1aa7fc640 1 -- 172.21.15.98:0/1168844491 >> v2:172.21.15.105:3300/0 conn(0x7fa1ac0d2820 msgr2=0x7fa1ac0b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:51.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1aa7fc640 1 --2- 172.21.15.98:0/1168844491 >> v2:172.21.15.105:3300/0 conn(0x7fa1ac0d2820 0x7fa1ac0b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1aa7fc640 1 -- 172.21.15.98:0/1168844491 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ac0cf5d0 con 0x7fa1ac0b2a30 2024-05-09T10:50:51.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1aa7fc640 1 -- 172.21.15.98:0/1168844491 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7fa194001840 con 0x7fa1ac0b2a30 2024-05-09T10:50:51.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1aa7fc640 1 -- 172.21.15.98:0/1168844491 >> v1:172.21.15.98:6789/0 conn(0x7fa1ac0b2a30 legacy=0x7fa1ac0d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:51.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1aa7fc640 1 --2- 172.21.15.98:0/1168844491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1980039b0 0x7fa198003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:51.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1aa7fc640 1 --2- 172.21.15.98:0/1168844491 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1ac161450 0x7fa198004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:51.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1aa7fc640 1 --2- 172.21.15.98:0/1168844491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa198004d40 0x7fa1980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:51.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1aa7fc640 1 -- 172.21.15.98:0/1168844491 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac0b0760 con 0x7fa1980039b0 2024-05-09T10:50:51.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1aa7fc640 1 -- 172.21.15.98:0/1168844491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac0cf2f0 con 0x7fa198004d40 2024-05-09T10:50:51.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1aa7fc640 1 -- 172.21.15.98:0/1168844491 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa19800b9c0 con 0x7fa1ac161450 2024-05-09T10:50:51.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.085+0000 7fa1aaffd640 1 --2- 172.21.15.98:0/1168844491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1980039b0 0x7fa198003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:51.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.086+0000 7fa1a27fc640 1 --2- 172.21.15.98:0/1168844491 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1ac161450 0x7fa198004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:51.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.086+0000 7fa1ab7fe640 1 --2- 172.21.15.98:0/1168844491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa198004d40 0x7fa1980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:51.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.086+0000 7fa1aaffd640 1 -- 172.21.15.98:0/1168844491 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1ac161450 msgr2=0x7fa198004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:51.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.086+0000 7fa1aaffd640 1 --2- 172.21.15.98:0/1168844491 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1ac161450 0x7fa198004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.086+0000 7fa1aaffd640 1 -- 172.21.15.98:0/1168844491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa198004d40 msgr2=0x7fa1980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:51.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.086+0000 7fa1aaffd640 1 --2- 172.21.15.98:0/1168844491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa198004d40 0x7fa1980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.086+0000 7fa1aaffd640 1 -- 172.21.15.98:0/1168844491 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa198003520 con 0x7fa1980039b0 2024-05-09T10:50:51.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.086+0000 7fa1a27fc640 1 --2- 172.21.15.98:0/1168844491 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1ac161450 0x7fa198004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:51.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.086+0000 7fa1aaffd640 1 --2- 172.21.15.98:0/1168844491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1980039b0 0x7fa198003dd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009ac0 tx=0x7fa198005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce42ca02ccb151ee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:51.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.086+0000 7fa1aa7fc640 1 -- 172.21.15.98:0/1168844491 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c036070 con 0x7fa1980039b0 2024-05-09T10:50:51.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.086+0000 7fa1aa7fc640 1 -- 172.21.15.98:0/1168844491 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19c031400 con 0x7fa1980039b0 2024-05-09T10:50:51.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.086+0000 7fa1aa7fc640 1 -- 172.21.15.98:0/1168844491 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c009690 con 0x7fa1980039b0 2024-05-09T10:50:51.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.087+0000 7fa1b12cb640 1 -- 172.21.15.98:0/1168844491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1980039b0 msgr2=0x7fa198003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:51.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.087+0000 7fa1b12cb640 1 --2- 172.21.15.98:0/1168844491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1980039b0 0x7fa198003dd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009ac0 tx=0x7fa198005830 comp rx=0 tx=0).stop 2024-05-09T10:50:51.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.087+0000 7fa1ab7fe640 1 -- 172.21.15.98:0/1168844491 reap_dead start 2024-05-09T10:50:51.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.087+0000 7fa1b12cb640 1 -- 172.21.15.98:0/1168844491 shutdown_connections 2024-05-09T10:50:51.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.087+0000 7fa1b12cb640 1 -- 172.21.15.98:0/1168844491 >> 172.21.15.98:0/1168844491 conn(0x7fa1ac0d3b30 msgr2=0x7fa1ac0d3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:51.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.087+0000 7fa1b12cb640 1 -- 172.21.15.98:0/1168844491 shutdown_connections 2024-05-09T10:50:51.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.087+0000 7fa1b12cb640 1 -- 172.21.15.98:0/1168844491 wait complete. 2024-05-09T10:50:51.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.088+0000 7fa1b12cb640 1 Processor -- start 2024-05-09T10:50:51.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.088+0000 7fa1b12cb640 1 -- start start 2024-05-09T10:50:51.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.089+0000 7fa1b12cb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1980039b0 0x7fa1ac0cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:51.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.089+0000 7fa1b12cb640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1ac159a30 0x7fa1ac0c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:51.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.089+0000 7fa1b12cb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1ac0cd890 0x7fa1ac0c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:51.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.089+0000 7fa1b12cb640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac0d4650 con 0x7fa1ac0cd890 2024-05-09T10:50:51.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.089+0000 7fa1b12cb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac0b09b0 con 0x7fa1980039b0 2024-05-09T10:50:51.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.089+0000 7fa1b12cb640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac0d47d0 con 0x7fa1ac159a30 2024-05-09T10:50:51.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.089+0000 7fa1ab7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1ac0cd890 0x7fa1ac0c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:51.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.089+0000 7fa1ab7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1ac0cd890 0x7fa1ac0c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34400/0 (socket says 172.21.15.98:34400) 2024-05-09T10:50:51.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.089+0000 7fa1ab7fe640 1 -- 172.21.15.98:0/3617033944 learned_addr learned my addr 172.21.15.98:0/3617033944 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:51.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.089+0000 7fa1aaffd640 1 --2- 172.21.15.98:0/3617033944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1980039b0 0x7fa1ac0cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:51.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.089+0000 7fa1a27fc640 1 --2- 172.21.15.98:0/3617033944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1ac159a30 0x7fa1ac0c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:51.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.090+0000 7fa1ab7fe640 1 -- 172.21.15.98:0/3617033944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1ac159a30 msgr2=0x7fa1ac0c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:51.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.090+0000 7fa1ab7fe640 1 --2- 172.21.15.98:0/3617033944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1ac159a30 0x7fa1ac0c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.090+0000 7fa1ab7fe640 1 -- 172.21.15.98:0/3617033944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1980039b0 msgr2=0x7fa1ac0cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:51.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.090+0000 7fa1ab7fe640 1 --2- 172.21.15.98:0/3617033944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1980039b0 0x7fa1ac0cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.090+0000 7fa1ab7fe640 1 -- 172.21.15.98:0/3617033944 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ac09dcc0 con 0x7fa1ac0cd890 2024-05-09T10:50:51.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.090+0000 7fa1ab7fe640 1 --2- 172.21.15.98:0/3617033944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1ac0cd890 0x7fa1ac0c9770 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac08ca40 tx=0x7fa1a40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:51.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.090+0000 7fa1a3fff640 1 -- 172.21.15.98:0/3617033944 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4011e40 con 0x7fa1ac0cd890 2024-05-09T10:50:51.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.090+0000 7fa1a3fff640 1 -- 172.21.15.98:0/3617033944 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a401b070 con 0x7fa1ac0cd890 2024-05-09T10:50:51.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.090+0000 7fa1b12cb640 1 -- 172.21.15.98:0/3617033944 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1ac0c8750 con 0x7fa1ac0cd890 2024-05-09T10:50:51.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.090+0000 7fa1a3fff640 1 -- 172.21.15.98:0/3617033944 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4003990 con 0x7fa1ac0cd890 2024-05-09T10:50:51.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.090+0000 7fa1b12cb640 1 -- 172.21.15.98:0/3617033944 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ac0c8d10 con 0x7fa1ac0cd890 2024-05-09T10:50:51.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.091+0000 7fa1b12cb640 1 -- 172.21.15.98:0/3617033944 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1ac0a1ae0 con 0x7fa1ac0cd890 2024-05-09T10:50:51.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.091+0000 7fa1a3fff640 1 -- 172.21.15.98:0/3617033944 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1a4005d10 con 0x7fa1ac0cd890 2024-05-09T10:50:51.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.092+0000 7fa1a3fff640 1 --2- 172.21.15.98:0/3617033944 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa18c06b4d0 0x7fa18c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:51.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.092+0000 7fa1aaffd640 1 --2- 172.21.15.98:0/3617033944 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa18c06b4d0 0x7fa18c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:51.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.092+0000 7fa1a3fff640 1 -- 172.21.15.98:0/3617033944 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1a408a5e0 con 0x7fa1ac0cd890 2024-05-09T10:50:51.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.092+0000 7fa1aaffd640 1 --2- 172.21.15.98:0/3617033944 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa18c06b4d0 0x7fa18c06d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa19c033000 tx=0x7fa19c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:51.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.096+0000 7fa1a3fff640 1 -- 172.21.15.98:0/3617033944 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa1a4025020 con 0x7fa1ac0cd890 2024-05-09T10:50:51.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.269+0000 7fa1b12cb640 1 -- 172.21.15.98:0/3617033944 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa1ac0b27f0 con 0x7fa1ac0cd890 2024-05-09T10:50:51.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.269+0000 7fa1a3fff640 1 -- 172.21.15.98:0/3617033944 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa1a4058e10 con 0x7fa1ac0cd890 2024-05-09T10:50:51.264 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2024-05-09T10:50:51.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.271+0000 7fa1b12cb640 1 -- 172.21.15.98:0/3617033944 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa18c06b4d0 msgr2=0x7fa18c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:51.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.271+0000 7fa1b12cb640 1 --2- 172.21.15.98:0/3617033944 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa18c06b4d0 0x7fa18c06d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa19c033000 tx=0x7fa19c008030 comp rx=0 tx=0).stop 2024-05-09T10:50:51.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.271+0000 7fa1b12cb640 1 -- 172.21.15.98:0/3617033944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1ac0cd890 msgr2=0x7fa1ac0c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:51.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.271+0000 7fa1b12cb640 1 --2- 172.21.15.98:0/3617033944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1ac0cd890 0x7fa1ac0c9770 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac08ca40 tx=0x7fa1a40115a0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.271+0000 7fa1b12cb640 1 -- 172.21.15.98:0/3617033944 shutdown_connections 2024-05-09T10:50:51.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.271+0000 7fa1b12cb640 1 --2- 172.21.15.98:0/3617033944 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa18c06b4d0 0x7fa18c06d990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.271+0000 7fa1b12cb640 1 --2- 172.21.15.98:0/3617033944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1ac0cd890 0x7fa1ac0c9770 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.271+0000 7fa1b12cb640 1 --2- 172.21.15.98:0/3617033944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa1ac159a30 0x7fa1ac0c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.271+0000 7fa1b12cb640 1 --2- 172.21.15.98:0/3617033944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1980039b0 0x7fa1ac0cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.271+0000 7fa1b12cb640 1 -- 172.21.15.98:0/3617033944 >> 172.21.15.98:0/3617033944 conn(0x7fa1ac0d3b30 msgr2=0x7fa1ac15b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:51.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.272+0000 7fa1b12cb640 1 -- 172.21.15.98:0/3617033944 shutdown_connections 2024-05-09T10:50:51.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.272+0000 7fa1b12cb640 1 -- 172.21.15.98:0/3617033944 wait complete. 2024-05-09T10:50:51.277 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T10:50:51.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.386+0000 7fae75dab640 1 Processor -- start 2024-05-09T10:50:51.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.386+0000 7fae75dab640 1 -- start start 2024-05-09T10:50:51.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.387+0000 7fae75dab640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae700b2a00 0x7fae700d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:51.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.387+0000 7fae75dab640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae700cf300 con 0x7fae700d0ac0 2024-05-09T10:50:51.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.387+0000 7fae75dab640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae700cf5e0 con 0x7fae7015de50 2024-05-09T10:50:51.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.387+0000 7fae75dab640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae700b34a0 con 0x7fae700b2a00 2024-05-09T10:50:51.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.387+0000 7fae6ffff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fae700d0ac0 0x7fae700b3930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41158/0 (socket says 172.21.15.98:41158) 2024-05-09T10:50:51.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.387+0000 7fae6ffff640 1 -- 172.21.15.98:0/3794780706 learned_addr learned my addr 172.21.15.98:0/3794780706 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:51.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.387+0000 7fae6effd640 1 --2- 172.21.15.98:0/3794780706 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae700b2a00 0x7fae700d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:51.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.387+0000 7fae6e7fc640 1 -- 172.21.15.98:0/3794780706 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167595630 0 0) 0x7fae700cf300 con 0x7fae700d0ac0 2024-05-09T10:50:51.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.387+0000 7fae6e7fc640 1 -- 172.21.15.98:0/3794780706 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae58003520 con 0x7fae700d0ac0 2024-05-09T10:50:51.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.387+0000 7fae6e7fc640 1 -- 172.21.15.98:0/3794780706 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2371301982 0 0) 0x7fae58003520 con 0x7fae700d0ac0 2024-05-09T10:50:51.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.387+0000 7fae6e7fc640 1 -- 172.21.15.98:0/3794780706 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae700b2a00 msgr2=0x7fae700d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:51.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.387+0000 7fae6e7fc640 1 --2- 172.21.15.98:0/3794780706 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae700b2a00 0x7fae700d0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.387+0000 7fae6e7fc640 1 -- 172.21.15.98:0/3794780706 >> v1:172.21.15.105:6789/0 conn(0x7fae7015de50 legacy=0x7fae700d42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:51.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.387+0000 7fae6e7fc640 1 -- 172.21.15.98:0/3794780706 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae700cf8c0 con 0x7fae700d0ac0 2024-05-09T10:50:51.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.387+0000 7fae6e7fc640 1 -- 172.21.15.98:0/3794780706 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7fae68001670 con 0x7fae700d0ac0 2024-05-09T10:50:51.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.387+0000 7fae6e7fc640 1 -- 172.21.15.98:0/3794780706 >> v1:172.21.15.98:6789/0 conn(0x7fae700d0ac0 legacy=0x7fae700b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:51.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.387+0000 7fae6e7fc640 1 --2- 172.21.15.98:0/3794780706 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae700b2a00 0x7fae580039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.388+0000 7fae6e7fc640 1 --2- 172.21.15.98:0/3794780706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae58003f20 0x7fae58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.388+0000 7fae6e7fc640 1 --2- 172.21.15.98:0/3794780706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae58004d40 0x7fae580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.388+0000 7fae6e7fc640 1 -- 172.21.15.98:0/3794780706 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae700b34a0 con 0x7fae58003f20 2024-05-09T10:50:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.388+0000 7fae6e7fc640 1 -- 172.21.15.98:0/3794780706 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5800bad0 con 0x7fae58004d40 2024-05-09T10:50:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.388+0000 7fae6e7fc640 1 -- 172.21.15.98:0/3794780706 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5800bcd0 con 0x7fae700b2a00 2024-05-09T10:50:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.388+0000 7fae6effd640 1 --2- 172.21.15.98:0/3794780706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae58003f20 0x7fae58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.388+0000 7fae6f7fe640 1 --2- 172.21.15.98:0/3794780706 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae700b2a00 0x7fae580039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.388+0000 7fae6ffff640 1 --2- 172.21.15.98:0/3794780706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae58004d40 0x7fae580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.388+0000 7fae6effd640 1 -- 172.21.15.98:0/3794780706 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae700b2a00 msgr2=0x7fae580039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.388+0000 7fae6effd640 1 --2- 172.21.15.98:0/3794780706 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae700b2a00 0x7fae580039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.388+0000 7fae6effd640 1 -- 172.21.15.98:0/3794780706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae58004d40 msgr2=0x7fae580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.388+0000 7fae6effd640 1 --2- 172.21.15.98:0/3794780706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae58004d40 0x7fae580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.388+0000 7fae6effd640 1 -- 172.21.15.98:0/3794780706 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae58003520 con 0x7fae58003f20 2024-05-09T10:50:51.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.389+0000 7fae6effd640 1 --2- 172.21.15.98:0/3794780706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae58003f20 0x7fae58004800 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fae64009ac0 tx=0x7fae58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb04be0c1cde75f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:51.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.389+0000 7fae6e7fc640 1 -- 172.21.15.98:0/3794780706 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae64036070 con 0x7fae58003f20 2024-05-09T10:50:51.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.389+0000 7fae6e7fc640 1 -- 172.21.15.98:0/3794780706 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae64031400 con 0x7fae58003f20 2024-05-09T10:50:51.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.389+0000 7fae6e7fc640 1 -- 172.21.15.98:0/3794780706 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae64009690 con 0x7fae58003f20 2024-05-09T10:50:51.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.389+0000 7fae75dab640 1 -- 172.21.15.98:0/3794780706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae58003f20 msgr2=0x7fae58004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:51.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.389+0000 7fae75dab640 1 --2- 172.21.15.98:0/3794780706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae58003f20 0x7fae58004800 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fae64009ac0 tx=0x7fae58005830 comp rx=0 tx=0).stop 2024-05-09T10:50:51.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.389+0000 7fae6ffff640 1 -- 172.21.15.98:0/3794780706 reap_dead start 2024-05-09T10:50:51.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.389+0000 7fae75dab640 1 -- 172.21.15.98:0/3794780706 shutdown_connections 2024-05-09T10:50:51.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.389+0000 7fae75dab640 1 -- 172.21.15.98:0/3794780706 >> 172.21.15.98:0/3794780706 conn(0x7fae7009cef0 msgr2=0x7fae700d3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:51.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.389+0000 7fae75dab640 1 -- 172.21.15.98:0/3794780706 shutdown_connections 2024-05-09T10:50:51.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.389+0000 7fae75dab640 1 -- 172.21.15.98:0/3794780706 wait complete. 2024-05-09T10:50:51.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.391+0000 7fae75dab640 1 Processor -- start 2024-05-09T10:50:51.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.391+0000 7fae75dab640 1 -- start start 2024-05-09T10:50:51.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.391+0000 7fae75dab640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae58003f20 0x7fae70147d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:51.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.391+0000 7fae75dab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae7015a320 0x7fae70157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:51.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.392+0000 7fae75dab640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae701f6f90 0x7fae701f73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:51.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.392+0000 7fae75dab640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae700d0700 con 0x7fae58003f20 2024-05-09T10:50:51.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.392+0000 7fae75dab640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae700d0580 con 0x7fae7015a320 2024-05-09T10:50:51.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.392+0000 7fae75dab640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae700d0880 con 0x7fae701f6f90 2024-05-09T10:50:51.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.392+0000 7fae6f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae58003f20 0x7fae70147d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:51.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.392+0000 7fae6f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae58003f20 0x7fae70147d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34420/0 (socket says 172.21.15.98:34420) 2024-05-09T10:50:51.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.392+0000 7fae6f7fe640 1 -- 172.21.15.98:0/3441545264 learned_addr learned my addr 172.21.15.98:0/3441545264 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:51.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.392+0000 7fae6effd640 1 --2- 172.21.15.98:0/3441545264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae7015a320 0x7fae70157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:51.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.392+0000 7fae6ffff640 1 --2- 172.21.15.98:0/3441545264 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae701f6f90 0x7fae701f73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:51.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.392+0000 7fae6f7fe640 1 -- 172.21.15.98:0/3441545264 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae701f6f90 msgr2=0x7fae701f73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:51.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.392+0000 7fae6f7fe640 1 --2- 172.21.15.98:0/3441545264 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae701f6f90 0x7fae701f73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.392+0000 7fae6f7fe640 1 -- 172.21.15.98:0/3441545264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae7015a320 msgr2=0x7fae70157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:51.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.392+0000 7fae6f7fe640 1 --2- 172.21.15.98:0/3441545264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae7015a320 0x7fae70157eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.392+0000 7fae6f7fe640 1 -- 172.21.15.98:0/3441545264 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7015a780 con 0x7fae58003f20 2024-05-09T10:50:51.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.392+0000 7fae6ffff640 1 --2- 172.21.15.98:0/3441545264 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae701f6f90 0x7fae701f73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:51.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.392+0000 7fae6f7fe640 1 --2- 172.21.15.98:0/3441545264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae58003f20 0x7fae70147d20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fae6000fb60 tx=0x7fae6000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:51.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.392+0000 7fae57fff640 1 -- 172.21.15.98:0/3441545264 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6000dc30 con 0x7fae58003f20 2024-05-09T10:50:51.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.392+0000 7fae57fff640 1 -- 172.21.15.98:0/3441545264 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae60018070 con 0x7fae58003f20 2024-05-09T10:50:51.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.392+0000 7fae75dab640 1 -- 172.21.15.98:0/3441545264 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7015aa30 con 0x7fae58003f20 2024-05-09T10:50:51.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.392+0000 7fae57fff640 1 -- 172.21.15.98:0/3441545264 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae60003970 con 0x7fae58003f20 2024-05-09T10:50:51.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.392+0000 7fae75dab640 1 -- 172.21.15.98:0/3441545264 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae700b7bb0 con 0x7fae58003f20 2024-05-09T10:50:51.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.393+0000 7fae75dab640 1 -- 172.21.15.98:0/3441545264 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae700a1b60 con 0x7fae58003f20 2024-05-09T10:50:51.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.394+0000 7fae57fff640 1 -- 172.21.15.98:0/3441545264 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae60003b30 con 0x7fae58003f20 2024-05-09T10:50:51.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.394+0000 7fae57fff640 1 --2- 172.21.15.98:0/3441545264 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fae5c06b500 0x7fae5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:51.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.394+0000 7fae57fff640 1 -- 172.21.15.98:0/3441545264 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae6008a520 con 0x7fae58003f20 2024-05-09T10:50:51.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.394+0000 7fae6effd640 1 --2- 172.21.15.98:0/3441545264 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fae5c06b500 0x7fae5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:51.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.394+0000 7fae6effd640 1 --2- 172.21.15.98:0/3441545264 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fae5c06b500 0x7fae5c06d9c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fae64033000 tx=0x7fae64008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:51.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.398+0000 7fae57fff640 1 -- 172.21.15.98:0/3441545264 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae60003e20 con 0x7fae58003f20 2024-05-09T10:50:51.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.569+0000 7fae75dab640 1 -- 172.21.15.98:0/3441545264 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fae7004f000 con 0x7fae58003f20 2024-05-09T10:50:51.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.570+0000 7fae57fff640 1 -- 172.21.15.98:0/3441545264 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fae600593a0 con 0x7fae58003f20 2024-05-09T10:50:51.564 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2024-05-09T10:50:51.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.572+0000 7fae75dab640 1 -- 172.21.15.98:0/3441545264 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fae5c06b500 msgr2=0x7fae5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:51.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.572+0000 7fae75dab640 1 --2- 172.21.15.98:0/3441545264 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fae5c06b500 0x7fae5c06d9c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fae64033000 tx=0x7fae64008030 comp rx=0 tx=0).stop 2024-05-09T10:50:51.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.572+0000 7fae75dab640 1 -- 172.21.15.98:0/3441545264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae58003f20 msgr2=0x7fae70147d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:51.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.572+0000 7fae75dab640 1 --2- 172.21.15.98:0/3441545264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae58003f20 0x7fae70147d20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fae6000fb60 tx=0x7fae6000d520 comp rx=0 tx=0).stop 2024-05-09T10:50:51.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.572+0000 7fae75dab640 1 -- 172.21.15.98:0/3441545264 shutdown_connections 2024-05-09T10:50:51.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.572+0000 7fae75dab640 1 --2- 172.21.15.98:0/3441545264 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fae5c06b500 0x7fae5c06d9c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.572+0000 7fae75dab640 1 --2- 172.21.15.98:0/3441545264 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae701f6f90 0x7fae701f73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.572+0000 7fae75dab640 1 --2- 172.21.15.98:0/3441545264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae7015a320 0x7fae70157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.572+0000 7fae75dab640 1 --2- 172.21.15.98:0/3441545264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae58003f20 0x7fae70147d20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.572+0000 7fae75dab640 1 -- 172.21.15.98:0/3441545264 >> 172.21.15.98:0/3441545264 conn(0x7fae7009cef0 msgr2=0x7fae7006c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:51.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.572+0000 7fae75dab640 1 -- 172.21.15.98:0/3441545264 shutdown_connections 2024-05-09T10:50:51.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.572+0000 7fae75dab640 1 -- 172.21.15.98:0/3441545264 wait complete. 2024-05-09T10:50:51.577 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T10:50:51.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.682+0000 7fb73c81f640 1 Processor -- start 2024-05-09T10:50:51.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.682+0000 7fb73c81f640 1 -- start start 2024-05-09T10:50:51.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.683+0000 7fb73c81f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb73815f1c0 0x7fb73806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:51.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.683+0000 7fb73c81f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb7380cf430 0x7fb73806bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:51.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.683+0000 7fb73c81f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb73815fb70 con 0x7fb73806c130 2024-05-09T10:50:51.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.683+0000 7fb73c81f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb73806c5a0 con 0x7fb73815f1c0 2024-05-09T10:50:51.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.683+0000 7fb73c81f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb738160930 con 0x7fb7380cf430 2024-05-09T10:50:51.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.683+0000 7fb73659c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb73815f1c0 0x7fb73806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:51.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.683+0000 7fb73659c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb73815f1c0 0x7fb73806edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34428/0 (socket says 172.21.15.98:34428) 2024-05-09T10:50:51.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.683+0000 7fb73659c640 1 -- 172.21.15.98:0/2063903143 learned_addr learned my addr 172.21.15.98:0/2063903143 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:51.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.683+0000 7fb73659c640 1 -- 172.21.15.98:0/2063903143 >> v2:172.21.15.105:3300/0 conn(0x7fb7380cf430 msgr2=0x7fb73806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:51.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.683+0000 7fb735d9b640 1 --2- 172.21.15.98:0/2063903143 >> v2:172.21.15.105:3300/0 conn(0x7fb7380cf430 0x7fb73806bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:51.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.683+0000 7fb73659c640 1 --2- 172.21.15.98:0/2063903143 >> v2:172.21.15.105:3300/0 conn(0x7fb7380cf430 0x7fb73806bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.683+0000 7fb73659c640 1 -- 172.21.15.98:0/2063903143 >> v1:172.21.15.105:6789/0 conn(0x7fb73806c130 legacy=0x7fb7380716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:51.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.683+0000 7fb73659c640 1 -- 172.21.15.98:0/2063903143 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb73815fe50 con 0x7fb73815f1c0 2024-05-09T10:50:51.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.683+0000 7fb735d9b640 1 --2- 172.21.15.98:0/2063903143 >> v2:172.21.15.105:3300/0 conn(0x7fb7380cf430 0x7fb73806bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:51.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.684+0000 7fb73659c640 1 --2- 172.21.15.98:0/2063903143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb73815f1c0 0x7fb73806edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb7380684e0 tx=0x7fb72802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da65815f1297ae5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:51.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.684+0000 7fb73559a640 1 -- 172.21.15.98:0/2063903143 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb728036070 con 0x7fb73815f1c0 2024-05-09T10:50:51.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.684+0000 7fb73559a640 1 -- 172.21.15.98:0/2063903143 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb728031600 con 0x7fb73815f1c0 2024-05-09T10:50:51.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.684+0000 7fb73559a640 1 -- 172.21.15.98:0/2063903143 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7280319d0 con 0x7fb73815f1c0 2024-05-09T10:50:51.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.685+0000 7fb73c81f640 1 -- 172.21.15.98:0/2063903143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb73815f1c0 msgr2=0x7fb73806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:51.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.685+0000 7fb73c81f640 1 --2- 172.21.15.98:0/2063903143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb73815f1c0 0x7fb73806edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb7380684e0 tx=0x7fb72802a6f0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.685+0000 7fb73c81f640 1 -- 172.21.15.98:0/2063903143 shutdown_connections 2024-05-09T10:50:51.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.685+0000 7fb73c81f640 1 --2- 172.21.15.98:0/2063903143 >> v2:172.21.15.105:3300/0 conn(0x7fb7380cf430 0x7fb73806bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.685+0000 7fb73c81f640 1 --2- 172.21.15.98:0/2063903143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb73815f1c0 0x7fb73806edb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.685+0000 7fb73c81f640 1 -- 172.21.15.98:0/2063903143 >> 172.21.15.98:0/2063903143 conn(0x7fb7380ccaa0 msgr2=0x7fb7380cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:51.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.685+0000 7fb73c81f640 1 -- 172.21.15.98:0/2063903143 shutdown_connections 2024-05-09T10:50:51.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.685+0000 7fb73c81f640 1 -- 172.21.15.98:0/2063903143 wait complete. 2024-05-09T10:50:51.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.686+0000 7fb73c81f640 1 Processor -- start 2024-05-09T10:50:51.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.686+0000 7fb73c81f640 1 -- start start 2024-05-09T10:50:51.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.687+0000 7fb73c81f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb73806c130 0x7fb73809adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:51.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.687+0000 7fb73c81f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7380cf430 0x7fb73809b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:51.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.687+0000 7fb73c81f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb73815f1c0 0x7fb7381f2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:51.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.687+0000 7fb73c81f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73806f2f0 con 0x7fb73806c130 2024-05-09T10:50:51.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.687+0000 7fb73c81f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb738160410 con 0x7fb7380cf430 2024-05-09T10:50:51.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.687+0000 7fb73c81f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb73815f5a0 con 0x7fb73815f1c0 2024-05-09T10:50:51.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.687+0000 7fb73659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb73806c130 0x7fb73809adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:51.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.687+0000 7fb73659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb73806c130 0x7fb73809adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34430/0 (socket says 172.21.15.98:34430) 2024-05-09T10:50:51.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.687+0000 7fb73659c640 1 -- 172.21.15.98:0/25290710 learned_addr learned my addr 172.21.15.98:0/25290710 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:51.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.687+0000 7fb73659c640 1 -- 172.21.15.98:0/25290710 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb73815f1c0 msgr2=0x7fb7381f2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:51.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.687+0000 7fb73659c640 1 --2- 172.21.15.98:0/25290710 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb73815f1c0 0x7fb7381f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.687+0000 7fb73659c640 1 -- 172.21.15.98:0/25290710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7380cf430 msgr2=0x7fb73809b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:51.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.687+0000 7fb73659c640 1 --2- 172.21.15.98:0/25290710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7380cf430 0x7fb73809b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.687+0000 7fb73659c640 1 -- 172.21.15.98:0/25290710 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7380b19a0 con 0x7fb73806c130 2024-05-09T10:50:51.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.688+0000 7fb73659c640 1 --2- 172.21.15.98:0/25290710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb73806c130 0x7fb73809adb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb728000f80 tx=0x7fb728004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:51.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.688+0000 7fb726ffd640 1 -- 172.21.15.98:0/25290710 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb728036050 con 0x7fb73806c130 2024-05-09T10:50:51.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.688+0000 7fb726ffd640 1 -- 172.21.15.98:0/25290710 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb728042070 con 0x7fb73806c130 2024-05-09T10:50:51.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.688+0000 7fb73c81f640 1 -- 172.21.15.98:0/25290710 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb73809bb70 con 0x7fb73806c130 2024-05-09T10:50:51.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.688+0000 7fb726ffd640 1 -- 172.21.15.98:0/25290710 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb728009510 con 0x7fb73806c130 2024-05-09T10:50:51.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.688+0000 7fb73c81f640 1 -- 172.21.15.98:0/25290710 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb73809c130 con 0x7fb73806c130 2024-05-09T10:50:51.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.689+0000 7fb73c81f640 1 -- 172.21.15.98:0/25290710 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb738069cb0 con 0x7fb73806c130 2024-05-09T10:50:51.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.689+0000 7fb726ffd640 1 -- 172.21.15.98:0/25290710 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb728007c70 con 0x7fb73806c130 2024-05-09T10:50:51.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.690+0000 7fb726ffd640 1 --2- 172.21.15.98:0/25290710 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb71806b4d0 0x7fb71806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:51.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.690+0000 7fb726ffd640 1 -- 172.21.15.98:0/25290710 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7280ab130 con 0x7fb73806c130 2024-05-09T10:50:51.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.690+0000 7fb735d9b640 1 --2- 172.21.15.98:0/25290710 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb71806b4d0 0x7fb71806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:51.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.690+0000 7fb735d9b640 1 --2- 172.21.15.98:0/25290710 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb71806b4d0 0x7fb71806d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb72c006fd0 tx=0x7fb72c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:51.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.694+0000 7fb726ffd640 1 -- 172.21.15.98:0/25290710 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb72807a040 con 0x7fb73806c130 2024-05-09T10:50:51.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.876+0000 7fb73c81f640 1 -- 172.21.15.98:0/25290710 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb7380c34a0 con 0x7fb73806c130 2024-05-09T10:50:51.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.876+0000 7fb726ffd640 1 -- 172.21.15.98:0/25290710 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb728079940 con 0x7fb73806c130 2024-05-09T10:50:51.871 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-05-09T10:50:51.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.878+0000 7fb73c81f640 1 -- 172.21.15.98:0/25290710 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb71806b4d0 msgr2=0x7fb71806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:51.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.878+0000 7fb73c81f640 1 --2- 172.21.15.98:0/25290710 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb71806b4d0 0x7fb71806d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb72c006fd0 tx=0x7fb72c008040 comp rx=0 tx=0).stop 2024-05-09T10:50:51.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.878+0000 7fb73c81f640 1 -- 172.21.15.98:0/25290710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb73806c130 msgr2=0x7fb73809adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:51.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.878+0000 7fb73c81f640 1 --2- 172.21.15.98:0/25290710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb73806c130 0x7fb73809adb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb728000f80 tx=0x7fb728004a40 comp rx=0 tx=0).stop 2024-05-09T10:50:51.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.878+0000 7fb73c81f640 1 -- 172.21.15.98:0/25290710 shutdown_connections 2024-05-09T10:50:51.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.878+0000 7fb73c81f640 1 --2- 172.21.15.98:0/25290710 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb71806b4d0 0x7fb71806d990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.878+0000 7fb73c81f640 1 --2- 172.21.15.98:0/25290710 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb73815f1c0 0x7fb7381f2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.878+0000 7fb73c81f640 1 --2- 172.21.15.98:0/25290710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7380cf430 0x7fb73809b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.878+0000 7fb73c81f640 1 --2- 172.21.15.98:0/25290710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb73806c130 0x7fb73809adb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:51.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.878+0000 7fb73c81f640 1 -- 172.21.15.98:0/25290710 >> 172.21.15.98:0/25290710 conn(0x7fb7380ccaa0 msgr2=0x7fb7380c5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:51.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.878+0000 7fb73c81f640 1 -- 172.21.15.98:0/25290710 shutdown_connections 2024-05-09T10:50:51.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:51.878+0000 7fb73c81f640 1 -- 172.21.15.98:0/25290710 wait complete. 2024-05-09T10:50:51.884 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-05-09T10:50:51.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:51.884 DEBUG:teuthology.orchestra.run.smithi098:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-05-09T10:50:51.909 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-05-09T10:50:51.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:51.910 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-09T10:50:51.966 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:50:51.966 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:50:51.985 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:50:51.986 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:50:52.044 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:50:52.044 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:50:52.102 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:52.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.211+0000 7f3eb386b640 1 Processor -- start 2024-05-09T10:50:52.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.211+0000 7f3eb386b640 1 -- start start 2024-05-09T10:50:52.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.212+0000 7f3eb386b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3eac15db90 0x7f3eac06d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:52.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.212+0000 7f3eb386b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3eac15df70 0x7f3eac0713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:52.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.212+0000 7f3eb386b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3eac071910 0x7f3eac06abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:52.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.212+0000 7f3eb386b640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eac0cdb30 con 0x7f3eac15df70 2024-05-09T10:50:52.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.212+0000 7f3eb386b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eac071f40 con 0x7f3eac071910 2024-05-09T10:50:52.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.212+0000 7f3eb386b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac072220 con 0x7f3eac15db90 2024-05-09T10:50:52.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.212+0000 7f3eb0e06640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3eac15df70 0x7f3eac0713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:52.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.212+0000 7f3eb0e06640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3eac15df70 0x7f3eac0713d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34436/0 (socket says 172.21.15.98:34436) 2024-05-09T10:50:52.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.212+0000 7f3eb0e06640 1 -- 172.21.15.98:0/4104084042 learned_addr learned my addr 172.21.15.98:0/4104084042 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:52.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.213+0000 7f3eb1607640 1 --2- 172.21.15.98:0/4104084042 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3eac15db90 0x7f3eac06d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:52.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.213+0000 7f3eb0e06640 1 -- 172.21.15.98:0/4104084042 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3eac15db90 msgr2=0x7f3eac06d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:52.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.213+0000 7f3eb1e08640 1 --2- 172.21.15.98:0/4104084042 >> v2:172.21.15.105:3300/0 conn(0x7f3eac071910 0x7f3eac06abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:52.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.213+0000 7f3eb0e06640 1 --2- 172.21.15.98:0/4104084042 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3eac15db90 0x7f3eac06d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.213+0000 7f3eb0e06640 1 -- 172.21.15.98:0/4104084042 >> v2:172.21.15.105:3300/0 conn(0x7f3eac071910 msgr2=0x7f3eac06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:52.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.213+0000 7f3eb0e06640 1 --2- 172.21.15.98:0/4104084042 >> v2:172.21.15.105:3300/0 conn(0x7f3eac071910 0x7f3eac06abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.213+0000 7f3eb0e06640 1 -- 172.21.15.98:0/4104084042 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac072500 con 0x7f3eac15df70 2024-05-09T10:50:52.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.213+0000 7f3eb0e06640 1 --2- 172.21.15.98:0/4104084042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3eac15df70 0x7f3eac0713d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3eac162a60 tx=0x7f3ea002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b416dbebc1dca5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:52.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.214+0000 7f3eabfff640 1 -- 172.21.15.98:0/4104084042 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0037070 con 0x7f3eac15df70 2024-05-09T10:50:52.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.214+0000 7f3eabfff640 1 -- 172.21.15.98:0/4104084042 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea0032440 con 0x7f3eac15df70 2024-05-09T10:50:52.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.214+0000 7f3eabfff640 1 -- 172.21.15.98:0/4104084042 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0032810 con 0x7f3eac15df70 2024-05-09T10:50:52.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.214+0000 7f3eb386b640 1 -- 172.21.15.98:0/4104084042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3eac15df70 msgr2=0x7f3eac0713d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:52.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.214+0000 7f3eb386b640 1 --2- 172.21.15.98:0/4104084042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3eac15df70 0x7f3eac0713d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3eac162a60 tx=0x7f3ea002a140 comp rx=0 tx=0).stop 2024-05-09T10:50:52.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.214+0000 7f3eb386b640 1 -- 172.21.15.98:0/4104084042 shutdown_connections 2024-05-09T10:50:52.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.214+0000 7f3eb386b640 1 --2- 172.21.15.98:0/4104084042 >> v2:172.21.15.105:3300/0 conn(0x7f3eac071910 0x7f3eac06abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.214+0000 7f3eb386b640 1 --2- 172.21.15.98:0/4104084042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3eac15df70 0x7f3eac0713d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.214+0000 7f3eb386b640 1 --2- 172.21.15.98:0/4104084042 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3eac15db90 0x7f3eac06d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.214+0000 7f3eb386b640 1 -- 172.21.15.98:0/4104084042 >> 172.21.15.98:0/4104084042 conn(0x7f3eac066f20 msgr2=0x7f3eac06bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:52.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.214+0000 7f3eb386b640 1 -- 172.21.15.98:0/4104084042 shutdown_connections 2024-05-09T10:50:52.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.214+0000 7f3eb386b640 1 -- 172.21.15.98:0/4104084042 wait complete. 2024-05-09T10:50:52.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.215+0000 7f3eb386b640 1 Processor -- start 2024-05-09T10:50:52.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.215+0000 7f3eb386b640 1 -- start start 2024-05-09T10:50:52.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.216+0000 7f3eb386b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eac071910 0x7f3eac156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:52.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.216+0000 7f3eb386b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3eac15db90 0x7f3eac1e6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:52.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.216+0000 7f3eb386b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3eac15df70 0x7f3eac0ad980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:52.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.216+0000 7f3eb386b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac0cefd0 con 0x7f3eac15df70 2024-05-09T10:50:52.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.216+0000 7f3eb386b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac06d970 con 0x7f3eac071910 2024-05-09T10:50:52.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.216+0000 7f3eb386b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac06b120 con 0x7f3eac15db90 2024-05-09T10:50:52.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.216+0000 7f3eb1e08640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3eac15df70 0x7f3eac0ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:52.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.216+0000 7f3eb1e08640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3eac15df70 0x7f3eac0ad980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34442/0 (socket says 172.21.15.98:34442) 2024-05-09T10:50:52.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.216+0000 7f3eb1e08640 1 -- 172.21.15.98:0/1677655308 learned_addr learned my addr 172.21.15.98:0/1677655308 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:52.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.216+0000 7f3eb1e08640 1 -- 172.21.15.98:0/1677655308 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3eac15db90 msgr2=0x7f3eac1e6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:52.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.216+0000 7f3eb1e08640 1 --2- 172.21.15.98:0/1677655308 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3eac15db90 0x7f3eac1e6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.216+0000 7f3eb1e08640 1 -- 172.21.15.98:0/1677655308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eac071910 msgr2=0x7f3eac156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:52.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.216+0000 7f3eb1e08640 1 --2- 172.21.15.98:0/1677655308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eac071910 0x7f3eac156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.216+0000 7f3eb1e08640 1 -- 172.21.15.98:0/1677655308 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac1e70c0 con 0x7f3eac15df70 2024-05-09T10:50:52.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.217+0000 7f3eb1e08640 1 --2- 172.21.15.98:0/1677655308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3eac15df70 0x7f3eac0ad980 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400c570 tx=0x7f3ea400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:52.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.217+0000 7f3ea9ffb640 1 -- 172.21.15.98:0/1677655308 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4015070 con 0x7f3eac15df70 2024-05-09T10:50:52.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.217+0000 7f3ea9ffb640 1 -- 172.21.15.98:0/1677655308 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea4004220 con 0x7f3eac15df70 2024-05-09T10:50:52.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.217+0000 7f3ea9ffb640 1 -- 172.21.15.98:0/1677655308 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4003610 con 0x7f3eac15df70 2024-05-09T10:50:52.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.217+0000 7f3eb386b640 1 -- 172.21.15.98:0/1677655308 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eac1e7840 con 0x7f3eac15df70 2024-05-09T10:50:52.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.217+0000 7f3eb386b640 1 -- 172.21.15.98:0/1677655308 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eac1e7e00 con 0x7f3eac15df70 2024-05-09T10:50:52.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.218+0000 7f3eb386b640 1 -- 172.21.15.98:0/1677655308 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eac069cb0 con 0x7f3eac15df70 2024-05-09T10:50:52.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.218+0000 7f3ea9ffb640 1 -- 172.21.15.98:0/1677655308 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ea40037b0 con 0x7f3eac15df70 2024-05-09T10:50:52.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.219+0000 7f3ea9ffb640 1 --2- 172.21.15.98:0/1677655308 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3e9806b4d0 0x7f3e9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:52.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.219+0000 7f3ea9ffb640 1 -- 172.21.15.98:0/1677655308 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ea4089eb0 con 0x7f3eac15df70 2024-05-09T10:50:52.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.219+0000 7f3eb1607640 1 --2- 172.21.15.98:0/1677655308 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3e9806b4d0 0x7f3e9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:52.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.219+0000 7f3eb1607640 1 --2- 172.21.15.98:0/1677655308 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3e9806b4d0 0x7f3e9806d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3eac1e8c40 tx=0x7f3e9c007390 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:52.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.223+0000 7f3ea9ffb640 1 -- 172.21.15.98:0/1677655308 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ea4058dc0 con 0x7f3eac15df70 2024-05-09T10:50:52.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.396+0000 7f3eb386b640 1 -- 172.21.15.98:0/1677655308 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3eac0b1ef0 con 0x7f3eac15df70 2024-05-09T10:50:52.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.397+0000 7f3ea9ffb640 1 -- 172.21.15.98:0/1677655308 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f3ea40586c0 con 0x7f3eac15df70 2024-05-09T10:50:52.392 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:52.392 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":20,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:21.176419+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:50:17.323444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T10:50:18.482315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:52.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.399+0000 7f3eb386b640 1 -- 172.21.15.98:0/1677655308 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3e9806b4d0 msgr2=0x7f3e9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:52.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.399+0000 7f3eb386b640 1 --2- 172.21.15.98:0/1677655308 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3e9806b4d0 0x7f3e9806d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3eac1e8c40 tx=0x7f3e9c007390 comp rx=0 tx=0).stop 2024-05-09T10:50:52.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.399+0000 7f3eb386b640 1 -- 172.21.15.98:0/1677655308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3eac15df70 msgr2=0x7f3eac0ad980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:52.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.399+0000 7f3eb386b640 1 --2- 172.21.15.98:0/1677655308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3eac15df70 0x7f3eac0ad980 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400c570 tx=0x7f3ea400ca30 comp rx=0 tx=0).stop 2024-05-09T10:50:52.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.399+0000 7f3eb386b640 1 -- 172.21.15.98:0/1677655308 shutdown_connections 2024-05-09T10:50:52.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.399+0000 7f3eb386b640 1 --2- 172.21.15.98:0/1677655308 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3e9806b4d0 0x7f3e9806d990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.399+0000 7f3eb386b640 1 --2- 172.21.15.98:0/1677655308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3eac15df70 0x7f3eac0ad980 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.399+0000 7f3eb386b640 1 --2- 172.21.15.98:0/1677655308 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3eac15db90 0x7f3eac1e6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.399+0000 7f3eb386b640 1 --2- 172.21.15.98:0/1677655308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3eac071910 0x7f3eac156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.399+0000 7f3eb386b640 1 -- 172.21.15.98:0/1677655308 >> 172.21.15.98:0/1677655308 conn(0x7f3eac066f20 msgr2=0x7f3eac1603c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:52.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.400+0000 7f3eb386b640 1 -- 172.21.15.98:0/1677655308 shutdown_connections 2024-05-09T10:50:52.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.400+0000 7f3eb386b640 1 -- 172.21.15.98:0/1677655308 wait complete. 2024-05-09T10:50:52.405 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T10:50:07.396423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T10:50:17.323444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T10:50:18.482315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T10:50:52.405 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T10:50:52.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.514+0000 7fe2f044a640 1 Processor -- start 2024-05-09T10:50:52.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.515+0000 7fe2f044a640 1 -- start start 2024-05-09T10:50:52.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.515+0000 7fe2f044a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe2e815e0c0 0x7fe2e80d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:52.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.516+0000 7fe2f044a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2e80ced40 0x7fe2e80ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:52.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.516+0000 7fe2f044a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e80b4380 con 0x7fe2e80ce1c0 2024-05-09T10:50:52.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.516+0000 7fe2f044a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e806bcf0 con 0x7fe2e815e0c0 2024-05-09T10:50:52.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.516+0000 7fe2ed9e5640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe2e80ce1c0 0x7fe2e80ce630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41170/0 (socket says 172.21.15.98:41170) 2024-05-09T10:50:52.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.516+0000 7fe2ed9e5640 1 -- 172.21.15.98:0/1892234864 learned_addr learned my addr 172.21.15.98:0/1892234864 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:52.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.516+0000 7fe2f044a640 1 -- 172.21.15.98:0/1892234864 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e815e4a0 con 0x7fe2e80ced40 2024-05-09T10:50:52.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.516+0000 7fe2ee1e6640 1 --2- 172.21.15.98:0/1892234864 >> v2:172.21.15.98:3300/0 conn(0x7fe2e815e0c0 0x7fe2e80d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:52.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.516+0000 7fe2ee9e7640 1 --2- 172.21.15.98:0/1892234864 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2e80ced40 0x7fe2e80ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:52.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.516+0000 7fe2ee1e6640 1 -- 172.21.15.98:0/1892234864 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2e80ced40 msgr2=0x7fe2e80ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:52.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.516+0000 7fe2ee1e6640 1 --2- 172.21.15.98:0/1892234864 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2e80ced40 0x7fe2e80ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.516+0000 7fe2ee1e6640 1 -- 172.21.15.98:0/1892234864 >> v1:172.21.15.98:6789/0 conn(0x7fe2e80ce1c0 legacy=0x7fe2e80ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:52.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.516+0000 7fe2ee1e6640 1 -- 172.21.15.98:0/1892234864 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e80d0d70 con 0x7fe2e815e0c0 2024-05-09T10:50:52.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.516+0000 7fe2ed1e4640 1 -- 172.21.15.98:0/1892234864 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3008586688 0 0) 0x7fe2d8001110 con 0x7fe2e80ce1c0 2024-05-09T10:50:52.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.517+0000 7fe2ee1e6640 1 --2- 172.21.15.98:0/1892234864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2e815e0c0 0x7fe2e80d0690 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe2e40097a0 tx=0x7fe2e402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f904e8157ed898 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:52.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.517+0000 7fe2ed1e4640 1 -- 172.21.15.98:0/1892234864 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4037070 con 0x7fe2e815e0c0 2024-05-09T10:50:52.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.517+0000 7fe2ed1e4640 1 -- 172.21.15.98:0/1892234864 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e4032440 con 0x7fe2e815e0c0 2024-05-09T10:50:52.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.517+0000 7fe2ed1e4640 1 -- 172.21.15.98:0/1892234864 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4032810 con 0x7fe2e815e0c0 2024-05-09T10:50:52.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.517+0000 7fe2f044a640 1 -- 172.21.15.98:0/1892234864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2e815e0c0 msgr2=0x7fe2e80d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:52.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.517+0000 7fe2f044a640 1 --2- 172.21.15.98:0/1892234864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2e815e0c0 0x7fe2e80d0690 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe2e40097a0 tx=0x7fe2e402a180 comp rx=0 tx=0).stop 2024-05-09T10:50:52.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.518+0000 7fe2f044a640 1 -- 172.21.15.98:0/1892234864 shutdown_connections 2024-05-09T10:50:52.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.518+0000 7fe2f044a640 1 --2- 172.21.15.98:0/1892234864 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2e80ced40 0x7fe2e80ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.518+0000 7fe2f044a640 1 --2- 172.21.15.98:0/1892234864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2e815e0c0 0x7fe2e80d0690 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.518+0000 7fe2f044a640 1 -- 172.21.15.98:0/1892234864 >> 172.21.15.98:0/1892234864 conn(0x7fe2e80b3450 msgr2=0x7fe2e80b3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:52.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.518+0000 7fe2f044a640 1 -- 172.21.15.98:0/1892234864 shutdown_connections 2024-05-09T10:50:52.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.518+0000 7fe2f044a640 1 -- 172.21.15.98:0/1892234864 wait complete. 2024-05-09T10:50:52.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.519+0000 7fe2f044a640 1 Processor -- start 2024-05-09T10:50:52.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.519+0000 7fe2f044a640 1 -- start start 2024-05-09T10:50:52.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.519+0000 7fe2f044a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2e80ce1c0 0x7fe2e815c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:52.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.519+0000 7fe2f044a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2e80ced40 0x7fe2e814ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:52.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.519+0000 7fe2f044a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2e815e0c0 0x7fe2e80c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:52.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.519+0000 7fe2f044a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e80cc6f0 con 0x7fe2e815e0c0 2024-05-09T10:50:52.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.520+0000 7fe2f044a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e80cc410 con 0x7fe2e80ce1c0 2024-05-09T10:50:52.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.520+0000 7fe2f044a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e80cd150 con 0x7fe2e80ced40 2024-05-09T10:50:52.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.520+0000 7fe2ee9e7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2e815e0c0 0x7fe2e80c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:52.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.520+0000 7fe2ee9e7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2e815e0c0 0x7fe2e80c7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34470/0 (socket says 172.21.15.98:34470) 2024-05-09T10:50:52.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.520+0000 7fe2ee9e7640 1 -- 172.21.15.98:0/3227496930 learned_addr learned my addr 172.21.15.98:0/3227496930 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:52.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.520+0000 7fe2ed9e5640 1 --2- 172.21.15.98:0/3227496930 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2e80ced40 0x7fe2e814ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:52.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.520+0000 7fe2ee1e6640 1 --2- 172.21.15.98:0/3227496930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2e80ce1c0 0x7fe2e815c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:52.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.520+0000 7fe2ee9e7640 1 -- 172.21.15.98:0/3227496930 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2e80ced40 msgr2=0x7fe2e814ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:52.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.520+0000 7fe2ee9e7640 1 --2- 172.21.15.98:0/3227496930 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2e80ced40 0x7fe2e814ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.520+0000 7fe2ee9e7640 1 -- 172.21.15.98:0/3227496930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2e80ce1c0 msgr2=0x7fe2e815c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:52.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.520+0000 7fe2ee9e7640 1 --2- 172.21.15.98:0/3227496930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2e80ce1c0 0x7fe2e815c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.520+0000 7fe2ee9e7640 1 -- 172.21.15.98:0/3227496930 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e80c8f80 con 0x7fe2e815e0c0 2024-05-09T10:50:52.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.520+0000 7fe2ed9e5640 1 --2- 172.21.15.98:0/3227496930 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2e80ced40 0x7fe2e814ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:52.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.520+0000 7fe2ee9e7640 1 --2- 172.21.15.98:0/3227496930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2e815e0c0 0x7fe2e80c7770 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000efc0 tx=0x7fe2e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:52.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.520+0000 7fe2deffd640 1 -- 172.21.15.98:0/3227496930 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0018020 con 0x7fe2e815e0c0 2024-05-09T10:50:52.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.520+0000 7fe2deffd640 1 -- 172.21.15.98:0/3227496930 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e0014480 con 0x7fe2e815e0c0 2024-05-09T10:50:52.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.520+0000 7fe2f044a640 1 -- 172.21.15.98:0/3227496930 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e815c750 con 0x7fe2e815e0c0 2024-05-09T10:50:52.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.520+0000 7fe2deffd640 1 -- 172.21.15.98:0/3227496930 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0014870 con 0x7fe2e815e0c0 2024-05-09T10:50:52.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.520+0000 7fe2f044a640 1 -- 172.21.15.98:0/3227496930 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e80c6f70 con 0x7fe2e815e0c0 2024-05-09T10:50:52.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.521+0000 7fe2f044a640 1 -- 172.21.15.98:0/3227496930 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e809fab0 con 0x7fe2e815e0c0 2024-05-09T10:50:52.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.522+0000 7fe2deffd640 1 -- 172.21.15.98:0/3227496930 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2e0005d10 con 0x7fe2e815e0c0 2024-05-09T10:50:52.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.522+0000 7fe2deffd640 1 --2- 172.21.15.98:0/3227496930 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe2cc06b500 0x7fe2cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:52.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.522+0000 7fe2deffd640 1 -- 172.21.15.98:0/3227496930 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2e008ad20 con 0x7fe2e815e0c0 2024-05-09T10:50:52.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.522+0000 7fe2ee1e6640 1 --2- 172.21.15.98:0/3227496930 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe2cc06b500 0x7fe2cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:52.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.523+0000 7fe2ee1e6640 1 --2- 172.21.15.98:0/3227496930 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe2cc06b500 0x7fe2cc06d9c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4034000 tx=0x7fe2e402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:52.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.527+0000 7fe2deffd640 1 -- 172.21.15.98:0/3227496930 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe2e0022070 con 0x7fe2e815e0c0 2024-05-09T10:50:52.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.696+0000 7fe2f044a640 1 -- 172.21.15.98:0/3227496930 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe2e806eb40 con 0x7fe2e815e0c0 2024-05-09T10:50:52.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.696+0000 7fe2deffd640 1 -- 172.21.15.98:0/3227496930 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe2e80c8f80 con 0x7fe2e815e0c0 2024-05-09T10:50:52.691 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-05-09T10:50:52.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.698+0000 7fe2f044a640 1 -- 172.21.15.98:0/3227496930 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe2cc06b500 msgr2=0x7fe2cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:52.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.698+0000 7fe2f044a640 1 --2- 172.21.15.98:0/3227496930 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe2cc06b500 0x7fe2cc06d9c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4034000 tx=0x7fe2e402e000 comp rx=0 tx=0).stop 2024-05-09T10:50:52.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.698+0000 7fe2f044a640 1 -- 172.21.15.98:0/3227496930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2e815e0c0 msgr2=0x7fe2e80c7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:52.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.698+0000 7fe2f044a640 1 --2- 172.21.15.98:0/3227496930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2e815e0c0 0x7fe2e80c7770 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe2e000efc0 tx=0x7fe2e000c490 comp rx=0 tx=0).stop 2024-05-09T10:50:52.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.698+0000 7fe2f044a640 1 -- 172.21.15.98:0/3227496930 shutdown_connections 2024-05-09T10:50:52.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.698+0000 7fe2f044a640 1 --2- 172.21.15.98:0/3227496930 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe2cc06b500 0x7fe2cc06d9c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.698+0000 7fe2f044a640 1 --2- 172.21.15.98:0/3227496930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2e815e0c0 0x7fe2e80c7770 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.698+0000 7fe2f044a640 1 --2- 172.21.15.98:0/3227496930 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2e80ced40 0x7fe2e814ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.698+0000 7fe2f044a640 1 --2- 172.21.15.98:0/3227496930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2e80ce1c0 0x7fe2e815c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.698+0000 7fe2f044a640 1 -- 172.21.15.98:0/3227496930 >> 172.21.15.98:0/3227496930 conn(0x7fe2e80b3450 msgr2=0x7fe2e80cd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:52.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.698+0000 7fe2f044a640 1 -- 172.21.15.98:0/3227496930 shutdown_connections 2024-05-09T10:50:52.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.698+0000 7fe2f044a640 1 -- 172.21.15.98:0/3227496930 wait complete. 2024-05-09T10:50:52.703 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T10:50:52.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.811+0000 7fb1b6eb3640 1 Processor -- start 2024-05-09T10:50:52.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.811+0000 7fb1b6eb3640 1 -- start start 2024-05-09T10:50:52.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.812+0000 7fb1b6eb3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb1b00cf900 0x7fb1b00b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:52.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.812+0000 7fb1b6eb3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1b00cfce0 0x7fb1b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:52.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.812+0000 7fb1b6eb3640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b006ded0 con 0x7fb1b006c520 2024-05-09T10:50:52.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.812+0000 7fb1b6eb3640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1b015f320 con 0x7fb1b00cf900 2024-05-09T10:50:52.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.812+0000 7fb1b6eb3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b015e310 con 0x7fb1b00cfce0 2024-05-09T10:50:52.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.812+0000 7fb1b5450640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb1b006c520 0x7fb1b006c9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41186/0 (socket says 172.21.15.98:41186) 2024-05-09T10:50:52.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.812+0000 7fb1b5450640 1 -- 172.21.15.98:0/1597218049 learned_addr learned my addr 172.21.15.98:0/1597218049 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:52.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.812+0000 7fb1affff640 1 --2- 172.21.15.98:0/1597218049 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1b00cfce0 0x7fb1b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:52.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.812+0000 7fb1b4c4f640 1 --2- 172.21.15.98:0/1597218049 >> v2:172.21.15.105:3300/0 conn(0x7fb1b00cf900 0x7fb1b00b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:52.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.813+0000 7fb1af7fe640 1 -- 172.21.15.98:0/1597218049 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4188894089 0 0) 0x7fb1b006ded0 con 0x7fb1b006c520 2024-05-09T10:50:52.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.813+0000 7fb1af7fe640 1 -- 172.21.15.98:0/1597218049 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb198003520 con 0x7fb1b006c520 2024-05-09T10:50:52.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.813+0000 7fb1affff640 1 -- 172.21.15.98:0/1597218049 >> v2:172.21.15.105:3300/0 conn(0x7fb1b00cf900 msgr2=0x7fb1b00b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:52.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.813+0000 7fb1affff640 1 --2- 172.21.15.98:0/1597218049 >> v2:172.21.15.105:3300/0 conn(0x7fb1b00cf900 0x7fb1b00b4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.813+0000 7fb1affff640 1 -- 172.21.15.98:0/1597218049 >> v1:172.21.15.98:6789/0 conn(0x7fb1b006c520 legacy=0x7fb1b006c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:52.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.813+0000 7fb1affff640 1 -- 172.21.15.98:0/1597218049 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b015e560 con 0x7fb1b00cfce0 2024-05-09T10:50:52.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.814+0000 7fb1affff640 1 --2- 172.21.15.98:0/1597218049 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1b00cfce0 0x7fb1b006d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40097a0 tx=0x7fb1a402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d0047b8a6c8d64a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:52.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.814+0000 7fb1af7fe640 1 -- 172.21.15.98:0/1597218049 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4037070 con 0x7fb1b00cfce0 2024-05-09T10:50:52.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.814+0000 7fb1af7fe640 1 -- 172.21.15.98:0/1597218049 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a4032440 con 0x7fb1b00cfce0 2024-05-09T10:50:52.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.814+0000 7fb1af7fe640 1 -- 172.21.15.98:0/1597218049 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a40327f0 con 0x7fb1b00cfce0 2024-05-09T10:50:52.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.814+0000 7fb1b6eb3640 1 -- 172.21.15.98:0/1597218049 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1b00cfce0 msgr2=0x7fb1b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:52.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.814+0000 7fb1b6eb3640 1 --2- 172.21.15.98:0/1597218049 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1b00cfce0 0x7fb1b006d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40097a0 tx=0x7fb1a402a180 comp rx=0 tx=0).stop 2024-05-09T10:50:52.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.814+0000 7fb1b6eb3640 1 -- 172.21.15.98:0/1597218049 shutdown_connections 2024-05-09T10:50:52.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.814+0000 7fb1b6eb3640 1 --2- 172.21.15.98:0/1597218049 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1b00cfce0 0x7fb1b006d990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.814+0000 7fb1b6eb3640 1 --2- 172.21.15.98:0/1597218049 >> v2:172.21.15.105:3300/0 conn(0x7fb1b00cf900 0x7fb1b00b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.814+0000 7fb1b6eb3640 1 -- 172.21.15.98:0/1597218049 >> 172.21.15.98:0/1597218049 conn(0x7fb1b006abe0 msgr2=0x7fb1b006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:52.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.814+0000 7fb1b6eb3640 1 -- 172.21.15.98:0/1597218049 shutdown_connections 2024-05-09T10:50:52.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.814+0000 7fb1b6eb3640 1 -- 172.21.15.98:0/1597218049 wait complete. 2024-05-09T10:50:52.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.815+0000 7fb1b6eb3640 1 Processor -- start 2024-05-09T10:50:52.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.816+0000 7fb1b6eb3640 1 -- start start 2024-05-09T10:50:52.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.816+0000 7fb1b6eb3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1b006c520 0x7fb1b00cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:52.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.816+0000 7fb1b6eb3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1b00cf900 0x7fb1b00acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:52.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.816+0000 7fb1b6eb3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1b00cfce0 0x7fb1b015d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:52.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.816+0000 7fb1b6eb3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b015ee00 con 0x7fb1b006c520 2024-05-09T10:50:52.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.816+0000 7fb1b6eb3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b015eb20 con 0x7fb1b00cfce0 2024-05-09T10:50:52.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.816+0000 7fb1b6eb3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b00d0160 con 0x7fb1b00cf900 2024-05-09T10:50:52.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.817+0000 7fb1b4c4f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1b006c520 0x7fb1b00cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:52.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.817+0000 7fb1b4c4f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1b006c520 0x7fb1b00cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34480/0 (socket says 172.21.15.98:34480) 2024-05-09T10:50:52.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.817+0000 7fb1b4c4f640 1 -- 172.21.15.98:0/517822977 learned_addr learned my addr 172.21.15.98:0/517822977 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:52.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.817+0000 7fb1affff640 1 --2- 172.21.15.98:0/517822977 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1b00cf900 0x7fb1b00acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:52.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.817+0000 7fb1b5450640 1 --2- 172.21.15.98:0/517822977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1b00cfce0 0x7fb1b015d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:52.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.817+0000 7fb1b4c4f640 1 -- 172.21.15.98:0/517822977 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1b00cf900 msgr2=0x7fb1b00acfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:52.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.817+0000 7fb1b4c4f640 1 --2- 172.21.15.98:0/517822977 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1b00cf900 0x7fb1b00acfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.817+0000 7fb1b4c4f640 1 -- 172.21.15.98:0/517822977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1b00cfce0 msgr2=0x7fb1b015d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:52.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.817+0000 7fb1b4c4f640 1 --2- 172.21.15.98:0/517822977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1b00cfce0 0x7fb1b015d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.817+0000 7fb1b4c4f640 1 -- 172.21.15.98:0/517822977 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b00ad750 con 0x7fb1b006c520 2024-05-09T10:50:52.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.817+0000 7fb1b4c4f640 1 --2- 172.21.15.98:0/517822977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1b006c520 0x7fb1b00cb350 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000e9e0 tx=0x7fb1a000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:52.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.817+0000 7fb1ad7fa640 1 -- 172.21.15.98:0/517822977 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0017070 con 0x7fb1b006c520 2024-05-09T10:50:52.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.817+0000 7fb1ad7fa640 1 -- 172.21.15.98:0/517822977 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a000c9d0 con 0x7fb1b006c520 2024-05-09T10:50:52.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.817+0000 7fb1b6eb3640 1 -- 172.21.15.98:0/517822977 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b00ada30 con 0x7fb1b006c520 2024-05-09T10:50:52.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.817+0000 7fb1ad7fa640 1 -- 172.21.15.98:0/517822977 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a000cdc0 con 0x7fb1b006c520 2024-05-09T10:50:52.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.817+0000 7fb1b6eb3640 1 -- 172.21.15.98:0/517822977 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b00adff0 con 0x7fb1b006c520 2024-05-09T10:50:52.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.818+0000 7fb1b6eb3640 1 -- 172.21.15.98:0/517822977 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b004f000 con 0x7fb1b006c520 2024-05-09T10:50:52.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.819+0000 7fb1ad7fa640 1 -- 172.21.15.98:0/517822977 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb1a001a070 con 0x7fb1b006c520 2024-05-09T10:50:52.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.819+0000 7fb1ad7fa640 1 --2- 172.21.15.98:0/517822977 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb19c06b500 0x7fb19c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:52.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.819+0000 7fb1ad7fa640 1 -- 172.21.15.98:0/517822977 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1a008a230 con 0x7fb1b006c520 2024-05-09T10:50:52.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.819+0000 7fb1affff640 1 --2- 172.21.15.98:0/517822977 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb19c06b500 0x7fb19c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:52.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.820+0000 7fb1affff640 1 --2- 172.21.15.98:0/517822977 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb19c06b500 0x7fb19c06d9c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4034000 tx=0x7fb1a402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:52.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.823+0000 7fb1ad7fa640 1 -- 172.21.15.98:0/517822977 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb1a00589d0 con 0x7fb1b006c520 2024-05-09T10:50:52.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.995+0000 7fb1b6eb3640 1 -- 172.21.15.98:0/517822977 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb1b00b4a00 con 0x7fb1b006c520 2024-05-09T10:50:52.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.996+0000 7fb1ad7fa640 1 -- 172.21.15.98:0/517822977 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb1a005ca70 con 0x7fb1b006c520 2024-05-09T10:50:52.991 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2024-05-09T10:50:52.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.998+0000 7fb1b6eb3640 1 -- 172.21.15.98:0/517822977 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb19c06b500 msgr2=0x7fb19c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:52.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.998+0000 7fb1b6eb3640 1 --2- 172.21.15.98:0/517822977 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb19c06b500 0x7fb19c06d9c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4034000 tx=0x7fb1a402e000 comp rx=0 tx=0).stop 2024-05-09T10:50:52.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.998+0000 7fb1b6eb3640 1 -- 172.21.15.98:0/517822977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1b006c520 msgr2=0x7fb1b00cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:52.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.998+0000 7fb1b6eb3640 1 --2- 172.21.15.98:0/517822977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1b006c520 0x7fb1b00cb350 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000e9e0 tx=0x7fb1a000eea0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.998+0000 7fb1b6eb3640 1 -- 172.21.15.98:0/517822977 shutdown_connections 2024-05-09T10:50:52.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.998+0000 7fb1b6eb3640 1 --2- 172.21.15.98:0/517822977 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb19c06b500 0x7fb19c06d9c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.998+0000 7fb1b6eb3640 1 --2- 172.21.15.98:0/517822977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1b00cfce0 0x7fb1b015d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.998+0000 7fb1b6eb3640 1 --2- 172.21.15.98:0/517822977 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1b00cf900 0x7fb1b00acfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.998+0000 7fb1b6eb3640 1 --2- 172.21.15.98:0/517822977 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1b006c520 0x7fb1b00cb350 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:52.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.998+0000 7fb1b6eb3640 1 -- 172.21.15.98:0/517822977 >> 172.21.15.98:0/517822977 conn(0x7fb1b006abe0 msgr2=0x7fb1b006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:52.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.998+0000 7fb1b6eb3640 1 -- 172.21.15.98:0/517822977 shutdown_connections 2024-05-09T10:50:52.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:52.998+0000 7fb1b6eb3640 1 -- 172.21.15.98:0/517822977 wait complete. 2024-05-09T10:50:53.003 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T10:50:53.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.111+0000 7fe26985a640 1 Processor -- start 2024-05-09T10:50:53.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.111+0000 7fe26985a640 1 -- start start 2024-05-09T10:50:53.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.112+0000 7fe26985a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe26415dd60 0x7fe2640d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:53.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.112+0000 7fe26985a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe2640d24f0 0x7fe2640cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:53.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.112+0000 7fe26985a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2640cf8e0 con 0x7fe26415e140 2024-05-09T10:50:53.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.112+0000 7fe26985a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2640cf6b0 con 0x7fe2640d24f0 2024-05-09T10:50:53.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.112+0000 7fe26985a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2640b2f50 con 0x7fe26415dd60 2024-05-09T10:50:53.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.112+0000 7fe2637fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe2640d24f0 0x7fe2640cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:53.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.112+0000 7fe262ffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe26415dd60 0x7fe2640d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:53.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.112+0000 7fe25a7fc640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe26415e140 0x7fe2640d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.98:46548/0 (socket says 172.21.15.98:46548) 2024-05-09T10:50:53.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.113+0000 7fe25a7fc640 1 -- 172.21.15.98:0/3805941287 learned_addr learned my addr 172.21.15.98:0/3805941287 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:53.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.113+0000 7fe262ffd640 1 -- 172.21.15.98:0/3805941287 >> v2:172.21.15.105:3300/0 conn(0x7fe2640d24f0 msgr2=0x7fe2640cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:53.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.113+0000 7fe262ffd640 1 --2- 172.21.15.98:0/3805941287 >> v2:172.21.15.105:3300/0 conn(0x7fe2640d24f0 0x7fe2640cf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.113+0000 7fe262ffd640 1 -- 172.21.15.98:0/3805941287 >> v1:172.21.15.105:6789/0 conn(0x7fe26415e140 legacy=0x7fe2640d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:53.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.113+0000 7fe262ffd640 1 -- 172.21.15.98:0/3805941287 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2640b31a0 con 0x7fe26415dd60 2024-05-09T10:50:53.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.113+0000 7fe2637fe640 1 --2- 172.21.15.98:0/3805941287 >> v2:172.21.15.105:3300/0 conn(0x7fe2640d24f0 0x7fe2640cf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:50:53.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.114+0000 7fe262ffd640 1 --2- 172.21.15.98:0/3805941287 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe26415dd60 0x7fe2640d1790 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe2540097a0 tx=0x7fe25402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ddfcc08a81c0316 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:53.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.114+0000 7fe2627fc640 1 -- 172.21.15.98:0/3805941287 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe254037070 con 0x7fe26415dd60 2024-05-09T10:50:53.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.114+0000 7fe2627fc640 1 -- 172.21.15.98:0/3805941287 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe254032440 con 0x7fe26415dd60 2024-05-09T10:50:53.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.114+0000 7fe2627fc640 1 -- 172.21.15.98:0/3805941287 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe254032810 con 0x7fe26415dd60 2024-05-09T10:50:53.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.114+0000 7fe26985a640 1 -- 172.21.15.98:0/3805941287 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe26415dd60 msgr2=0x7fe2640d1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:53.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.114+0000 7fe26985a640 1 --2- 172.21.15.98:0/3805941287 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe26415dd60 0x7fe2640d1790 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe2540097a0 tx=0x7fe25402a180 comp rx=0 tx=0).stop 2024-05-09T10:50:53.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.114+0000 7fe26985a640 1 -- 172.21.15.98:0/3805941287 shutdown_connections 2024-05-09T10:50:53.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.114+0000 7fe26985a640 1 --2- 172.21.15.98:0/3805941287 >> v2:172.21.15.105:3300/0 conn(0x7fe2640d24f0 0x7fe2640cf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.114+0000 7fe26985a640 1 --2- 172.21.15.98:0/3805941287 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe26415dd60 0x7fe2640d1790 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.114+0000 7fe26985a640 1 -- 172.21.15.98:0/3805941287 >> 172.21.15.98:0/3805941287 conn(0x7fe2640d3b50 msgr2=0x7fe2640d3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:53.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.114+0000 7fe26985a640 1 -- 172.21.15.98:0/3805941287 shutdown_connections 2024-05-09T10:50:53.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.115+0000 7fe26985a640 1 -- 172.21.15.98:0/3805941287 wait complete. 2024-05-09T10:50:53.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.116+0000 7fe26985a640 1 Processor -- start 2024-05-09T10:50:53.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.116+0000 7fe26985a640 1 -- start start 2024-05-09T10:50:53.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.116+0000 7fe26985a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2640d24f0 0x7fe2640cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:53.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.116+0000 7fe26985a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe26415dd60 0x7fe264154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:53.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.116+0000 7fe26985a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe26415fb00 0x7fe26415ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:53.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.116+0000 7fe26985a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2640b3c80 con 0x7fe2640d24f0 2024-05-09T10:50:53.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.116+0000 7fe26985a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2640b3760 con 0x7fe26415fb00 2024-05-09T10:50:53.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.116+0000 7fe26985a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2640b3e00 con 0x7fe26415dd60 2024-05-09T10:50:53.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.117+0000 7fe262ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2640d24f0 0x7fe2640cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:53.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.117+0000 7fe262ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2640d24f0 0x7fe2640cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34488/0 (socket says 172.21.15.98:34488) 2024-05-09T10:50:53.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.117+0000 7fe262ffd640 1 -- 172.21.15.98:0/2936611442 learned_addr learned my addr 172.21.15.98:0/2936611442 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:53.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.117+0000 7fe2637fe640 1 --2- 172.21.15.98:0/2936611442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe26415fb00 0x7fe26415ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:53.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.117+0000 7fe25a7fc640 1 --2- 172.21.15.98:0/2936611442 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe26415dd60 0x7fe264154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:53.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.117+0000 7fe262ffd640 1 -- 172.21.15.98:0/2936611442 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe26415dd60 msgr2=0x7fe264154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:53.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.117+0000 7fe262ffd640 1 --2- 172.21.15.98:0/2936611442 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe26415dd60 0x7fe264154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.117+0000 7fe262ffd640 1 -- 172.21.15.98:0/2936611442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe26415fb00 msgr2=0x7fe26415ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:53.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.117+0000 7fe262ffd640 1 --2- 172.21.15.98:0/2936611442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe26415fb00 0x7fe26415ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.117+0000 7fe262ffd640 1 -- 172.21.15.98:0/2936611442 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2640cacc0 con 0x7fe2640d24f0 2024-05-09T10:50:53.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.117+0000 7fe2637fe640 1 --2- 172.21.15.98:0/2936611442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe26415fb00 0x7fe26415ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:53.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.117+0000 7fe262ffd640 1 --2- 172.21.15.98:0/2936611442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2640d24f0 0x7fe2640cba60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe254035040 tx=0x7fe2540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:53.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.117+0000 7fe260ff9640 1 -- 172.21.15.98:0/2936611442 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe254037040 con 0x7fe2640d24f0 2024-05-09T10:50:53.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.117+0000 7fe260ff9640 1 -- 172.21.15.98:0/2936611442 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe254030500 con 0x7fe2640d24f0 2024-05-09T10:50:53.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.117+0000 7fe26985a640 1 -- 172.21.15.98:0/2936611442 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2640caf80 con 0x7fe2640d24f0 2024-05-09T10:50:53.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.117+0000 7fe260ff9640 1 -- 172.21.15.98:0/2936611442 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe254030910 con 0x7fe2640d24f0 2024-05-09T10:50:53.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.117+0000 7fe26985a640 1 -- 172.21.15.98:0/2936611442 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2640cb540 con 0x7fe2640d24f0 2024-05-09T10:50:53.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.118+0000 7fe26985a640 1 -- 172.21.15.98:0/2936611442 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2640d3400 con 0x7fe2640d24f0 2024-05-09T10:50:53.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.119+0000 7fe260ff9640 1 -- 172.21.15.98:0/2936611442 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe254044020 con 0x7fe2640d24f0 2024-05-09T10:50:53.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.119+0000 7fe260ff9640 1 --2- 172.21.15.98:0/2936611442 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe24406b4d0 0x7fe24406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:53.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.119+0000 7fe260ff9640 1 -- 172.21.15.98:0/2936611442 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2540aa490 con 0x7fe2640d24f0 2024-05-09T10:50:53.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.119+0000 7fe25a7fc640 1 --2- 172.21.15.98:0/2936611442 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe24406b4d0 0x7fe24406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:53.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.120+0000 7fe25a7fc640 1 --2- 172.21.15.98:0/2936611442 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe24406b4d0 0x7fe24406d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe264160920 tx=0x7fe24c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:53.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.123+0000 7fe260ff9640 1 -- 172.21.15.98:0/2936611442 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe2640d3400 con 0x7fe2640d24f0 2024-05-09T10:50:53.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.294+0000 7fe26985a640 1 -- 172.21.15.98:0/2936611442 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe2640a1b60 con 0x7fe2640d24f0 2024-05-09T10:50:53.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.295+0000 7fe260ff9640 1 -- 172.21.15.98:0/2936611442 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe254079320 con 0x7fe2640d24f0 2024-05-09T10:50:53.290 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2024-05-09T10:50:53.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.297+0000 7fe26985a640 1 -- 172.21.15.98:0/2936611442 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe24406b4d0 msgr2=0x7fe24406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:53.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.297+0000 7fe26985a640 1 --2- 172.21.15.98:0/2936611442 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe24406b4d0 0x7fe24406d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe264160920 tx=0x7fe24c009040 comp rx=0 tx=0).stop 2024-05-09T10:50:53.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.297+0000 7fe26985a640 1 -- 172.21.15.98:0/2936611442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2640d24f0 msgr2=0x7fe2640cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:53.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.297+0000 7fe26985a640 1 --2- 172.21.15.98:0/2936611442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2640d24f0 0x7fe2640cba60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe254035040 tx=0x7fe2540046c0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.297+0000 7fe26985a640 1 -- 172.21.15.98:0/2936611442 shutdown_connections 2024-05-09T10:50:53.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.297+0000 7fe26985a640 1 --2- 172.21.15.98:0/2936611442 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe24406b4d0 0x7fe24406d990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.297+0000 7fe26985a640 1 --2- 172.21.15.98:0/2936611442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe26415fb00 0x7fe26415ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.297+0000 7fe26985a640 1 --2- 172.21.15.98:0/2936611442 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe26415dd60 0x7fe264154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.297+0000 7fe26985a640 1 --2- 172.21.15.98:0/2936611442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2640d24f0 0x7fe2640cba60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.297+0000 7fe26985a640 1 -- 172.21.15.98:0/2936611442 >> 172.21.15.98:0/2936611442 conn(0x7fe2640d3b50 msgr2=0x7fe2640b0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:53.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.297+0000 7fe26985a640 1 -- 172.21.15.98:0/2936611442 shutdown_connections 2024-05-09T10:50:53.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.297+0000 7fe26985a640 1 -- 172.21.15.98:0/2936611442 wait complete. 2024-05-09T10:50:53.302 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T10:50:53.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.410+0000 7f8253b5c640 1 Processor -- start 2024-05-09T10:50:53.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.410+0000 7f8253b5c640 1 -- start start 2024-05-09T10:50:53.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.411+0000 7f8253b5c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f824c0d0b80 0x7f824c0b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:53.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.411+0000 7f8253b5c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f824c0d2c30 con 0x7f824c0d27a0 2024-05-09T10:50:53.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.411+0000 7f8253b5c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f824c15e220 con 0x7f824c15de40 2024-05-09T10:50:53.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.411+0000 7f8253b5c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f824c0d1790 con 0x7f824c0d0b80 2024-05-09T10:50:53.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.411+0000 7f82510f7640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f824c0d27a0 0x7f824c0d0470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41196/0 (socket says 172.21.15.98:41196) 2024-05-09T10:50:53.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.411+0000 7f82510f7640 1 -- 172.21.15.98:0/1752451729 learned_addr learned my addr 172.21.15.98:0/1752451729 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:53.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82508f6640 1 -- 172.21.15.98:0/1752451729 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596636956 0 0) 0x7f824c0d2c30 con 0x7f824c0d27a0 2024-05-09T10:50:53.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82508f6640 1 -- 172.21.15.98:0/1752451729 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8238003520 con 0x7f824c0d27a0 2024-05-09T10:50:53.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82520f9640 1 --2- 172.21.15.98:0/1752451729 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f824c0d0b80 0x7f824c0b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:53.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82508f6640 1 -- 172.21.15.98:0/1752451729 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1458695548 0 0) 0x7f8238003520 con 0x7f824c0d27a0 2024-05-09T10:50:53.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82508f6640 1 -- 172.21.15.98:0/1752451729 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f824c0d0b80 msgr2=0x7f824c0b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:53.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82508f6640 1 --2- 172.21.15.98:0/1752451729 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f824c0d0b80 0x7f824c0b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82508f6640 1 -- 172.21.15.98:0/1752451729 >> v1:172.21.15.105:6789/0 conn(0x7f824c15de40 legacy=0x7f824c0cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:53.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82508f6640 1 -- 172.21.15.98:0/1752451729 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f824c0d19e0 con 0x7f824c0d27a0 2024-05-09T10:50:53.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82508f6640 1 -- 172.21.15.98:0/1752451729 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f823c001670 con 0x7f824c0d27a0 2024-05-09T10:50:53.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82508f6640 1 -- 172.21.15.98:0/1752451729 >> v1:172.21.15.98:6789/0 conn(0x7f824c0d27a0 legacy=0x7f824c0d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:53.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82508f6640 1 --2- 172.21.15.98:0/1752451729 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f824c0d0b80 0x7f82380039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:53.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82508f6640 1 --2- 172.21.15.98:0/1752451729 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8238003f60 0x7f8238004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:53.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82508f6640 1 --2- 172.21.15.98:0/1752451729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8238004d40 0x7f82380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:53.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82508f6640 1 -- 172.21.15.98:0/1752451729 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824c0d1790 con 0x7f8238003f60 2024-05-09T10:50:53.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82508f6640 1 -- 172.21.15.98:0/1752451729 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823800baa0 con 0x7f8238004d40 2024-05-09T10:50:53.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82508f6640 1 -- 172.21.15.98:0/1752451729 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823800bca0 con 0x7f824c0d0b80 2024-05-09T10:50:53.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82510f7640 1 --2- 172.21.15.98:0/1752451729 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8238003f60 0x7f8238004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:53.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82520f9640 1 --2- 172.21.15.98:0/1752451729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8238004d40 0x7f82380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:53.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82518f8640 1 --2- 172.21.15.98:0/1752451729 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f824c0d0b80 0x7f82380039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:53.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82510f7640 1 -- 172.21.15.98:0/1752451729 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f824c0d0b80 msgr2=0x7f82380039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:53.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82510f7640 1 --2- 172.21.15.98:0/1752451729 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f824c0d0b80 0x7f82380039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82510f7640 1 -- 172.21.15.98:0/1752451729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8238004d40 msgr2=0x7f82380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:53.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82510f7640 1 --2- 172.21.15.98:0/1752451729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8238004d40 0x7f82380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.412+0000 7f82510f7640 1 -- 172.21.15.98:0/1752451729 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8238003520 con 0x7f8238003f60 2024-05-09T10:50:53.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.413+0000 7f82510f7640 1 --2- 172.21.15.98:0/1752451729 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8238003f60 0x7f8238004800 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f823c00c210 tx=0x7f8238005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1834991ce98e73dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:53.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.413+0000 7f82508f6640 1 -- 172.21.15.98:0/1752451729 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c007d30 con 0x7f8238003f60 2024-05-09T10:50:53.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.413+0000 7f82508f6640 1 -- 172.21.15.98:0/1752451729 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823c005af0 con 0x7f8238003f60 2024-05-09T10:50:53.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.413+0000 7f82508f6640 1 -- 172.21.15.98:0/1752451729 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c007070 con 0x7f8238003f60 2024-05-09T10:50:53.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.414+0000 7f8253b5c640 1 -- 172.21.15.98:0/1752451729 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8238003f60 msgr2=0x7f8238004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:53.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.414+0000 7f8253b5c640 1 --2- 172.21.15.98:0/1752451729 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8238003f60 0x7f8238004800 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f823c00c210 tx=0x7f8238005830 comp rx=0 tx=0).stop 2024-05-09T10:50:53.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.414+0000 7f82520f9640 1 -- 172.21.15.98:0/1752451729 reap_dead start 2024-05-09T10:50:53.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.414+0000 7f8253b5c640 1 -- 172.21.15.98:0/1752451729 shutdown_connections 2024-05-09T10:50:53.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.414+0000 7f8253b5c640 1 -- 172.21.15.98:0/1752451729 >> 172.21.15.98:0/1752451729 conn(0x7f824c09ce90 msgr2=0x7f824c0d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:53.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.414+0000 7f8253b5c640 1 -- 172.21.15.98:0/1752451729 shutdown_connections 2024-05-09T10:50:53.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.414+0000 7f8253b5c640 1 -- 172.21.15.98:0/1752451729 wait complete. 2024-05-09T10:50:53.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.415+0000 7f8253b5c640 1 Processor -- start 2024-05-09T10:50:53.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.415+0000 7f8253b5c640 1 -- start start 2024-05-09T10:50:53.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.416+0000 7f8253b5c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8238003f60 0x7f824c14ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:53.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.416+0000 7f8253b5c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f824c0c9320 0x7f824c14d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:53.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.416+0000 7f8253b5c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f824c160fd0 0x7f824c15e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:53.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.416+0000 7f8253b5c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824c0d2280 con 0x7f8238003f60 2024-05-09T10:50:53.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.416+0000 7f8253b5c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824c0d1fa0 con 0x7f824c0c9320 2024-05-09T10:50:53.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.416+0000 7f8253b5c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f824c0cf960 con 0x7f824c160fd0 2024-05-09T10:50:53.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.416+0000 7f82518f8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8238003f60 0x7f824c14ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:53.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.416+0000 7f82518f8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8238003f60 0x7f824c14ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34502/0 (socket says 172.21.15.98:34502) 2024-05-09T10:50:53.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.416+0000 7f82518f8640 1 -- 172.21.15.98:0/345543042 learned_addr learned my addr 172.21.15.98:0/345543042 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:53.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.416+0000 7f82520f9640 1 --2- 172.21.15.98:0/345543042 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f824c160fd0 0x7f824c15e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:53.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.416+0000 7f82510f7640 1 --2- 172.21.15.98:0/345543042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f824c0c9320 0x7f824c14d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:53.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.416+0000 7f82518f8640 1 -- 172.21.15.98:0/345543042 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f824c160fd0 msgr2=0x7f824c15e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:53.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.416+0000 7f82518f8640 1 --2- 172.21.15.98:0/345543042 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f824c160fd0 0x7f824c15e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.416+0000 7f82518f8640 1 -- 172.21.15.98:0/345543042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f824c0c9320 msgr2=0x7f824c14d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:53.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.416+0000 7f82518f8640 1 --2- 172.21.15.98:0/345543042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f824c0c9320 0x7f824c14d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.416+0000 7f82518f8640 1 -- 172.21.15.98:0/345543042 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f824c14c0b0 con 0x7f8238003f60 2024-05-09T10:50:53.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.416+0000 7f82518f8640 1 --2- 172.21.15.98:0/345543042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8238003f60 0x7f824c14ba20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f824800fb90 tx=0x7f824800d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:53.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.417+0000 7f82427fc640 1 -- 172.21.15.98:0/345543042 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824800fe90 con 0x7f8238003f60 2024-05-09T10:50:53.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.417+0000 7f82427fc640 1 -- 172.21.15.98:0/345543042 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8248018070 con 0x7f8238003f60 2024-05-09T10:50:53.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.417+0000 7f8253b5c640 1 -- 172.21.15.98:0/345543042 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f824c14c390 con 0x7f8238003f60 2024-05-09T10:50:53.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.417+0000 7f82427fc640 1 -- 172.21.15.98:0/345543042 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8248003970 con 0x7f8238003f60 2024-05-09T10:50:53.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.417+0000 7f8253b5c640 1 -- 172.21.15.98:0/345543042 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f824c14c950 con 0x7f8238003f60 2024-05-09T10:50:53.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.417+0000 7f8253b5c640 1 -- 172.21.15.98:0/345543042 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f824c069cb0 con 0x7f8238003f60 2024-05-09T10:50:53.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.418+0000 7f82427fc640 1 -- 172.21.15.98:0/345543042 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8248003b10 con 0x7f8238003f60 2024-05-09T10:50:53.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.418+0000 7f82427fc640 1 --2- 172.21.15.98:0/345543042 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f823006b6a0 0x7f823006db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:53.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.419+0000 7f82510f7640 1 --2- 172.21.15.98:0/345543042 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f823006b6a0 0x7f823006db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:53.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.419+0000 7f82427fc640 1 -- 172.21.15.98:0/345543042 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f824808a470 con 0x7f8238003f60 2024-05-09T10:50:53.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.419+0000 7f82510f7640 1 --2- 172.21.15.98:0/345543042 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f823006b6a0 0x7f823006db60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f823c033000 tx=0x7f823c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:53.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.422+0000 7f82427fc640 1 -- 172.21.15.98:0/345543042 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f82480592f0 con 0x7f8238003f60 2024-05-09T10:50:53.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.606+0000 7f8253b5c640 1 -- 172.21.15.98:0/345543042 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f824c0d42d0 con 0x7f8238003f60 2024-05-09T10:50:53.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.606+0000 7f82427fc640 1 -- 172.21.15.98:0/345543042 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8248058bf0 con 0x7f8238003f60 2024-05-09T10:50:53.601 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-05-09T10:50:53.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.608+0000 7f8253b5c640 1 -- 172.21.15.98:0/345543042 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f823006b6a0 msgr2=0x7f823006db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:53.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.608+0000 7f8253b5c640 1 --2- 172.21.15.98:0/345543042 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f823006b6a0 0x7f823006db60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f823c033000 tx=0x7f823c02e000 comp rx=0 tx=0).stop 2024-05-09T10:50:53.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.608+0000 7f8253b5c640 1 -- 172.21.15.98:0/345543042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8238003f60 msgr2=0x7f824c14ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:53.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.608+0000 7f8253b5c640 1 --2- 172.21.15.98:0/345543042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8238003f60 0x7f824c14ba20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f824800fb90 tx=0x7f824800d520 comp rx=0 tx=0).stop 2024-05-09T10:50:53.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.608+0000 7f8253b5c640 1 -- 172.21.15.98:0/345543042 shutdown_connections 2024-05-09T10:50:53.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.608+0000 7f8253b5c640 1 --2- 172.21.15.98:0/345543042 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f823006b6a0 0x7f823006db60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.608+0000 7f8253b5c640 1 --2- 172.21.15.98:0/345543042 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f824c160fd0 0x7f824c15e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.608+0000 7f8253b5c640 1 --2- 172.21.15.98:0/345543042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f824c0c9320 0x7f824c14d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.608+0000 7f8253b5c640 1 --2- 172.21.15.98:0/345543042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8238003f60 0x7f824c14ba20 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.608+0000 7f8253b5c640 1 -- 172.21.15.98:0/345543042 >> 172.21.15.98:0/345543042 conn(0x7f824c09ce90 msgr2=0x7f824c0b67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:53.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.608+0000 7f8253b5c640 1 -- 172.21.15.98:0/345543042 shutdown_connections 2024-05-09T10:50:53.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.608+0000 7f8253b5c640 1 -- 172.21.15.98:0/345543042 wait complete. 2024-05-09T10:50:53.613 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-05-09T10:50:53.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:53.614 DEBUG:teuthology.orchestra.run.smithi105:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-05-09T10:50:53.646 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2024-05-09T10:50:53.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:50:53.646 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-09T10:50:53.705 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-09T10:50:53.705 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T10:50:53.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.812+0000 7f85b5b22640 1 Processor -- start 2024-05-09T10:50:53.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.812+0000 7f85b5b22640 1 -- start start 2024-05-09T10:50:53.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.813+0000 7f85b5b22640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85b015dce0 0x7f85b006bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:53.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.813+0000 7f85b5b22640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f85b006cac0 0x7f85b00b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:53.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.813+0000 7f85b5b22640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b00d0c70 con 0x7f85b015e0c0 2024-05-09T10:50:53.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.813+0000 7f85b5b22640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85b00b4350 con 0x7f85b006cac0 2024-05-09T10:50:53.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.813+0000 7f85b5b22640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b00d0f50 con 0x7f85b015dce0 2024-05-09T10:50:53.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.813+0000 7f85affff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f85b006cac0 0x7f85b00b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:53.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.813+0000 7f85aeffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f85b015e0c0 0x7f85b006c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38814/0 (socket says 172.21.15.98:38814) 2024-05-09T10:50:53.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.813+0000 7f85aeffd640 1 -- 172.21.15.98:0/641051605 learned_addr learned my addr 172.21.15.98:0/641051605 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:53.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.813+0000 7f85af7fe640 1 --2- 172.21.15.98:0/641051605 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85b015dce0 0x7f85b006bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:53.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.813+0000 7f85affff640 1 -- 172.21.15.98:0/641051605 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85b015dce0 msgr2=0x7f85b006bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:53.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.813+0000 7f85affff640 1 --2- 172.21.15.98:0/641051605 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85b015dce0 0x7f85b006bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.813+0000 7f85affff640 1 -- 172.21.15.98:0/641051605 >> v1:172.21.15.98:6789/0 conn(0x7f85b015e0c0 legacy=0x7f85b006c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:53.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.813+0000 7f85ae7fc640 1 -- 172.21.15.98:0/641051605 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2637419069 0 0) 0x7f85b00d0c70 con 0x7f85b015e0c0 2024-05-09T10:50:53.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.813+0000 7f85affff640 1 -- 172.21.15.98:0/641051605 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b00b0fd0 con 0x7f85b006cac0 2024-05-09T10:50:53.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.814+0000 7f85affff640 1 --2- 172.21.15.98:0/641051605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85b006cac0 0x7f85b00b1ab0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f85a80097a0 tx=0x7f85a802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8667c9d9f92f03 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:53.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.814+0000 7f85ae7fc640 1 -- 172.21.15.98:0/641051605 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a8037070 con 0x7f85b006cac0 2024-05-09T10:50:53.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.814+0000 7f85ae7fc640 1 -- 172.21.15.98:0/641051605 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a8032440 con 0x7f85b006cac0 2024-05-09T10:50:53.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.814+0000 7f85ae7fc640 1 -- 172.21.15.98:0/641051605 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a8032810 con 0x7f85b006cac0 2024-05-09T10:50:53.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.815+0000 7f85b5b22640 1 -- 172.21.15.98:0/641051605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85b006cac0 msgr2=0x7f85b00b1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:53.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.815+0000 7f85b5b22640 1 --2- 172.21.15.98:0/641051605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85b006cac0 0x7f85b00b1ab0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f85a80097a0 tx=0x7f85a802a210 comp rx=0 tx=0).stop 2024-05-09T10:50:53.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.815+0000 7f85b5b22640 1 -- 172.21.15.98:0/641051605 shutdown_connections 2024-05-09T10:50:53.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.815+0000 7f85b5b22640 1 --2- 172.21.15.98:0/641051605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85b006cac0 0x7f85b00b1ab0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.815+0000 7f85b5b22640 1 --2- 172.21.15.98:0/641051605 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85b015dce0 0x7f85b006bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.815+0000 7f85b5b22640 1 -- 172.21.15.98:0/641051605 >> 172.21.15.98:0/641051605 conn(0x7f85b00b34e0 msgr2=0x7f85b00b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:53.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.815+0000 7f85b5b22640 1 -- 172.21.15.98:0/641051605 shutdown_connections 2024-05-09T10:50:53.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.815+0000 7f85b5b22640 1 -- 172.21.15.98:0/641051605 wait complete. 2024-05-09T10:50:53.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.816+0000 7f85b5b22640 1 Processor -- start 2024-05-09T10:50:53.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.816+0000 7f85b5b22640 1 -- start start 2024-05-09T10:50:53.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.817+0000 7f85b5b22640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85b006cac0 0x7f85b0158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:53.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.817+0000 7f85b5b22640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85b015dce0 0x7f85b01f4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:53.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.817+0000 7f85b5b22640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85b015e0c0 0x7f85b00b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:53.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.817+0000 7f85b5b22640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b00d0700 con 0x7f85b015e0c0 2024-05-09T10:50:53.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.817+0000 7f85b5b22640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b00b1590 con 0x7f85b006cac0 2024-05-09T10:50:53.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.817+0000 7f85b5b22640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b00d0880 con 0x7f85b015dce0 2024-05-09T10:50:53.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.817+0000 7f85affff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85b015e0c0 0x7f85b00b6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:53.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.817+0000 7f85affff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85b015e0c0 0x7f85b00b6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35406/0 (socket says 172.21.15.98:35406) 2024-05-09T10:50:53.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.817+0000 7f85affff640 1 -- 172.21.15.98:0/3831245809 learned_addr learned my addr 172.21.15.98:0/3831245809 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:53.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.817+0000 7f85affff640 1 -- 172.21.15.98:0/3831245809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85b015dce0 msgr2=0x7f85b01f4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:53.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.817+0000 7f85affff640 1 --2- 172.21.15.98:0/3831245809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85b015dce0 0x7f85b01f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.817+0000 7f85affff640 1 -- 172.21.15.98:0/3831245809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85b006cac0 msgr2=0x7f85b0158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:53.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.817+0000 7f85affff640 1 --2- 172.21.15.98:0/3831245809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85b006cac0 0x7f85b0158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:53.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.817+0000 7f85affff640 1 -- 172.21.15.98:0/3831245809 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b01f4b70 con 0x7f85b015e0c0 2024-05-09T10:50:53.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.817+0000 7f85affff640 1 --2- 172.21.15.98:0/3831245809 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85b015e0c0 0x7f85b00b6db0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f85a8035040 tx=0x7f85a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:53.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.818+0000 7f8597fff640 1 -- 172.21.15.98:0/3831245809 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a8037040 con 0x7f85b015e0c0 2024-05-09T10:50:53.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.818+0000 7f8597fff640 1 -- 172.21.15.98:0/3831245809 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a8042070 con 0x7f85b015e0c0 2024-05-09T10:50:53.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.818+0000 7f8597fff640 1 -- 172.21.15.98:0/3831245809 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a8030b50 con 0x7f85b015e0c0 2024-05-09T10:50:53.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.818+0000 7f85b5b22640 1 -- 172.21.15.98:0/3831245809 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b006eb60 con 0x7f85b015e0c0 2024-05-09T10:50:53.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.818+0000 7f85b5b22640 1 -- 172.21.15.98:0/3831245809 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b006f120 con 0x7f85b015e0c0 2024-05-09T10:50:53.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.819+0000 7f85b5b22640 1 -- 172.21.15.98:0/3831245809 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b0068c70 con 0x7f85b015e0c0 2024-05-09T10:50:53.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.819+0000 7f8597fff640 1 -- 172.21.15.98:0/3831245809 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f85a8030cf0 con 0x7f85b015e0c0 2024-05-09T10:50:53.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.819+0000 7f8597fff640 1 --2- 172.21.15.98:0/3831245809 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f859c06b4d0 0x7f859c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:53.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.820+0000 7f8597fff640 1 -- 172.21.15.98:0/3831245809 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85a80aad30 con 0x7f85b015e0c0 2024-05-09T10:50:53.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.820+0000 7f85af7fe640 1 --2- 172.21.15.98:0/3831245809 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f859c06b4d0 0x7f859c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:53.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.820+0000 7f85af7fe640 1 --2- 172.21.15.98:0/3831245809 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f859c06b4d0 0x7f859c06d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f85a00070b0 tx=0x7f85a0007040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:53.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:53.824+0000 7f8597fff640 1 -- 172.21.15.98:0/3831245809 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f85b0068c70 con 0x7f85b015e0c0 2024-05-09T10:50:54.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.028+0000 7f85b5b22640 1 -- 172.21.15.98:0/3831245809 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85b00d14f0 con 0x7f85b015e0c0 2024-05-09T10:50:54.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.028+0000 7f8597fff640 1 -- 172.21.15.98:0/3831245809 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f85a8032e20 con 0x7f85b015e0c0 2024-05-09T10:50:54.023 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:54.023 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2903091926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2903091926},{"type":"v1","addr":"172.21.15.98:6837","nonce":2903091926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/968936316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":968936316},{"type":"v1","addr":"172.21.15.105:6834","nonce":968936316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/937700297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":937700297},{"type":"v1","addr":"172.21.15.98:6835","nonce":937700297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T10:50:19.544432+0000","modified":"2024-05-09T10:50:21.512942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6835/1780062557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1780062557},{"type":"v1","addr":"172.21.15.105:6835","nonce":1780062557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T10:50:54.023 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 6 2024-05-09T10:50:54.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.030+0000 7f85b5b22640 1 -- 172.21.15.98:0/3831245809 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f859c06b4d0 msgr2=0x7f859c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:54.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.030+0000 7f85b5b22640 1 --2- 172.21.15.98:0/3831245809 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f859c06b4d0 0x7f859c06d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f85a00070b0 tx=0x7f85a0007040 comp rx=0 tx=0).stop 2024-05-09T10:50:54.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.030+0000 7f85b5b22640 1 -- 172.21.15.98:0/3831245809 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85b015e0c0 msgr2=0x7f85b00b6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:54.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.030+0000 7f85b5b22640 1 --2- 172.21.15.98:0/3831245809 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85b015e0c0 0x7f85b00b6db0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f85a8035040 tx=0x7f85a80046c0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.030+0000 7f85b5b22640 1 -- 172.21.15.98:0/3831245809 shutdown_connections 2024-05-09T10:50:54.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.030+0000 7f85b5b22640 1 --2- 172.21.15.98:0/3831245809 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f859c06b4d0 0x7f859c06d990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.030+0000 7f85b5b22640 1 --2- 172.21.15.98:0/3831245809 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85b015e0c0 0x7f85b00b6db0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.030+0000 7f85b5b22640 1 --2- 172.21.15.98:0/3831245809 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85b015dce0 0x7f85b01f4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.030+0000 7f85b5b22640 1 --2- 172.21.15.98:0/3831245809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85b006cac0 0x7f85b0158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.030+0000 7f85b5b22640 1 -- 172.21.15.98:0/3831245809 >> 172.21.15.98:0/3831245809 conn(0x7f85b00b34e0 msgr2=0x7f85b0159e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:54.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.031+0000 7f85b5b22640 1 -- 172.21.15.98:0/3831245809 shutdown_connections 2024-05-09T10:50:54.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.031+0000 7f85b5b22640 1 -- 172.21.15.98:0/3831245809 wait complete. 2024-05-09T10:50:54.036 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:54.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.140+0000 7f2e2b2ac640 1 Processor -- start 2024-05-09T10:50:54.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.140+0000 7f2e2b2ac640 1 -- start start 2024-05-09T10:50:54.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.141+0000 7f2e2b2ac640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2e2415a3f0 0x7f2e2415de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:54.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.141+0000 7f2e2b2ac640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2e2406dc50 0x7f2e2406c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:54.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.141+0000 7f2e2b2ac640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e24071ad0 con 0x7f2e240b40b0 2024-05-09T10:50:54.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.141+0000 7f2e2b2ac640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e240718a0 con 0x7f2e2415a3f0 2024-05-09T10:50:54.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.141+0000 7f2e2b2ac640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e240af120 con 0x7f2e2406dc50 2024-05-09T10:50:54.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.141+0000 7f2e29048640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2e2415a3f0 0x7f2e2415de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:54.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.141+0000 7f2e28847640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2e240b40b0 0x7f2e2406d540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38818/0 (socket says 172.21.15.98:38818) 2024-05-09T10:50:54.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.141+0000 7f2e28847640 1 -- 172.21.15.98:0/971685052 learned_addr learned my addr 172.21.15.98:0/971685052 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:54.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.141+0000 7f2e29048640 1 -- 172.21.15.98:0/971685052 >> v2:172.21.15.105:3300/0 conn(0x7f2e2406dc50 msgr2=0x7f2e2406c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:54.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.141+0000 7f2e29048640 1 --2- 172.21.15.98:0/971685052 >> v2:172.21.15.105:3300/0 conn(0x7f2e2406dc50 0x7f2e2406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.141+0000 7f2e29048640 1 -- 172.21.15.98:0/971685052 >> v1:172.21.15.98:6789/0 conn(0x7f2e240b40b0 legacy=0x7f2e2406d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:54.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.141+0000 7f2e17fff640 1 -- 172.21.15.98:0/971685052 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2652967402 0 0) 0x7f2e24071ad0 con 0x7f2e240b40b0 2024-05-09T10:50:54.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.141+0000 7f2e29048640 1 -- 172.21.15.98:0/971685052 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e24071db0 con 0x7f2e2415a3f0 2024-05-09T10:50:54.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.142+0000 7f2e29048640 1 --2- 172.21.15.98:0/971685052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e2415a3f0 0x7f2e2415de20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2e2415dd60 tx=0x7f2e2002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d863fc08b6e2e40e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:54.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.142+0000 7f2e17fff640 1 -- 172.21.15.98:0/971685052 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e20037070 con 0x7f2e2415a3f0 2024-05-09T10:50:54.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.142+0000 7f2e17fff640 1 -- 172.21.15.98:0/971685052 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e20032440 con 0x7f2e2415a3f0 2024-05-09T10:50:54.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.142+0000 7f2e17fff640 1 -- 172.21.15.98:0/971685052 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2002ae60 con 0x7f2e2415a3f0 2024-05-09T10:50:54.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.142+0000 7f2e2b2ac640 1 -- 172.21.15.98:0/971685052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e2415a3f0 msgr2=0x7f2e2415de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:54.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.142+0000 7f2e2b2ac640 1 --2- 172.21.15.98:0/971685052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e2415a3f0 0x7f2e2415de20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2e2415dd60 tx=0x7f2e2002a110 comp rx=0 tx=0).stop 2024-05-09T10:50:54.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.142+0000 7f2e2b2ac640 1 -- 172.21.15.98:0/971685052 shutdown_connections 2024-05-09T10:50:54.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.142+0000 7f2e2b2ac640 1 --2- 172.21.15.98:0/971685052 >> v2:172.21.15.105:3300/0 conn(0x7f2e2406dc50 0x7f2e2406c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.142+0000 7f2e2b2ac640 1 --2- 172.21.15.98:0/971685052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e2415a3f0 0x7f2e2415de20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.142+0000 7f2e2b2ac640 1 -- 172.21.15.98:0/971685052 >> 172.21.15.98:0/971685052 conn(0x7f2e2406abe0 msgr2=0x7f2e2406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:54.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.143+0000 7f2e2b2ac640 1 -- 172.21.15.98:0/971685052 shutdown_connections 2024-05-09T10:50:54.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.143+0000 7f2e2b2ac640 1 -- 172.21.15.98:0/971685052 wait complete. 2024-05-09T10:50:54.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.144+0000 7f2e2b2ac640 1 Processor -- start 2024-05-09T10:50:54.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.144+0000 7f2e2b2ac640 1 -- start start 2024-05-09T10:50:54.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.144+0000 7f2e2b2ac640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e2406dc50 0x7f2e24154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:54.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.144+0000 7f2e2b2ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e240b40b0 0x7f2e24144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:54.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.144+0000 7f2e2b2ac640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2e2415a3f0 0x7f2e240cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:54.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.144+0000 7f2e2b2ac640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e240aec20 con 0x7f2e2406dc50 2024-05-09T10:50:54.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.144+0000 7f2e2b2ac640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e24072400 con 0x7f2e240b40b0 2024-05-09T10:50:54.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.144+0000 7f2e2b2ac640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e240aeda0 con 0x7f2e2415a3f0 2024-05-09T10:50:54.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.145+0000 7f2e29048640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e2406dc50 0x7f2e24154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:54.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.145+0000 7f2e29048640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e2406dc50 0x7f2e24154530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35414/0 (socket says 172.21.15.98:35414) 2024-05-09T10:50:54.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.145+0000 7f2e29048640 1 -- 172.21.15.98:0/499299397 learned_addr learned my addr 172.21.15.98:0/499299397 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:54.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.145+0000 7f2e28847640 1 --2- 172.21.15.98:0/499299397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e240b40b0 0x7f2e24144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:54.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.145+0000 7f2e29849640 1 --2- 172.21.15.98:0/499299397 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2e2415a3f0 0x7f2e240cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:54.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.145+0000 7f2e29048640 1 -- 172.21.15.98:0/499299397 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2e2415a3f0 msgr2=0x7f2e240cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:54.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.145+0000 7f2e29048640 1 --2- 172.21.15.98:0/499299397 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2e2415a3f0 0x7f2e240cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.145+0000 7f2e29048640 1 -- 172.21.15.98:0/499299397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e240b40b0 msgr2=0x7f2e24144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:54.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.145+0000 7f2e29048640 1 --2- 172.21.15.98:0/499299397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e240b40b0 0x7f2e24144320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.145+0000 7f2e29048640 1 -- 172.21.15.98:0/499299397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e24144860 con 0x7f2e2406dc50 2024-05-09T10:50:54.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.145+0000 7f2e29048640 1 --2- 172.21.15.98:0/499299397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e2406dc50 0x7f2e24154530 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2e20000c00 tx=0x7f2e200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:54.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.145+0000 7f2e15ffb640 1 -- 172.21.15.98:0/499299397 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e20041070 con 0x7f2e2406dc50 2024-05-09T10:50:54.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.145+0000 7f2e15ffb640 1 -- 172.21.15.98:0/499299397 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2003c950 con 0x7f2e2406dc50 2024-05-09T10:50:54.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.145+0000 7f2e15ffb640 1 -- 172.21.15.98:0/499299397 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e20037040 con 0x7f2e2406dc50 2024-05-09T10:50:54.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.145+0000 7f2e2b2ac640 1 -- 172.21.15.98:0/499299397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e24153490 con 0x7f2e2406dc50 2024-05-09T10:50:54.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.145+0000 7f2e2b2ac640 1 -- 172.21.15.98:0/499299397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e24153a50 con 0x7f2e2406dc50 2024-05-09T10:50:54.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.146+0000 7f2e2b2ac640 1 -- 172.21.15.98:0/499299397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e24069cb0 con 0x7f2e2406dc50 2024-05-09T10:50:54.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.147+0000 7f2e15ffb640 1 -- 172.21.15.98:0/499299397 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e20044070 con 0x7f2e2406dc50 2024-05-09T10:50:54.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.147+0000 7f2e15ffb640 1 --2- 172.21.15.98:0/499299397 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2e0806b500 0x7f2e0806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:54.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.147+0000 7f2e28847640 1 --2- 172.21.15.98:0/499299397 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2e0806b500 0x7f2e0806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:54.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.147+0000 7f2e15ffb640 1 -- 172.21.15.98:0/499299397 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e200ab000 con 0x7f2e2406dc50 2024-05-09T10:50:54.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.148+0000 7f2e28847640 1 --2- 172.21.15.98:0/499299397 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2e0806b500 0x7f2e0806d9c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2e24152e30 tx=0x7f2e18007b50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:54.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.151+0000 7f2e15ffb640 1 -- 172.21.15.98:0/499299397 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e20079fa0 con 0x7f2e2406dc50 2024-05-09T10:50:54.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.325+0000 7f2e2b2ac640 1 -- 172.21.15.98:0/499299397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e24074480 con 0x7f2e2406dc50 2024-05-09T10:50:54.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.326+0000 7f2e15ffb640 1 -- 172.21.15.98:0/499299397 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f2e200798a0 con 0x7f2e2406dc50 2024-05-09T10:50:54.320 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:54.320 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":20,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:21.176419+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:50:17.323444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T10:50:18.482315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:54.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.328+0000 7f2e2b2ac640 1 -- 172.21.15.98:0/499299397 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2e0806b500 msgr2=0x7f2e0806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:54.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.328+0000 7f2e2b2ac640 1 --2- 172.21.15.98:0/499299397 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2e0806b500 0x7f2e0806d9c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2e24152e30 tx=0x7f2e18007b50 comp rx=0 tx=0).stop 2024-05-09T10:50:54.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.328+0000 7f2e2b2ac640 1 -- 172.21.15.98:0/499299397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e2406dc50 msgr2=0x7f2e24154530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:54.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.328+0000 7f2e2b2ac640 1 --2- 172.21.15.98:0/499299397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e2406dc50 0x7f2e24154530 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2e20000c00 tx=0x7f2e200046c0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.328+0000 7f2e2b2ac640 1 -- 172.21.15.98:0/499299397 shutdown_connections 2024-05-09T10:50:54.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.328+0000 7f2e2b2ac640 1 --2- 172.21.15.98:0/499299397 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2e0806b500 0x7f2e0806d9c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.328+0000 7f2e2b2ac640 1 --2- 172.21.15.98:0/499299397 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2e2415a3f0 0x7f2e240cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.328+0000 7f2e2b2ac640 1 --2- 172.21.15.98:0/499299397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e240b40b0 0x7f2e24144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.328+0000 7f2e2b2ac640 1 --2- 172.21.15.98:0/499299397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e2406dc50 0x7f2e24154530 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.328+0000 7f2e2b2ac640 1 -- 172.21.15.98:0/499299397 >> 172.21.15.98:0/499299397 conn(0x7f2e2406abe0 msgr2=0x7f2e24155340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:54.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.328+0000 7f2e2b2ac640 1 -- 172.21.15.98:0/499299397 shutdown_connections 2024-05-09T10:50:54.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.328+0000 7f2e2b2ac640 1 -- 172.21.15.98:0/499299397 wait complete. 2024-05-09T10:50:54.334 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T10:50:07.396423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T10:50:17.323444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T10:50:18.482315+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T10:50:54.334 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T10:50:54.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.442+0000 7f89bdc7d640 1 Processor -- start 2024-05-09T10:50:54.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.442+0000 7f89bdc7d640 1 -- start start 2024-05-09T10:50:54.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.443+0000 7f89bdc7d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f89b815db90 0x7f89b80af720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:54.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.443+0000 7f89bdc7d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89b80720d0 0x7f89b806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:54.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.443+0000 7f89bdc7d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89b8072690 con 0x7f89b806abe0 2024-05-09T10:50:54.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.443+0000 7f89bdc7d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b80afc60 con 0x7f89b815db90 2024-05-09T10:50:54.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.443+0000 7f89bdc7d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b80b5d70 con 0x7f89b80720d0 2024-05-09T10:50:54.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.443+0000 7f89b77fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f89b815db90 0x7f89b80af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:54.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.443+0000 7f89b7fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f89b806abe0 0x7f89b806aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.98:41006/0 (socket says 172.21.15.98:41006) 2024-05-09T10:50:54.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.443+0000 7f89b6ffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89b80720d0 0x7f89b806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:54.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.443+0000 7f89b77fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f89b815db90 0x7f89b80af720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.98:52410/0 (socket says 172.21.15.98:52410) 2024-05-09T10:50:54.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.443+0000 7f89b7fff640 1 -- 172.21.15.98:0/507283916 learned_addr learned my addr 172.21.15.98:0/507283916 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:54.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.444+0000 7f89b77fe640 1 -- 172.21.15.98:0/507283916 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89b80720d0 msgr2=0x7f89b806d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:54.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.444+0000 7f89b77fe640 1 --2- 172.21.15.98:0/507283916 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89b80720d0 0x7f89b806d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.444+0000 7f89b77fe640 1 -- 172.21.15.98:0/507283916 >> v1:172.21.15.105:6789/0 conn(0x7f89b806abe0 legacy=0x7f89b806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:54.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.444+0000 7f89b77fe640 1 -- 172.21.15.98:0/507283916 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b80b57b0 con 0x7f89b815db90 2024-05-09T10:50:54.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.444+0000 7f89b6ffd640 1 --2- 172.21.15.98:0/507283916 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89b80720d0 0x7f89b806d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:50:54.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.444+0000 7f89b77fe640 1 --2- 172.21.15.98:0/507283916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89b815db90 0x7f89b80af720 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009d90 tx=0x7f89a802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36b426137f8fb78c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:54.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.445+0000 7f89b67fc640 1 -- 172.21.15.98:0/507283916 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8031600 con 0x7f89b815db90 2024-05-09T10:50:54.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.445+0000 7f89b67fc640 1 -- 172.21.15.98:0/507283916 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a80317a0 con 0x7f89b815db90 2024-05-09T10:50:54.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.445+0000 7f89b67fc640 1 -- 172.21.15.98:0/507283916 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8031b70 con 0x7f89b815db90 2024-05-09T10:50:54.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.445+0000 7f89bdc7d640 1 -- 172.21.15.98:0/507283916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89b815db90 msgr2=0x7f89b80af720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:54.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.445+0000 7f89bdc7d640 1 --2- 172.21.15.98:0/507283916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89b815db90 0x7f89b80af720 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009d90 tx=0x7f89a802a5d0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.445+0000 7f89bdc7d640 1 -- 172.21.15.98:0/507283916 shutdown_connections 2024-05-09T10:50:54.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.445+0000 7f89bdc7d640 1 --2- 172.21.15.98:0/507283916 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89b80720d0 0x7f89b806d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.445+0000 7f89bdc7d640 1 --2- 172.21.15.98:0/507283916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89b815db90 0x7f89b80af720 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.445+0000 7f89bdc7d640 1 -- 172.21.15.98:0/507283916 >> 172.21.15.98:0/507283916 conn(0x7f89b8066fd0 msgr2=0x7f89b806bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:54.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.445+0000 7f89bdc7d640 1 -- 172.21.15.98:0/507283916 shutdown_connections 2024-05-09T10:50:54.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.445+0000 7f89bdc7d640 1 -- 172.21.15.98:0/507283916 wait complete. 2024-05-09T10:50:54.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.446+0000 7f89bdc7d640 1 Processor -- start 2024-05-09T10:50:54.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.446+0000 7f89bdc7d640 1 -- start start 2024-05-09T10:50:54.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.447+0000 7f89bdc7d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89b806abe0 0x7f89b81e76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:54.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.447+0000 7f89bdc7d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89b80720d0 0x7f89b81e8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:54.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.447+0000 7f89bdc7d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89b815db90 0x7f89b815f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:54.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.447+0000 7f89bdc7d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b806d930 con 0x7f89b80720d0 2024-05-09T10:50:54.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.447+0000 7f89bdc7d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b80d0670 con 0x7f89b806abe0 2024-05-09T10:50:54.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.447+0000 7f89bdc7d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b815df70 con 0x7f89b815db90 2024-05-09T10:50:54.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.447+0000 7f89b6ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89b80720d0 0x7f89b81e8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:54.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.447+0000 7f89b6ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89b80720d0 0x7f89b81e8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35430/0 (socket says 172.21.15.98:35430) 2024-05-09T10:50:54.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.447+0000 7f89b6ffd640 1 -- 172.21.15.98:0/4280128780 learned_addr learned my addr 172.21.15.98:0/4280128780 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:54.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.447+0000 7f89b77fe640 1 --2- 172.21.15.98:0/4280128780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89b806abe0 0x7f89b81e76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:54.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.448+0000 7f89b7fff640 1 --2- 172.21.15.98:0/4280128780 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89b815db90 0x7f89b815f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:54.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.448+0000 7f89b6ffd640 1 -- 172.21.15.98:0/4280128780 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89b815db90 msgr2=0x7f89b815f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:54.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.448+0000 7f89b6ffd640 1 --2- 172.21.15.98:0/4280128780 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89b815db90 0x7f89b815f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.448+0000 7f89b6ffd640 1 -- 172.21.15.98:0/4280128780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89b806abe0 msgr2=0x7f89b81e76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:54.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.448+0000 7f89b6ffd640 1 --2- 172.21.15.98:0/4280128780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89b806abe0 0x7f89b81e76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.448+0000 7f89b6ffd640 1 -- 172.21.15.98:0/4280128780 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b81e7cd0 con 0x7f89b80720d0 2024-05-09T10:50:54.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.448+0000 7f89b6ffd640 1 --2- 172.21.15.98:0/4280128780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89b80720d0 0x7f89b81e8d70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00ebe0 tx=0x7f89ac00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:54.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.448+0000 7f899ffff640 1 -- 172.21.15.98:0/4280128780 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac017070 con 0x7f89b80720d0 2024-05-09T10:50:54.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.448+0000 7f899ffff640 1 -- 172.21.15.98:0/4280128780 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ac00cc30 con 0x7f89b80720d0 2024-05-09T10:50:54.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.448+0000 7f89bdc7d640 1 -- 172.21.15.98:0/4280128780 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b81e7fb0 con 0x7f89b80720d0 2024-05-09T10:50:54.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.448+0000 7f899ffff640 1 -- 172.21.15.98:0/4280128780 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac013470 con 0x7f89b80720d0 2024-05-09T10:50:54.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.448+0000 7f89bdc7d640 1 -- 172.21.15.98:0/4280128780 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b81e8570 con 0x7f89b80720d0 2024-05-09T10:50:54.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.449+0000 7f89bdc7d640 1 -- 172.21.15.98:0/4280128780 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b806ebe0 con 0x7f89b80720d0 2024-05-09T10:50:54.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.449+0000 7f899ffff640 1 -- 172.21.15.98:0/4280128780 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f89ac01a070 con 0x7f89b80720d0 2024-05-09T10:50:54.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.450+0000 7f899ffff640 1 --2- 172.21.15.98:0/4280128780 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f89a406b4d0 0x7f89a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:54.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.450+0000 7f89b77fe640 1 --2- 172.21.15.98:0/4280128780 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f89a406b4d0 0x7f89a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:54.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.450+0000 7f899ffff640 1 -- 172.21.15.98:0/4280128780 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89ac08a4f0 con 0x7f89b80720d0 2024-05-09T10:50:54.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.450+0000 7f89b77fe640 1 --2- 172.21.15.98:0/4280128780 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f89a406b4d0 0x7f89a406d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f89a8034000 tx=0x7f89a8033040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:54.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.454+0000 7f899ffff640 1 -- 172.21.15.98:0/4280128780 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f89b806ebe0 con 0x7f89b80720d0 2024-05-09T10:50:54.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.623+0000 7f89bdc7d640 1 -- 172.21.15.98:0/4280128780 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f89b8072dd0 con 0x7f89b80720d0 2024-05-09T10:50:54.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.624+0000 7f899ffff640 1 -- 172.21.15.98:0/4280128780 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f89ac059370 con 0x7f89b80720d0 2024-05-09T10:50:54.618 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-05-09T10:50:54.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.626+0000 7f89bdc7d640 1 -- 172.21.15.98:0/4280128780 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f89a406b4d0 msgr2=0x7f89a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:54.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.626+0000 7f89bdc7d640 1 --2- 172.21.15.98:0/4280128780 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f89a406b4d0 0x7f89a406d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f89a8034000 tx=0x7f89a8033040 comp rx=0 tx=0).stop 2024-05-09T10:50:54.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.626+0000 7f89bdc7d640 1 -- 172.21.15.98:0/4280128780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89b80720d0 msgr2=0x7f89b81e8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:54.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.626+0000 7f89bdc7d640 1 --2- 172.21.15.98:0/4280128780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89b80720d0 0x7f89b81e8d70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f89ac00ebe0 tx=0x7f89ac00c520 comp rx=0 tx=0).stop 2024-05-09T10:50:54.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.626+0000 7f89bdc7d640 1 -- 172.21.15.98:0/4280128780 shutdown_connections 2024-05-09T10:50:54.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.626+0000 7f89bdc7d640 1 --2- 172.21.15.98:0/4280128780 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f89a406b4d0 0x7f89a406d990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.626+0000 7f89bdc7d640 1 --2- 172.21.15.98:0/4280128780 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89b815db90 0x7f89b815f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.626+0000 7f89bdc7d640 1 --2- 172.21.15.98:0/4280128780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89b80720d0 0x7f89b81e8d70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.626+0000 7f89bdc7d640 1 --2- 172.21.15.98:0/4280128780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89b806abe0 0x7f89b81e76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.626+0000 7f89bdc7d640 1 -- 172.21.15.98:0/4280128780 >> 172.21.15.98:0/4280128780 conn(0x7f89b8066fd0 msgr2=0x7f89b80cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:54.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.626+0000 7f89bdc7d640 1 -- 172.21.15.98:0/4280128780 shutdown_connections 2024-05-09T10:50:54.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.626+0000 7f89bdc7d640 1 -- 172.21.15.98:0/4280128780 wait complete. 2024-05-09T10:50:54.631 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T10:50:54.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.739+0000 7f1199e9a640 1 Processor -- start 2024-05-09T10:50:54.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.739+0000 7f1199e9a640 1 -- start start 2024-05-09T10:50:54.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.740+0000 7f1199e9a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f11940ad880 0x7f11940b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:54.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.740+0000 7f1199e9a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f11940b5080 0x7f11940adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:54.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.740+0000 7f1199e9a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11940afa70 con 0x7f11940ae1a0 2024-05-09T10:50:54.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.740+0000 7f1199e9a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f119415e200 con 0x7f11940b5080 2024-05-09T10:50:54.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.740+0000 7f1199e9a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f119415e4c0 con 0x7f11940ad880 2024-05-09T10:50:54.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.740+0000 7f1192ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f11940b5080 0x7f11940adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:54.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.740+0000 7f1192ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f11940b5080 0x7f11940adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35444/0 (socket says 172.21.15.98:35444) 2024-05-09T10:50:54.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.740+0000 7f1193fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f11940ae1a0 0x7f119406abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38820/0 (socket says 172.21.15.98:38820) 2024-05-09T10:50:54.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.740+0000 7f1192ffd640 1 -- 172.21.15.98:0/224143760 learned_addr learned my addr 172.21.15.98:0/224143760 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:54.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.740+0000 7f11937fe640 1 --2- 172.21.15.98:0/224143760 >> v2:172.21.15.105:3300/0 conn(0x7f11940ad880 0x7f11940b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:54.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.740+0000 7f1192ffd640 1 -- 172.21.15.98:0/224143760 >> v2:172.21.15.105:3300/0 conn(0x7f11940ad880 msgr2=0x7f11940b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:54.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.740+0000 7f1192ffd640 1 --2- 172.21.15.98:0/224143760 >> v2:172.21.15.105:3300/0 conn(0x7f11940ad880 0x7f11940b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.740+0000 7f1192ffd640 1 -- 172.21.15.98:0/224143760 >> v1:172.21.15.98:6789/0 conn(0x7f11940ae1a0 legacy=0x7f119406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:54.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.740+0000 7f1192ffd640 1 -- 172.21.15.98:0/224143760 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119415e7a0 con 0x7f11940b5080 2024-05-09T10:50:54.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.740+0000 7f11927fc640 1 -- 172.21.15.98:0/224143760 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 264022183 0 0) 0x7f118c001490 con 0x7f11940ae1a0 2024-05-09T10:50:54.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.741+0000 7f1192ffd640 1 --2- 172.21.15.98:0/224143760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11940b5080 0x7f11940adc60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1188009870 tx=0x7f118802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2c4714cd2eb6cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:54.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.741+0000 7f11927fc640 1 -- 172.21.15.98:0/224143760 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1188037070 con 0x7f11940b5080 2024-05-09T10:50:54.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.741+0000 7f11927fc640 1 -- 172.21.15.98:0/224143760 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1188032440 con 0x7f11940b5080 2024-05-09T10:50:54.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.741+0000 7f11927fc640 1 -- 172.21.15.98:0/224143760 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1188032810 con 0x7f11940b5080 2024-05-09T10:50:54.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.742+0000 7f1199e9a640 1 -- 172.21.15.98:0/224143760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11940b5080 msgr2=0x7f11940adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:54.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.742+0000 7f1199e9a640 1 --2- 172.21.15.98:0/224143760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11940b5080 0x7f11940adc60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1188009870 tx=0x7f118802a180 comp rx=0 tx=0).stop 2024-05-09T10:50:54.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.742+0000 7f1199e9a640 1 -- 172.21.15.98:0/224143760 shutdown_connections 2024-05-09T10:50:54.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.742+0000 7f1199e9a640 1 --2- 172.21.15.98:0/224143760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11940b5080 0x7f11940adc60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.742+0000 7f1199e9a640 1 --2- 172.21.15.98:0/224143760 >> v2:172.21.15.105:3300/0 conn(0x7f11940ad880 0x7f11940b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.742+0000 7f1199e9a640 1 -- 172.21.15.98:0/224143760 >> 172.21.15.98:0/224143760 conn(0x7f119406cbf0 msgr2=0x7f119406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:54.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.742+0000 7f1199e9a640 1 -- 172.21.15.98:0/224143760 shutdown_connections 2024-05-09T10:50:54.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.742+0000 7f1199e9a640 1 -- 172.21.15.98:0/224143760 wait complete. 2024-05-09T10:50:54.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.743+0000 7f1199e9a640 1 Processor -- start 2024-05-09T10:50:54.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.743+0000 7f1199e9a640 1 -- start start 2024-05-09T10:50:54.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.744+0000 7f1199e9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11940ad880 0x7f11941551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:54.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.744+0000 7f1199e9a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11940ae1a0 0x7f1194152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:54.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.744+0000 7f1199e9a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11940b5080 0x7f1194153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:54.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.744+0000 7f1199e9a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11940ae630 con 0x7f11940b5080 2024-05-09T10:50:54.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.744+0000 7f1199e9a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119415ed60 con 0x7f11940ad880 2024-05-09T10:50:54.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.744+0000 7f1199e9a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11940b3520 con 0x7f11940ae1a0 2024-05-09T10:50:54.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.744+0000 7f1193fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11940b5080 0x7f1194153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:54.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.744+0000 7f1193fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11940b5080 0x7f1194153180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35450/0 (socket says 172.21.15.98:35450) 2024-05-09T10:50:54.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.744+0000 7f1193fff640 1 -- 172.21.15.98:0/3181702244 learned_addr learned my addr 172.21.15.98:0/3181702244 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:54.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.744+0000 7f11937fe640 1 --2- 172.21.15.98:0/3181702244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11940ad880 0x7f11941551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:54.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.744+0000 7f1192ffd640 1 --2- 172.21.15.98:0/3181702244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11940ae1a0 0x7f1194152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:54.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.744+0000 7f1193fff640 1 -- 172.21.15.98:0/3181702244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11940ae1a0 msgr2=0x7f1194152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:54.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.744+0000 7f1193fff640 1 --2- 172.21.15.98:0/3181702244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11940ae1a0 0x7f1194152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.744+0000 7f1193fff640 1 -- 172.21.15.98:0/3181702244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11940ad880 msgr2=0x7f11941551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:54.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.744+0000 7f1193fff640 1 --2- 172.21.15.98:0/3181702244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11940ad880 0x7f11941551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.744+0000 7f1193fff640 1 -- 172.21.15.98:0/3181702244 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11941536c0 con 0x7f11940b5080 2024-05-09T10:50:54.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.744+0000 7f11937fe640 1 --2- 172.21.15.98:0/3181702244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11940ad880 0x7f11941551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:54.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.744+0000 7f1192ffd640 1 --2- 172.21.15.98:0/3181702244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11940ae1a0 0x7f1194152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:54.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.744+0000 7f1193fff640 1 --2- 172.21.15.98:0/3181702244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11940b5080 0x7f1194153180 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f118c00c730 tx=0x7f118c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:54.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.744+0000 7f117bfff640 1 -- 172.21.15.98:0/3181702244 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c018070 con 0x7f11940b5080 2024-05-09T10:50:54.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.744+0000 7f117bfff640 1 -- 172.21.15.98:0/3181702244 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118c00ce40 con 0x7f11940b5080 2024-05-09T10:50:54.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.744+0000 7f1199e9a640 1 -- 172.21.15.98:0/3181702244 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1194153970 con 0x7f11940b5080 2024-05-09T10:50:54.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.744+0000 7f117bfff640 1 -- 172.21.15.98:0/3181702244 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c00a860 con 0x7f11940b5080 2024-05-09T10:50:54.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.745+0000 7f1199e9a640 1 -- 172.21.15.98:0/3181702244 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1194153f30 con 0x7f11940b5080 2024-05-09T10:50:54.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.746+0000 7f1199e9a640 1 -- 172.21.15.98:0/3181702244 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f119414e560 con 0x7f11940b5080 2024-05-09T10:50:54.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.746+0000 7f117bfff640 1 -- 172.21.15.98:0/3181702244 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f118c01b070 con 0x7f11940b5080 2024-05-09T10:50:54.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.747+0000 7f117bfff640 1 --2- 172.21.15.98:0/3181702244 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f118006b4d0 0x7f118006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:54.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.747+0000 7f11937fe640 1 --2- 172.21.15.98:0/3181702244 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f118006b4d0 0x7f118006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:54.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.747+0000 7f117bfff640 1 -- 172.21.15.98:0/3181702244 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f118c089780 con 0x7f11940b5080 2024-05-09T10:50:54.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.747+0000 7f11937fe640 1 --2- 172.21.15.98:0/3181702244 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f118006b4d0 0x7f118006d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1184006fd0 tx=0x7f1184008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:54.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.751+0000 7f117bfff640 1 -- 172.21.15.98:0/3181702244 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f119414e560 con 0x7f11940b5080 2024-05-09T10:50:54.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.921+0000 7f1199e9a640 1 -- 172.21.15.98:0/3181702244 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f119406dd40 con 0x7f11940b5080 2024-05-09T10:50:54.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.921+0000 7f117bfff640 1 -- 172.21.15.98:0/3181702244 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f11940afa70 con 0x7f11940b5080 2024-05-09T10:50:54.916 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2024-05-09T10:50:54.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.923+0000 7f1199e9a640 1 -- 172.21.15.98:0/3181702244 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f118006b4d0 msgr2=0x7f118006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:54.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.923+0000 7f1199e9a640 1 --2- 172.21.15.98:0/3181702244 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f118006b4d0 0x7f118006d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1184006fd0 tx=0x7f1184008040 comp rx=0 tx=0).stop 2024-05-09T10:50:54.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.924+0000 7f1199e9a640 1 -- 172.21.15.98:0/3181702244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11940b5080 msgr2=0x7f1194153180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:54.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.924+0000 7f1199e9a640 1 --2- 172.21.15.98:0/3181702244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11940b5080 0x7f1194153180 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f118c00c730 tx=0x7f118c00cbf0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.924+0000 7f1199e9a640 1 -- 172.21.15.98:0/3181702244 shutdown_connections 2024-05-09T10:50:54.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.924+0000 7f1199e9a640 1 --2- 172.21.15.98:0/3181702244 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f118006b4d0 0x7f118006d990 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.924+0000 7f1199e9a640 1 --2- 172.21.15.98:0/3181702244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11940b5080 0x7f1194153180 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.924+0000 7f1199e9a640 1 --2- 172.21.15.98:0/3181702244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11940ae1a0 0x7f1194152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.924+0000 7f1199e9a640 1 --2- 172.21.15.98:0/3181702244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11940ad880 0x7f11941551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:54.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.924+0000 7f1199e9a640 1 -- 172.21.15.98:0/3181702244 >> 172.21.15.98:0/3181702244 conn(0x7f119406cbf0 msgr2=0x7f119414ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:54.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.924+0000 7f1199e9a640 1 -- 172.21.15.98:0/3181702244 shutdown_connections 2024-05-09T10:50:54.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:54.924+0000 7f1199e9a640 1 -- 172.21.15.98:0/3181702244 wait complete. 2024-05-09T10:50:54.929 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T10:50:55.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.036+0000 7faea62cb640 1 Processor -- start 2024-05-09T10:50:55.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.036+0000 7faea62cb640 1 -- start start 2024-05-09T10:50:55.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.037+0000 7faea62cb640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faea00cfbb0 0x7faea0161b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.037+0000 7faea62cb640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea00d0090 con 0x7faea00b2d40 2024-05-09T10:50:55.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.037+0000 7faea62cb640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea00d0370 con 0x7faea0162070 2024-05-09T10:50:55.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.037+0000 7faea62cb640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea01624c0 con 0x7faea00cfbb0 2024-05-09T10:50:55.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.037+0000 7fae9ffff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7faea00b2d40 0x7faea006c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38836/0 (socket says 172.21.15.98:38836) 2024-05-09T10:50:55.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.037+0000 7fae9ffff640 1 -- 172.21.15.98:0/568280935 learned_addr learned my addr 172.21.15.98:0/568280935 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:55.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.037+0000 7fae9effd640 1 -- 172.21.15.98:0/568280935 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355523595 0 0) 0x7faea00d0090 con 0x7faea00b2d40 2024-05-09T10:50:55.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.037+0000 7fae9effd640 1 -- 172.21.15.98:0/568280935 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae88003520 con 0x7faea00b2d40 2024-05-09T10:50:55.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.037+0000 7fae9f7fe640 1 --2- 172.21.15.98:0/568280935 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faea00cfbb0 0x7faea0161b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.038+0000 7fae9effd640 1 -- 172.21.15.98:0/568280935 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 843730851 0 0) 0x7fae88003520 con 0x7faea00b2d40 2024-05-09T10:50:55.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.038+0000 7fae9effd640 1 -- 172.21.15.98:0/568280935 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faea00cfbb0 msgr2=0x7faea0161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:55.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.038+0000 7fae9effd640 1 --2- 172.21.15.98:0/568280935 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faea00cfbb0 0x7faea0161b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.038+0000 7fae9effd640 1 -- 172.21.15.98:0/568280935 >> v1:172.21.15.105:6789/0 conn(0x7faea0162070 legacy=0x7faea015d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:55.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.038+0000 7fae9effd640 1 -- 172.21.15.98:0/568280935 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea00d0650 con 0x7faea00b2d40 2024-05-09T10:50:55.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.038+0000 7fae9effd640 1 -- 172.21.15.98:0/568280935 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7fae90001670 con 0x7faea00b2d40 2024-05-09T10:50:55.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.038+0000 7fae9f7fe640 1 --2- 172.21.15.98:0/568280935 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faea00cfbb0 0x7faea0161b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:55.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.038+0000 7fae9effd640 1 -- 172.21.15.98:0/568280935 >> v1:172.21.15.98:6789/0 conn(0x7faea00b2d40 legacy=0x7faea006c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:55.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.038+0000 7fae9effd640 1 --2- 172.21.15.98:0/568280935 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faea00cfbb0 0x7fae880039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.038+0000 7fae9effd640 1 --2- 172.21.15.98:0/568280935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae88003fd0 0x7fae88004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.038+0000 7fae9effd640 1 --2- 172.21.15.98:0/568280935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae88004d40 0x7fae880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.038+0000 7fae9effd640 1 -- 172.21.15.98:0/568280935 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea01624c0 con 0x7fae88003fd0 2024-05-09T10:50:55.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.038+0000 7fae9effd640 1 -- 172.21.15.98:0/568280935 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8800bbb0 con 0x7fae88004d40 2024-05-09T10:50:55.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.038+0000 7fae9effd640 1 -- 172.21.15.98:0/568280935 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8800bdb0 con 0x7faea00cfbb0 2024-05-09T10:50:55.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.038+0000 7fae9f7fe640 1 --2- 172.21.15.98:0/568280935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae88003fd0 0x7fae88004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.038+0000 7fae9ffff640 1 --2- 172.21.15.98:0/568280935 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faea00cfbb0 0x7fae880039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.038+0000 7faea4868640 1 --2- 172.21.15.98:0/568280935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae88004d40 0x7fae880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.038+0000 7fae9f7fe640 1 -- 172.21.15.98:0/568280935 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faea00cfbb0 msgr2=0x7fae880039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:55.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.038+0000 7fae9f7fe640 1 --2- 172.21.15.98:0/568280935 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faea00cfbb0 0x7fae880039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.038+0000 7fae9f7fe640 1 -- 172.21.15.98:0/568280935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae88004d40 msgr2=0x7fae880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:55.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.038+0000 7fae9f7fe640 1 --2- 172.21.15.98:0/568280935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae88004d40 0x7fae880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.038+0000 7fae9f7fe640 1 -- 172.21.15.98:0/568280935 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae88003520 con 0x7fae88003fd0 2024-05-09T10:50:55.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.039+0000 7fae9f7fe640 1 --2- 172.21.15.98:0/568280935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae88003fd0 0x7fae88004800 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7faea00cffc0 tx=0x7fae88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14d4925ccc85a0b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:55.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.039+0000 7fae9effd640 1 -- 172.21.15.98:0/568280935 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9400f040 con 0x7fae88003fd0 2024-05-09T10:50:55.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.039+0000 7fae9effd640 1 -- 172.21.15.98:0/568280935 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9402ac90 con 0x7fae88003fd0 2024-05-09T10:50:55.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.039+0000 7fae9effd640 1 -- 172.21.15.98:0/568280935 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae94004250 con 0x7fae88003fd0 2024-05-09T10:50:55.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.039+0000 7faea62cb640 1 -- 172.21.15.98:0/568280935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae88003fd0 msgr2=0x7fae88004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:55.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.039+0000 7faea62cb640 1 --2- 172.21.15.98:0/568280935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae88003fd0 0x7fae88004800 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7faea00cffc0 tx=0x7fae88005830 comp rx=0 tx=0).stop 2024-05-09T10:50:55.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.040+0000 7faea4868640 1 -- 172.21.15.98:0/568280935 reap_dead start 2024-05-09T10:50:55.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.040+0000 7faea62cb640 1 -- 172.21.15.98:0/568280935 shutdown_connections 2024-05-09T10:50:55.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.040+0000 7faea62cb640 1 -- 172.21.15.98:0/568280935 >> 172.21.15.98:0/568280935 conn(0x7faea00d1970 msgr2=0x7faea00d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:55.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.040+0000 7faea62cb640 1 -- 172.21.15.98:0/568280935 shutdown_connections 2024-05-09T10:50:55.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.040+0000 7faea62cb640 1 -- 172.21.15.98:0/568280935 wait complete. 2024-05-09T10:50:55.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.041+0000 7faea62cb640 1 Processor -- start 2024-05-09T10:50:55.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.041+0000 7faea62cb640 1 -- start start 2024-05-09T10:50:55.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.042+0000 7faea62cb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae88003fd0 0x7faea01f4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.042+0000 7faea62cb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faea01f74f0 0x7faea015be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.042+0000 7faea62cb640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faea01f3730 0x7faea006f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.042+0000 7faea62cb640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea006d020 con 0x7fae88003fd0 2024-05-09T10:50:55.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.042+0000 7faea62cb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea00d0c10 con 0x7faea01f74f0 2024-05-09T10:50:55.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.042+0000 7faea62cb640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea00b3120 con 0x7faea01f3730 2024-05-09T10:50:55.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.042+0000 7fae9ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae88003fd0 0x7faea01f4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.042+0000 7fae9ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae88003fd0 0x7faea01f4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35480/0 (socket says 172.21.15.98:35480) 2024-05-09T10:50:55.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.042+0000 7fae9ffff640 1 -- 172.21.15.98:0/1023832359 learned_addr learned my addr 172.21.15.98:0/1023832359 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:55.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.042+0000 7fae9f7fe640 1 --2- 172.21.15.98:0/1023832359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faea01f74f0 0x7faea015be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.042+0000 7faea4868640 1 --2- 172.21.15.98:0/1023832359 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faea01f3730 0x7faea006f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.042+0000 7fae9ffff640 1 -- 172.21.15.98:0/1023832359 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faea01f3730 msgr2=0x7faea006f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:55.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.042+0000 7fae9ffff640 1 --2- 172.21.15.98:0/1023832359 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faea01f3730 0x7faea006f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.042+0000 7fae9ffff640 1 -- 172.21.15.98:0/1023832359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faea01f74f0 msgr2=0x7faea015be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:55.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.042+0000 7fae9ffff640 1 --2- 172.21.15.98:0/1023832359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faea01f74f0 0x7faea015be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.042+0000 7fae9ffff640 1 -- 172.21.15.98:0/1023832359 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea01f79d0 con 0x7fae88003fd0 2024-05-09T10:50:55.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.042+0000 7faea4868640 1 --2- 172.21.15.98:0/1023832359 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faea01f3730 0x7faea006f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:55.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.042+0000 7fae9ffff640 1 --2- 172.21.15.98:0/1023832359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae88003fd0 0x7faea01f4980 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fae900072c0 tx=0x7fae90007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:55.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.042+0000 7fae9cff9640 1 -- 172.21.15.98:0/1023832359 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9000f960 con 0x7fae88003fd0 2024-05-09T10:50:55.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.042+0000 7fae9cff9640 1 -- 172.21.15.98:0/1023832359 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae90018070 con 0x7fae88003fd0 2024-05-09T10:50:55.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.042+0000 7faea62cb640 1 -- 172.21.15.98:0/1023832359 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea01f7cb0 con 0x7fae88003fd0 2024-05-09T10:50:55.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.042+0000 7fae9cff9640 1 -- 172.21.15.98:0/1023832359 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9000fd60 con 0x7fae88003fd0 2024-05-09T10:50:55.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.042+0000 7faea62cb640 1 -- 172.21.15.98:0/1023832359 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea01f8270 con 0x7fae88003fd0 2024-05-09T10:50:55.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.043+0000 7faea62cb640 1 -- 172.21.15.98:0/1023832359 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea006ba50 con 0x7fae88003fd0 2024-05-09T10:50:55.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.044+0000 7fae9cff9640 1 -- 172.21.15.98:0/1023832359 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae9001d460 con 0x7fae88003fd0 2024-05-09T10:50:55.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.044+0000 7fae9cff9640 1 --2- 172.21.15.98:0/1023832359 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fae8c06b4d0 0x7fae8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.044+0000 7fae9cff9640 1 -- 172.21.15.98:0/1023832359 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae9008a2d0 con 0x7fae88003fd0 2024-05-09T10:50:55.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.044+0000 7fae9f7fe640 1 --2- 172.21.15.98:0/1023832359 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fae8c06b4d0 0x7fae8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.045+0000 7fae9f7fe640 1 --2- 172.21.15.98:0/1023832359 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fae8c06b4d0 0x7fae8c06d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fae94033000 tx=0x7fae9402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:55.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.048+0000 7fae9cff9640 1 -- 172.21.15.98:0/1023832359 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae90059150 con 0x7fae88003fd0 2024-05-09T10:50:55.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.220+0000 7faea62cb640 1 -- 172.21.15.98:0/1023832359 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faea009fa40 con 0x7fae88003fd0 2024-05-09T10:50:55.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.220+0000 7fae9cff9640 1 -- 172.21.15.98:0/1023832359 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fae90058a50 con 0x7fae88003fd0 2024-05-09T10:50:55.215 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2024-05-09T10:50:55.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.223+0000 7faea62cb640 1 -- 172.21.15.98:0/1023832359 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fae8c06b4d0 msgr2=0x7fae8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:55.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.223+0000 7faea62cb640 1 --2- 172.21.15.98:0/1023832359 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fae8c06b4d0 0x7fae8c06d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fae94033000 tx=0x7fae9402e000 comp rx=0 tx=0).stop 2024-05-09T10:50:55.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.223+0000 7faea62cb640 1 -- 172.21.15.98:0/1023832359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae88003fd0 msgr2=0x7faea01f4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:55.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.223+0000 7faea62cb640 1 --2- 172.21.15.98:0/1023832359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae88003fd0 0x7faea01f4980 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fae900072c0 tx=0x7fae90007780 comp rx=0 tx=0).stop 2024-05-09T10:50:55.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.223+0000 7faea62cb640 1 -- 172.21.15.98:0/1023832359 shutdown_connections 2024-05-09T10:50:55.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.223+0000 7faea62cb640 1 --2- 172.21.15.98:0/1023832359 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fae8c06b4d0 0x7fae8c06d990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.223+0000 7faea62cb640 1 --2- 172.21.15.98:0/1023832359 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faea01f3730 0x7faea006f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.223+0000 7faea62cb640 1 --2- 172.21.15.98:0/1023832359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faea01f74f0 0x7faea015be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.223+0000 7faea62cb640 1 --2- 172.21.15.98:0/1023832359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae88003fd0 0x7faea01f4980 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.223+0000 7faea62cb640 1 -- 172.21.15.98:0/1023832359 >> 172.21.15.98:0/1023832359 conn(0x7faea00d1970 msgr2=0x7faea00b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:55.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.223+0000 7faea62cb640 1 -- 172.21.15.98:0/1023832359 shutdown_connections 2024-05-09T10:50:55.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.223+0000 7faea62cb640 1 -- 172.21.15.98:0/1023832359 wait complete. 2024-05-09T10:50:55.228 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T10:50:55.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.334+0000 7f9b8a63f640 1 Processor -- start 2024-05-09T10:50:55.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.334+0000 7f9b8a63f640 1 -- start start 2024-05-09T10:50:55.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.335+0000 7f9b8a63f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9b840d0300 0x7f9b840af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.335+0000 7f9b8a63f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b8406f830 con 0x7f9b8415b9e0 2024-05-09T10:50:55.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.335+0000 7f9b8a63f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b8406fdb0 con 0x7f9b8406b2f0 2024-05-09T10:50:55.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.335+0000 7f9b8a63f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b84160210 con 0x7f9b840d0300 2024-05-09T10:50:55.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.335+0000 7f9b83fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f9b8415b9e0 0x7f9b8406abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38842/0 (socket says 172.21.15.98:38842) 2024-05-09T10:50:55.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.335+0000 7f9b83fff640 1 -- 172.21.15.98:0/1485256326 learned_addr learned my addr 172.21.15.98:0/1485256326 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:55.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.335+0000 7f9b88bdc640 1 --2- 172.21.15.98:0/1485256326 >> v2:172.21.15.98:3300/0 conn(0x7f9b840d0300 0x7f9b840af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.335+0000 7f9b837fe640 1 -- 172.21.15.98:0/1485256326 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808592971 0 0) 0x7f9b8406f830 con 0x7f9b8415b9e0 2024-05-09T10:50:55.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.335+0000 7f9b837fe640 1 -- 172.21.15.98:0/1485256326 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b70003520 con 0x7f9b8415b9e0 2024-05-09T10:50:55.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.335+0000 7f9b88bdc640 1 -- 172.21.15.98:0/1485256326 >> v1:172.21.15.105:6789/0 conn(0x7f9b8406b2f0 legacy=0x7f9b840cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:55.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.335+0000 7f9b88bdc640 1 -- 172.21.15.98:0/1485256326 >> v1:172.21.15.98:6789/0 conn(0x7f9b8415b9e0 legacy=0x7f9b8406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:55.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.335+0000 7f9b88bdc640 1 -- 172.21.15.98:0/1485256326 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8406fad0 con 0x7f9b840d0300 2024-05-09T10:50:55.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.336+0000 7f9b88bdc640 1 --2- 172.21.15.98:0/1485256326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b840d0300 0x7f9b840af6e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009d90 tx=0x7f9b7c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f31ec24907482 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:55.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.337+0000 7f9b837fe640 1 -- 172.21.15.98:0/1485256326 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c007740 con 0x7f9b840d0300 2024-05-09T10:50:55.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.338+0000 7f9b837fe640 1 -- 172.21.15.98:0/1485256326 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7c0078e0 con 0x7f9b840d0300 2024-05-09T10:50:55.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.338+0000 7f9b837fe640 1 -- 172.21.15.98:0/1485256326 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c007cb0 con 0x7f9b840d0300 2024-05-09T10:50:55.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.338+0000 7f9b8a63f640 1 -- 172.21.15.98:0/1485256326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b840d0300 msgr2=0x7f9b840af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:55.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.338+0000 7f9b8a63f640 1 --2- 172.21.15.98:0/1485256326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b840d0300 0x7f9b840af6e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009d90 tx=0x7f9b7c02a6a0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.338+0000 7f9b8a63f640 1 -- 172.21.15.98:0/1485256326 shutdown_connections 2024-05-09T10:50:55.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.338+0000 7f9b8a63f640 1 --2- 172.21.15.98:0/1485256326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b840d0300 0x7f9b840af6e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.338+0000 7f9b8a63f640 1 -- 172.21.15.98:0/1485256326 >> 172.21.15.98:0/1485256326 conn(0x7f9b840b24c0 msgr2=0x7f9b840b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:55.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.338+0000 7f9b8a63f640 1 -- 172.21.15.98:0/1485256326 shutdown_connections 2024-05-09T10:50:55.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.338+0000 7f9b8a63f640 1 -- 172.21.15.98:0/1485256326 wait complete. 2024-05-09T10:50:55.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.339+0000 7f9b8a63f640 1 Processor -- start 2024-05-09T10:50:55.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.340+0000 7f9b8a63f640 1 -- start start 2024-05-09T10:50:55.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.340+0000 7f9b8a63f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b8406b2f0 0x7f9b841499e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.340+0000 7f9b8a63f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b840d0300 0x7f9b8414ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.340+0000 7f9b8a63f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b8415b9e0 0x7f9b8415c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.340+0000 7f9b8a63f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8406d7c0 con 0x7f9b8406b2f0 2024-05-09T10:50:55.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.340+0000 7f9b8a63f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8406d640 con 0x7f9b840d0300 2024-05-09T10:50:55.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.340+0000 7f9b8a63f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8415bdc0 con 0x7f9b8415b9e0 2024-05-09T10:50:55.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.340+0000 7f9b83fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b8406b2f0 0x7f9b841499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.340+0000 7f9b83fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b8406b2f0 0x7f9b841499e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35504/0 (socket says 172.21.15.98:35504) 2024-05-09T10:50:55.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.340+0000 7f9b83fff640 1 -- 172.21.15.98:0/3135764333 learned_addr learned my addr 172.21.15.98:0/3135764333 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:55.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.340+0000 7f9b7bfff640 1 --2- 172.21.15.98:0/3135764333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b840d0300 0x7f9b8414ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.340+0000 7f9b88bdc640 1 --2- 172.21.15.98:0/3135764333 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b8415b9e0 0x7f9b8415c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.341+0000 7f9b83fff640 1 -- 172.21.15.98:0/3135764333 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b8415b9e0 msgr2=0x7f9b8415c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:55.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.341+0000 7f9b83fff640 1 --2- 172.21.15.98:0/3135764333 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b8415b9e0 0x7f9b8415c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.341+0000 7f9b83fff640 1 -- 172.21.15.98:0/3135764333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b840d0300 msgr2=0x7f9b8414ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:55.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.341+0000 7f9b83fff640 1 --2- 172.21.15.98:0/3135764333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b840d0300 0x7f9b8414ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.341+0000 7f9b83fff640 1 -- 172.21.15.98:0/3135764333 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8415cc30 con 0x7f9b8406b2f0 2024-05-09T10:50:55.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.341+0000 7f9b7bfff640 1 --2- 172.21.15.98:0/3135764333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b840d0300 0x7f9b8414ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:55.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.341+0000 7f9b83fff640 1 --2- 172.21.15.98:0/3135764333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b8406b2f0 0x7f9b841499e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400c390 tx=0x7f9b7400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:55.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.341+0000 7f9b817fa640 1 -- 172.21.15.98:0/3135764333 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74017070 con 0x7f9b8406b2f0 2024-05-09T10:50:55.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.341+0000 7f9b817fa640 1 -- 172.21.15.98:0/3135764333 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b74005ce0 con 0x7f9b8406b2f0 2024-05-09T10:50:55.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.341+0000 7f9b8a63f640 1 -- 172.21.15.98:0/3135764333 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8414b930 con 0x7f9b8406b2f0 2024-05-09T10:50:55.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.341+0000 7f9b817fa640 1 -- 172.21.15.98:0/3135764333 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b740072f0 con 0x7f9b8406b2f0 2024-05-09T10:50:55.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.341+0000 7f9b8a63f640 1 -- 172.21.15.98:0/3135764333 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b84149f20 con 0x7f9b8406b2f0 2024-05-09T10:50:55.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.342+0000 7f9b8a63f640 1 -- 172.21.15.98:0/3135764333 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b84069cb0 con 0x7f9b8406b2f0 2024-05-09T10:50:55.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.342+0000 7f9b817fa640 1 -- 172.21.15.98:0/3135764333 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b7401a070 con 0x7f9b8406b2f0 2024-05-09T10:50:55.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.343+0000 7f9b817fa640 1 --2- 172.21.15.98:0/3135764333 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9b6406b500 0x7f9b6406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.343+0000 7f9b7bfff640 1 --2- 172.21.15.98:0/3135764333 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9b6406b500 0x7f9b6406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.343+0000 7f9b817fa640 1 -- 172.21.15.98:0/3135764333 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b74089540 con 0x7f9b8406b2f0 2024-05-09T10:50:55.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.343+0000 7f9b7bfff640 1 --2- 172.21.15.98:0/3135764333 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9b6406b500 0x7f9b6406d9c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c003eb0 tx=0x7f9b6c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:55.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.347+0000 7f9b817fa640 1 -- 172.21.15.98:0/3135764333 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9b74058450 con 0x7f9b8406b2f0 2024-05-09T10:50:55.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.557+0000 7f9b8a63f640 1 -- 172.21.15.98:0/3135764333 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b840b3190 con 0x7f9b8406b2f0 2024-05-09T10:50:55.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.557+0000 7f9b817fa640 1 -- 172.21.15.98:0/3135764333 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f9b74057d50 con 0x7f9b8406b2f0 2024-05-09T10:50:55.552 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:55.552 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2903091926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2903091926},{"type":"v1","addr":"172.21.15.98:6837","nonce":2903091926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/968936316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":968936316},{"type":"v1","addr":"172.21.15.105:6834","nonce":968936316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/937700297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":937700297},{"type":"v1","addr":"172.21.15.98:6835","nonce":937700297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T10:50:19.544432+0000","modified":"2024-05-09T10:50:21.512942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6835/1780062557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1780062557},{"type":"v1","addr":"172.21.15.105:6835","nonce":1780062557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T10:50:55.553 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 6 2024-05-09T10:50:55.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.559+0000 7f9b8a63f640 1 -- 172.21.15.98:0/3135764333 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9b6406b500 msgr2=0x7f9b6406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:55.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.559+0000 7f9b8a63f640 1 --2- 172.21.15.98:0/3135764333 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9b6406b500 0x7f9b6406d9c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c003eb0 tx=0x7f9b6c009040 comp rx=0 tx=0).stop 2024-05-09T10:50:55.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.559+0000 7f9b8a63f640 1 -- 172.21.15.98:0/3135764333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b8406b2f0 msgr2=0x7f9b841499e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:55.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.559+0000 7f9b8a63f640 1 --2- 172.21.15.98:0/3135764333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b8406b2f0 0x7f9b841499e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400c390 tx=0x7f9b7400c850 comp rx=0 tx=0).stop 2024-05-09T10:50:55.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.559+0000 7f9b8a63f640 1 -- 172.21.15.98:0/3135764333 shutdown_connections 2024-05-09T10:50:55.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.559+0000 7f9b8a63f640 1 --2- 172.21.15.98:0/3135764333 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9b6406b500 0x7f9b6406d9c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.559+0000 7f9b8a63f640 1 --2- 172.21.15.98:0/3135764333 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9b8415b9e0 0x7f9b8415c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.559+0000 7f9b8a63f640 1 --2- 172.21.15.98:0/3135764333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b840d0300 0x7f9b8414ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.559+0000 7f9b8a63f640 1 --2- 172.21.15.98:0/3135764333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b8406b2f0 0x7f9b841499e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.559+0000 7f9b8a63f640 1 -- 172.21.15.98:0/3135764333 >> 172.21.15.98:0/3135764333 conn(0x7f9b840b24c0 msgr2=0x7f9b84157c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:55.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.559+0000 7f9b8a63f640 1 -- 172.21.15.98:0/3135764333 shutdown_connections 2024-05-09T10:50:55.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.560+0000 7f9b8a63f640 1 -- 172.21.15.98:0/3135764333 wait complete. 2024-05-09T10:50:55.565 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:55.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.670+0000 7fc228d89640 1 Processor -- start 2024-05-09T10:50:55.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.670+0000 7fc228d89640 1 -- start start 2024-05-09T10:50:55.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.670+0000 7fc228d89640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc22415f4e0 0x7fc2240ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.670+0000 7fc228d89640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc2240afe40 0x7fc2240b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.670+0000 7fc228d89640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2240aeda0 con 0x7fc2240b0810 2024-05-09T10:50:55.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.670+0000 7fc228d89640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2240af080 con 0x7fc22415f4e0 2024-05-09T10:50:55.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.670+0000 7fc228d89640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2240af360 con 0x7fc2240afe40 2024-05-09T10:50:55.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.671+0000 7fc22259c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc22415f4e0 0x7fc2240ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.671+0000 7fc222d9d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc2240b0810 0x7fc224164c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38856/0 (socket says 172.21.15.98:38856) 2024-05-09T10:50:55.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.671+0000 7fc222d9d640 1 -- 172.21.15.98:0/65566016 learned_addr learned my addr 172.21.15.98:0/65566016 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:55.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.671+0000 7fc21bfff640 1 --2- 172.21.15.98:0/65566016 >> v2:172.21.15.105:3300/0 conn(0x7fc2240afe40 0x7fc2240b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.671+0000 7fc22259c640 1 -- 172.21.15.98:0/65566016 >> v2:172.21.15.105:3300/0 conn(0x7fc2240afe40 msgr2=0x7fc2240b02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:55.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.671+0000 7fc22259c640 1 --2- 172.21.15.98:0/65566016 >> v2:172.21.15.105:3300/0 conn(0x7fc2240afe40 0x7fc2240b02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.671+0000 7fc22259c640 1 -- 172.21.15.98:0/65566016 >> v1:172.21.15.98:6789/0 conn(0x7fc2240b0810 legacy=0x7fc224164c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:55.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.671+0000 7fc22259c640 1 -- 172.21.15.98:0/65566016 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2240af640 con 0x7fc22415f4e0 2024-05-09T10:50:55.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.672+0000 7fc22259c640 1 --2- 172.21.15.98:0/65566016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc22415f4e0 0x7fc2240ae7c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc224071990 tx=0x7fc21402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd66f0126c9eb2e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:55.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.672+0000 7fc221d9b640 1 -- 172.21.15.98:0/65566016 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214037070 con 0x7fc22415f4e0 2024-05-09T10:50:55.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.672+0000 7fc221d9b640 1 -- 172.21.15.98:0/65566016 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc214032440 con 0x7fc22415f4e0 2024-05-09T10:50:55.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.672+0000 7fc221d9b640 1 -- 172.21.15.98:0/65566016 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21402ae60 con 0x7fc22415f4e0 2024-05-09T10:50:55.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.672+0000 7fc228d89640 1 -- 172.21.15.98:0/65566016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc22415f4e0 msgr2=0x7fc2240ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:55.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.672+0000 7fc228d89640 1 --2- 172.21.15.98:0/65566016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc22415f4e0 0x7fc2240ae7c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc224071990 tx=0x7fc21402a110 comp rx=0 tx=0).stop 2024-05-09T10:50:55.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.672+0000 7fc228d89640 1 -- 172.21.15.98:0/65566016 shutdown_connections 2024-05-09T10:50:55.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.672+0000 7fc228d89640 1 --2- 172.21.15.98:0/65566016 >> v2:172.21.15.105:3300/0 conn(0x7fc2240afe40 0x7fc2240b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.672+0000 7fc228d89640 1 --2- 172.21.15.98:0/65566016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc22415f4e0 0x7fc2240ae7c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.672+0000 7fc228d89640 1 -- 172.21.15.98:0/65566016 >> 172.21.15.98:0/65566016 conn(0x7fc2240b34e0 msgr2=0x7fc2240b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:55.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.672+0000 7fc228d89640 1 -- 172.21.15.98:0/65566016 shutdown_connections 2024-05-09T10:50:55.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.672+0000 7fc228d89640 1 -- 172.21.15.98:0/65566016 wait complete. 2024-05-09T10:50:55.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.674+0000 7fc228d89640 1 Processor -- start 2024-05-09T10:50:55.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.674+0000 7fc228d89640 1 -- start start 2024-05-09T10:50:55.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.674+0000 7fc228d89640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc2240afe40 0x7fc224153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.674+0000 7fc228d89640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2240b0810 0x7fc224153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.674+0000 7fc228d89640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc22415f4e0 0x7fc2241f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.674+0000 7fc228d89640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22415f8c0 con 0x7fc2240b0810 2024-05-09T10:50:55.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.674+0000 7fc228d89640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2240cbcf0 con 0x7fc22415f4e0 2024-05-09T10:50:55.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.674+0000 7fc228d89640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2240b0c20 con 0x7fc2240afe40 2024-05-09T10:50:55.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.675+0000 7fc21bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2240b0810 0x7fc224153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.675+0000 7fc21bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2240b0810 0x7fc224153560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35516/0 (socket says 172.21.15.98:35516) 2024-05-09T10:50:55.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.675+0000 7fc21bfff640 1 -- 172.21.15.98:0/3960554281 learned_addr learned my addr 172.21.15.98:0/3960554281 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:55.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.675+0000 7fc222d9d640 1 --2- 172.21.15.98:0/3960554281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc22415f4e0 0x7fc2241f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.675+0000 7fc22259c640 1 --2- 172.21.15.98:0/3960554281 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc2240afe40 0x7fc224153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.675+0000 7fc21bfff640 1 -- 172.21.15.98:0/3960554281 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc2240afe40 msgr2=0x7fc224153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:55.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.675+0000 7fc21bfff640 1 --2- 172.21.15.98:0/3960554281 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc2240afe40 0x7fc224153b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.675+0000 7fc21bfff640 1 -- 172.21.15.98:0/3960554281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc22415f4e0 msgr2=0x7fc2241f5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:55.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.675+0000 7fc21bfff640 1 --2- 172.21.15.98:0/3960554281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc22415f4e0 0x7fc2241f5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.675+0000 7fc21bfff640 1 -- 172.21.15.98:0/3960554281 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2240b6b70 con 0x7fc2240b0810 2024-05-09T10:50:55.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.675+0000 7fc22259c640 1 --2- 172.21.15.98:0/3960554281 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc2240afe40 0x7fc224153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:55.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.675+0000 7fc222d9d640 1 --2- 172.21.15.98:0/3960554281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc22415f4e0 0x7fc2241f5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:55.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.675+0000 7fc21bfff640 1 --2- 172.21.15.98:0/3960554281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2240b0810 0x7fc224153560 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00c7f0 tx=0x7fc20c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:55.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.675+0000 7fc21affd640 1 -- 172.21.15.98:0/3960554281 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c015070 con 0x7fc2240b0810 2024-05-09T10:50:55.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.675+0000 7fc21affd640 1 -- 172.21.15.98:0/3960554281 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20c004240 con 0x7fc2240b0810 2024-05-09T10:50:55.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.675+0000 7fc228d89640 1 -- 172.21.15.98:0/3960554281 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2241f6010 con 0x7fc2240b0810 2024-05-09T10:50:55.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.675+0000 7fc21affd640 1 -- 172.21.15.98:0/3960554281 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c007620 con 0x7fc2240b0810 2024-05-09T10:50:55.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.675+0000 7fc228d89640 1 -- 172.21.15.98:0/3960554281 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2240b6de0 con 0x7fc2240b0810 2024-05-09T10:50:55.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.676+0000 7fc228d89640 1 -- 172.21.15.98:0/3960554281 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2240cabe0 con 0x7fc2240b0810 2024-05-09T10:50:55.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.677+0000 7fc21affd640 1 -- 172.21.15.98:0/3960554281 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc20c0077c0 con 0x7fc2240b0810 2024-05-09T10:50:55.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.677+0000 7fc21affd640 1 --2- 172.21.15.98:0/3960554281 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc20406b4d0 0x7fc20406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.677+0000 7fc21affd640 1 -- 172.21.15.98:0/3960554281 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc20c08a400 con 0x7fc2240b0810 2024-05-09T10:50:55.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.677+0000 7fc22259c640 1 --2- 172.21.15.98:0/3960554281 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc20406b4d0 0x7fc20406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.678+0000 7fc22259c640 1 --2- 172.21.15.98:0/3960554281 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc20406b4d0 0x7fc20406d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc214034000 tx=0x7fc21402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:55.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.681+0000 7fc21affd640 1 -- 172.21.15.98:0/3960554281 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc2241f6010 con 0x7fc2240b0810 2024-05-09T10:50:55.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.854+0000 7fc228d89640 1 -- 172.21.15.98:0/3960554281 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc22406fde0 con 0x7fc2240b0810 2024-05-09T10:50:55.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.855+0000 7fc21affd640 1 -- 172.21.15.98:0/3960554281 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7fc20c059280 con 0x7fc2240b0810 2024-05-09T10:50:55.849 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:55.849 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":20,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:21.176419+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:50:17.323444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T10:50:18.482315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:55.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.856+0000 7fc228d89640 1 -- 172.21.15.98:0/3960554281 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc20406b4d0 msgr2=0x7fc20406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:55.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.856+0000 7fc228d89640 1 --2- 172.21.15.98:0/3960554281 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc20406b4d0 0x7fc20406d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc214034000 tx=0x7fc21402e000 comp rx=0 tx=0).stop 2024-05-09T10:50:55.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.857+0000 7fc228d89640 1 -- 172.21.15.98:0/3960554281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2240b0810 msgr2=0x7fc224153560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:55.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.857+0000 7fc228d89640 1 --2- 172.21.15.98:0/3960554281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2240b0810 0x7fc224153560 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00c7f0 tx=0x7fc20c00ccb0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.857+0000 7fc228d89640 1 -- 172.21.15.98:0/3960554281 shutdown_connections 2024-05-09T10:50:55.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.857+0000 7fc228d89640 1 --2- 172.21.15.98:0/3960554281 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc20406b4d0 0x7fc20406d990 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.857+0000 7fc228d89640 1 --2- 172.21.15.98:0/3960554281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc22415f4e0 0x7fc2241f5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.857+0000 7fc228d89640 1 --2- 172.21.15.98:0/3960554281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2240b0810 0x7fc224153560 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.857+0000 7fc228d89640 1 --2- 172.21.15.98:0/3960554281 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc2240afe40 0x7fc224153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.857+0000 7fc228d89640 1 -- 172.21.15.98:0/3960554281 >> 172.21.15.98:0/3960554281 conn(0x7fc2240b34e0 msgr2=0x7fc2240cfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:55.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.857+0000 7fc228d89640 1 -- 172.21.15.98:0/3960554281 shutdown_connections 2024-05-09T10:50:55.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.857+0000 7fc228d89640 1 -- 172.21.15.98:0/3960554281 wait complete. 2024-05-09T10:50:55.862 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T10:50:55.862 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T10:50:55.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.968+0000 7ff0a0529640 1 Processor -- start 2024-05-09T10:50:55.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.968+0000 7ff0a0529640 1 -- start start 2024-05-09T10:50:55.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.969+0000 7ff0a0529640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff098143740 0x7ff0980720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.969+0000 7ff0a0529640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0981443e0 0x7ff09815e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.969+0000 7ff0a0529640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff098143d70 0x7ff09815ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.969+0000 7ff0a0529640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff098143b20 con 0x7ff098143740 2024-05-09T10:50:55.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.969+0000 7ff0a0529640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff09815f460 con 0x7ff098143d70 2024-05-09T10:50:55.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.969+0000 7ff0a0529640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0980b28c0 con 0x7ff0981443e0 2024-05-09T10:50:55.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.969+0000 7ff09e2c5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff098143740 0x7ff0980720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.969+0000 7ff09e2c5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff098143740 0x7ff0980720a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35522/0 (socket says 172.21.15.98:35522) 2024-05-09T10:50:55.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.969+0000 7ff09e2c5640 1 -- 172.21.15.98:0/2699087501 learned_addr learned my addr 172.21.15.98:0/2699087501 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:55.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.969+0000 7ff09eac6640 1 --2- 172.21.15.98:0/2699087501 >> v2:172.21.15.105:3300/0 conn(0x7ff098143d70 0x7ff09815ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.969+0000 7ff09dac4640 1 --2- 172.21.15.98:0/2699087501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0981443e0 0x7ff09815e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.969+0000 7ff09e2c5640 1 -- 172.21.15.98:0/2699087501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0981443e0 msgr2=0x7ff09815e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:55.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.969+0000 7ff09e2c5640 1 --2- 172.21.15.98:0/2699087501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0981443e0 0x7ff09815e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.969+0000 7ff09e2c5640 1 -- 172.21.15.98:0/2699087501 >> v2:172.21.15.105:3300/0 conn(0x7ff098143d70 msgr2=0x7ff09815ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:55.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.969+0000 7ff09e2c5640 1 --2- 172.21.15.98:0/2699087501 >> v2:172.21.15.105:3300/0 conn(0x7ff098143d70 0x7ff09815ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.969+0000 7ff09e2c5640 1 -- 172.21.15.98:0/2699087501 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0980b2ba0 con 0x7ff098143740 2024-05-09T10:50:55.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.969+0000 7ff09dac4640 1 --2- 172.21.15.98:0/2699087501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0981443e0 0x7ff09815e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:55.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.970+0000 7ff09e2c5640 1 --2- 172.21.15.98:0/2699087501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff098143740 0x7ff0980720a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff0980b1340 tx=0x7ff09002c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8236434968542de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:55.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.970+0000 7ff09d2c3640 1 -- 172.21.15.98:0/2699087501 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff090032790 con 0x7ff098143740 2024-05-09T10:50:55.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.970+0000 7ff09d2c3640 1 -- 172.21.15.98:0/2699087501 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff090038070 con 0x7ff098143740 2024-05-09T10:50:55.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.970+0000 7ff09d2c3640 1 -- 172.21.15.98:0/2699087501 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff090032b50 con 0x7ff098143740 2024-05-09T10:50:55.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.970+0000 7ff0a0529640 1 -- 172.21.15.98:0/2699087501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff098143740 msgr2=0x7ff0980720a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:55.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.970+0000 7ff0a0529640 1 --2- 172.21.15.98:0/2699087501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff098143740 0x7ff0980720a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff0980b1340 tx=0x7ff09002c890 comp rx=0 tx=0).stop 2024-05-09T10:50:55.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.971+0000 7ff0a0529640 1 -- 172.21.15.98:0/2699087501 shutdown_connections 2024-05-09T10:50:55.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.971+0000 7ff0a0529640 1 --2- 172.21.15.98:0/2699087501 >> v2:172.21.15.105:3300/0 conn(0x7ff098143d70 0x7ff09815ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.971+0000 7ff0a0529640 1 --2- 172.21.15.98:0/2699087501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0981443e0 0x7ff09815e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.971+0000 7ff0a0529640 1 --2- 172.21.15.98:0/2699087501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff098143740 0x7ff0980720a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.971+0000 7ff0a0529640 1 -- 172.21.15.98:0/2699087501 >> 172.21.15.98:0/2699087501 conn(0x7ff09806abe0 msgr2=0x7ff09806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:55.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.971+0000 7ff0a0529640 1 -- 172.21.15.98:0/2699087501 shutdown_connections 2024-05-09T10:50:55.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.971+0000 7ff0a0529640 1 -- 172.21.15.98:0/2699087501 wait complete. 2024-05-09T10:50:55.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.972+0000 7ff0a0529640 1 Processor -- start 2024-05-09T10:50:55.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.972+0000 7ff0a0529640 1 -- start start 2024-05-09T10:50:55.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.972+0000 7ff0a0529640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff098143740 0x7ff0980adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.972+0000 7ff0a0529640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff098143d70 0x7ff098141110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.972+0000 7ff0a0529640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0981443e0 0x7ff09815a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.972+0000 7ff0a0529640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09806c4a0 con 0x7ff098143740 2024-05-09T10:50:55.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.972+0000 7ff0a0529640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09806c320 con 0x7ff098143d70 2024-05-09T10:50:55.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.972+0000 7ff0a0529640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09806c620 con 0x7ff0981443e0 2024-05-09T10:50:55.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.973+0000 7ff09e2c5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff098143740 0x7ff0980adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.973+0000 7ff09e2c5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff098143740 0x7ff0980adaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35536/0 (socket says 172.21.15.98:35536) 2024-05-09T10:50:55.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.973+0000 7ff09e2c5640 1 -- 172.21.15.98:0/1152027951 learned_addr learned my addr 172.21.15.98:0/1152027951 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:55.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.973+0000 7ff09dac4640 1 --2- 172.21.15.98:0/1152027951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff098143d70 0x7ff098141110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.973+0000 7ff09eac6640 1 --2- 172.21.15.98:0/1152027951 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0981443e0 0x7ff09815a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.973+0000 7ff09e2c5640 1 -- 172.21.15.98:0/1152027951 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0981443e0 msgr2=0x7ff09815a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:55.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.973+0000 7ff09e2c5640 1 --2- 172.21.15.98:0/1152027951 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0981443e0 0x7ff09815a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.973+0000 7ff09e2c5640 1 -- 172.21.15.98:0/1152027951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff098143d70 msgr2=0x7ff098141110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:55.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.973+0000 7ff09e2c5640 1 --2- 172.21.15.98:0/1152027951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff098143d70 0x7ff098141110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:55.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.973+0000 7ff09e2c5640 1 -- 172.21.15.98:0/1152027951 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0980b0f50 con 0x7ff098143740 2024-05-09T10:50:55.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.973+0000 7ff09e2c5640 1 --2- 172.21.15.98:0/1152027951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff098143740 0x7ff0980adaf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff09000b2c0 tx=0x7ff090004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:55.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.973+0000 7ff096ffd640 1 -- 172.21.15.98:0/1152027951 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff090043070 con 0x7ff098143740 2024-05-09T10:50:55.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.973+0000 7ff096ffd640 1 -- 172.21.15.98:0/1152027951 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09003c620 con 0x7ff098143740 2024-05-09T10:50:55.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.973+0000 7ff0a0529640 1 -- 172.21.15.98:0/1152027951 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09815b060 con 0x7ff098143740 2024-05-09T10:50:55.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.973+0000 7ff096ffd640 1 -- 172.21.15.98:0/1152027951 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09003c9f0 con 0x7ff098143740 2024-05-09T10:50:55.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.973+0000 7ff0a0529640 1 -- 172.21.15.98:0/1152027951 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0980ae030 con 0x7ff098143740 2024-05-09T10:50:55.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.974+0000 7ff0a0529640 1 -- 172.21.15.98:0/1152027951 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff098069cb0 con 0x7ff098143740 2024-05-09T10:50:55.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.975+0000 7ff096ffd640 1 -- 172.21.15.98:0/1152027951 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff090046070 con 0x7ff098143740 2024-05-09T10:50:55.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.975+0000 7ff096ffd640 1 --2- 172.21.15.98:0/1152027951 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff08406b4d0 0x7ff08406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:55.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.975+0000 7ff096ffd640 1 -- 172.21.15.98:0/1152027951 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff0900aa7f0 con 0x7ff098143740 2024-05-09T10:50:55.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.975+0000 7ff09dac4640 1 --2- 172.21.15.98:0/1152027951 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff08406b4d0 0x7ff08406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:55.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.976+0000 7ff09dac4640 1 --2- 172.21.15.98:0/1152027951 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff08406b4d0 0x7ff08406d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff0980ae510 tx=0x7ff08c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:55.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:55.979+0000 7ff096ffd640 1 -- 172.21.15.98:0/1152027951 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff090079700 con 0x7ff098143740 2024-05-09T10:50:56.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.187+0000 7ff0a0529640 1 -- 172.21.15.98:0/1152027951 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff09815e450 con 0x7ff098143740 2024-05-09T10:50:56.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.188+0000 7ff096ffd640 1 -- 172.21.15.98:0/1152027951 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4932 (secure 0 0 0) 0x7ff090079000 con 0x7ff098143740 2024-05-09T10:50:56.182 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:56.182 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2903091926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2903091926},{"type":"v1","addr":"172.21.15.98:6837","nonce":2903091926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/968936316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":968936316},{"type":"v1","addr":"172.21.15.105:6834","nonce":968936316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/937700297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":937700297},{"type":"v1","addr":"172.21.15.98:6835","nonce":937700297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T10:50:19.544432+0000","modified":"2024-05-09T10:50:21.512942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6835/1780062557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1780062557},{"type":"v1","addr":"172.21.15.105:6835","nonce":1780062557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T10:50:56.182 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 6 2024-05-09T10:50:56.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.190+0000 7ff0a0529640 1 -- 172.21.15.98:0/1152027951 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff08406b4d0 msgr2=0x7ff08406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:56.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.190+0000 7ff0a0529640 1 --2- 172.21.15.98:0/1152027951 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff08406b4d0 0x7ff08406d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff0980ae510 tx=0x7ff08c008040 comp rx=0 tx=0).stop 2024-05-09T10:50:56.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.190+0000 7ff0a0529640 1 -- 172.21.15.98:0/1152027951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff098143740 msgr2=0x7ff0980adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:56.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.190+0000 7ff0a0529640 1 --2- 172.21.15.98:0/1152027951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff098143740 0x7ff0980adaf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff09000b2c0 tx=0x7ff090004730 comp rx=0 tx=0).stop 2024-05-09T10:50:56.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.190+0000 7ff0a0529640 1 -- 172.21.15.98:0/1152027951 shutdown_connections 2024-05-09T10:50:56.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.190+0000 7ff0a0529640 1 --2- 172.21.15.98:0/1152027951 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff08406b4d0 0x7ff08406d990 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.190+0000 7ff0a0529640 1 --2- 172.21.15.98:0/1152027951 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0981443e0 0x7ff09815a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.190+0000 7ff0a0529640 1 --2- 172.21.15.98:0/1152027951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff098143d70 0x7ff098141110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.190+0000 7ff0a0529640 1 --2- 172.21.15.98:0/1152027951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff098143740 0x7ff0980adaf0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.190+0000 7ff0a0529640 1 -- 172.21.15.98:0/1152027951 >> 172.21.15.98:0/1152027951 conn(0x7ff09806abe0 msgr2=0x7ff09806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:56.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.190+0000 7ff0a0529640 1 -- 172.21.15.98:0/1152027951 shutdown_connections 2024-05-09T10:50:56.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.190+0000 7ff0a0529640 1 -- 172.21.15.98:0/1152027951 wait complete. 2024-05-09T10:50:56.195 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T10:50:56.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.301+0000 7f5e47af5640 1 Processor -- start 2024-05-09T10:50:56.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.301+0000 7f5e47af5640 1 -- start start 2024-05-09T10:50:56.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.302+0000 7f5e47af5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e400cb680 0x7f5e400ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:56.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.302+0000 7f5e47af5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5e4006c240 0x7f5e400ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:56.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.302+0000 7f5e47af5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e400aee40 con 0x7f5e4006be60 2024-05-09T10:50:56.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.302+0000 7f5e47af5640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e400ced40 con 0x7f5e4006c240 2024-05-09T10:50:56.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.302+0000 7f5e47af5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e40071a70 con 0x7f5e400cb680 2024-05-09T10:50:56.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.302+0000 7f5e46092640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5e4006c240 0x7f5e400ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:56.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.302+0000 7f5e45891640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f5e4006be60 0x7f5e40072830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38860/0 (socket says 172.21.15.98:38860) 2024-05-09T10:50:56.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.302+0000 7f5e45891640 1 -- 172.21.15.98:0/904204539 learned_addr learned my addr 172.21.15.98:0/904204539 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:56.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.302+0000 7f5e45090640 1 --2- 172.21.15.98:0/904204539 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e400cb680 0x7f5e400ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:56.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.302+0000 7f5e46092640 1 -- 172.21.15.98:0/904204539 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e400cb680 msgr2=0x7f5e400ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:56.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.302+0000 7f5e46092640 1 --2- 172.21.15.98:0/904204539 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e400cb680 0x7f5e400ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.302+0000 7f5e46092640 1 -- 172.21.15.98:0/904204539 >> v1:172.21.15.98:6789/0 conn(0x7f5e4006be60 legacy=0x7f5e40072830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:56.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.302+0000 7f5e4488f640 1 -- 172.21.15.98:0/904204539 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2694255916 0 0) 0x7f5e400aee40 con 0x7f5e4006be60 2024-05-09T10:50:56.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.302+0000 7f5e46092640 1 -- 172.21.15.98:0/904204539 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e40071d50 con 0x7f5e4006c240 2024-05-09T10:50:56.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.302+0000 7f5e45090640 1 --2- 172.21.15.98:0/904204539 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e400cb680 0x7f5e400ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:56.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.303+0000 7f5e46092640 1 --2- 172.21.15.98:0/904204539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e4006c240 0x7f5e400ce800 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009d90 tx=0x7f5e3802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caaec3500479c584 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:56.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.303+0000 7f5e4488f640 1 -- 172.21.15.98:0/904204539 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38007740 con 0x7f5e4006c240 2024-05-09T10:50:56.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.303+0000 7f5e4488f640 1 -- 172.21.15.98:0/904204539 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e380078e0 con 0x7f5e4006c240 2024-05-09T10:50:56.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.303+0000 7f5e4488f640 1 -- 172.21.15.98:0/904204539 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38007cb0 con 0x7f5e4006c240 2024-05-09T10:50:56.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.303+0000 7f5e47af5640 1 -- 172.21.15.98:0/904204539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e4006c240 msgr2=0x7f5e400ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:56.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.303+0000 7f5e47af5640 1 --2- 172.21.15.98:0/904204539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e4006c240 0x7f5e400ce800 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009d90 tx=0x7f5e3802a660 comp rx=0 tx=0).stop 2024-05-09T10:50:56.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.303+0000 7f5e47af5640 1 -- 172.21.15.98:0/904204539 shutdown_connections 2024-05-09T10:50:56.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.303+0000 7f5e47af5640 1 --2- 172.21.15.98:0/904204539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e4006c240 0x7f5e400ce800 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.303+0000 7f5e47af5640 1 --2- 172.21.15.98:0/904204539 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e400cb680 0x7f5e400ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.303+0000 7f5e47af5640 1 -- 172.21.15.98:0/904204539 >> 172.21.15.98:0/904204539 conn(0x7f5e400ccaa0 msgr2=0x7f5e400cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:56.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.304+0000 7f5e47af5640 1 -- 172.21.15.98:0/904204539 shutdown_connections 2024-05-09T10:50:56.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.304+0000 7f5e47af5640 1 -- 172.21.15.98:0/904204539 wait complete. 2024-05-09T10:50:56.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.305+0000 7f5e47af5640 1 Processor -- start 2024-05-09T10:50:56.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.305+0000 7f5e47af5640 1 -- start start 2024-05-09T10:50:56.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.305+0000 7f5e47af5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e4006be60 0x7f5e401f3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:56.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.305+0000 7f5e47af5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e4006c240 0x7f5e400b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:56.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.305+0000 7f5e47af5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e400cb680 0x7f5e400b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:56.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.305+0000 7f5e47af5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e400af100 con 0x7f5e400cb680 2024-05-09T10:50:56.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.305+0000 7f5e47af5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e40072310 con 0x7f5e4006c240 2024-05-09T10:50:56.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.305+0000 7f5e47af5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e400af470 con 0x7f5e4006be60 2024-05-09T10:50:56.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.306+0000 7f5e46092640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e400cb680 0x7f5e400b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:56.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.306+0000 7f5e46092640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e400cb680 0x7f5e400b5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35560/0 (socket says 172.21.15.98:35560) 2024-05-09T10:50:56.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.306+0000 7f5e46092640 1 -- 172.21.15.98:0/4232479069 learned_addr learned my addr 172.21.15.98:0/4232479069 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:56.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.306+0000 7f5e46092640 1 -- 172.21.15.98:0/4232479069 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e4006be60 msgr2=0x7f5e401f3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:56.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.306+0000 7f5e46092640 1 --2- 172.21.15.98:0/4232479069 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e4006be60 0x7f5e401f3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.306+0000 7f5e46092640 1 -- 172.21.15.98:0/4232479069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e4006c240 msgr2=0x7f5e400b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:56.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.306+0000 7f5e46092640 1 --2- 172.21.15.98:0/4232479069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e4006c240 0x7f5e400b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.306+0000 7f5e46092640 1 -- 172.21.15.98:0/4232479069 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e400c83a0 con 0x7f5e400cb680 2024-05-09T10:50:56.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.306+0000 7f5e46092640 1 --2- 172.21.15.98:0/4232479069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e400cb680 0x7f5e400b5080 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009eb0 tx=0x7f5e38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:56.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.306+0000 7f5e367fc640 1 -- 172.21.15.98:0/4232479069 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3803f070 con 0x7f5e400cb680 2024-05-09T10:50:56.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.306+0000 7f5e367fc640 1 -- 172.21.15.98:0/4232479069 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e38031c40 con 0x7f5e400cb680 2024-05-09T10:50:56.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.306+0000 7f5e47af5640 1 -- 172.21.15.98:0/4232479069 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e401f40a0 con 0x7f5e400cb680 2024-05-09T10:50:56.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.306+0000 7f5e367fc640 1 -- 172.21.15.98:0/4232479069 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38003b90 con 0x7f5e400cb680 2024-05-09T10:50:56.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.306+0000 7f5e47af5640 1 -- 172.21.15.98:0/4232479069 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e400c86a0 con 0x7f5e400cb680 2024-05-09T10:50:56.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.307+0000 7f5e47af5640 1 -- 172.21.15.98:0/4232479069 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e40069cb0 con 0x7f5e400cb680 2024-05-09T10:50:56.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.307+0000 7f5e367fc640 1 -- 172.21.15.98:0/4232479069 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e38004030 con 0x7f5e400cb680 2024-05-09T10:50:56.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.308+0000 7f5e367fc640 1 --2- 172.21.15.98:0/4232479069 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5e2406b500 0x7f5e2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:56.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.308+0000 7f5e367fc640 1 -- 172.21.15.98:0/4232479069 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e380ab2e0 con 0x7f5e400cb680 2024-05-09T10:50:56.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.308+0000 7f5e45891640 1 --2- 172.21.15.98:0/4232479069 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5e2406b500 0x7f5e2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:56.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.308+0000 7f5e45891640 1 --2- 172.21.15.98:0/4232479069 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5e2406b500 0x7f5e2406d9c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c007fd0 tx=0x7f5e3c007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:56.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.313+0000 7f5e367fc640 1 -- 172.21.15.98:0/4232479069 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e3807a1f0 con 0x7f5e400cb680 2024-05-09T10:50:56.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.519+0000 7f5e47af5640 1 -- 172.21.15.98:0/4232479069 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e40066b10 con 0x7f5e400cb680 2024-05-09T10:50:56.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.520+0000 7f5e367fc640 1 -- 172.21.15.98:0/4232479069 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4932 (secure 0 0 0) 0x7f5e38079af0 con 0x7f5e400cb680 2024-05-09T10:50:56.515 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:56.515 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2903091926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2903091926},{"type":"v1","addr":"172.21.15.98:6837","nonce":2903091926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/968936316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":968936316},{"type":"v1","addr":"172.21.15.105:6834","nonce":968936316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/937700297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":937700297},{"type":"v1","addr":"172.21.15.98:6835","nonce":937700297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T10:50:19.544432+0000","modified":"2024-05-09T10:50:21.512942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6835/1780062557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":1780062557},{"type":"v1","addr":"172.21.15.105:6835","nonce":1780062557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-09T10:50:56.515 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 6 2024-05-09T10:50:56.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.522+0000 7f5e47af5640 1 -- 172.21.15.98:0/4232479069 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5e2406b500 msgr2=0x7f5e2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:56.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.522+0000 7f5e47af5640 1 --2- 172.21.15.98:0/4232479069 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5e2406b500 0x7f5e2406d9c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c007fd0 tx=0x7f5e3c007960 comp rx=0 tx=0).stop 2024-05-09T10:50:56.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.522+0000 7f5e47af5640 1 -- 172.21.15.98:0/4232479069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e400cb680 msgr2=0x7f5e400b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:56.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.522+0000 7f5e47af5640 1 --2- 172.21.15.98:0/4232479069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e400cb680 0x7f5e400b5080 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009eb0 tx=0x7f5e38004a40 comp rx=0 tx=0).stop 2024-05-09T10:50:56.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.522+0000 7f5e47af5640 1 -- 172.21.15.98:0/4232479069 shutdown_connections 2024-05-09T10:50:56.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.522+0000 7f5e47af5640 1 --2- 172.21.15.98:0/4232479069 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5e2406b500 0x7f5e2406d9c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.522+0000 7f5e47af5640 1 --2- 172.21.15.98:0/4232479069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e400cb680 0x7f5e400b5080 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.522+0000 7f5e47af5640 1 --2- 172.21.15.98:0/4232479069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e4006c240 0x7f5e400b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.522+0000 7f5e47af5640 1 --2- 172.21.15.98:0/4232479069 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e4006be60 0x7f5e401f3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.522+0000 7f5e47af5640 1 -- 172.21.15.98:0/4232479069 >> 172.21.15.98:0/4232479069 conn(0x7f5e400ccaa0 msgr2=0x7f5e400cce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:56.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.522+0000 7f5e47af5640 1 -- 172.21.15.98:0/4232479069 shutdown_connections 2024-05-09T10:50:56.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.522+0000 7f5e47af5640 1 -- 172.21.15.98:0/4232479069 wait complete. 2024-05-09T10:50:56.528 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:50:56.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.632+0000 7ff10369e640 1 Processor -- start 2024-05-09T10:50:56.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.633+0000 7ff10369e640 1 -- start start 2024-05-09T10:50:56.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.634+0000 7ff10369e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff0fc06eb30 0x7ff0fc0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:56.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.634+0000 7ff10369e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0fc15ddc0 0x7ff0fc15e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:56.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.634+0000 7ff10369e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0fc0d07a0 con 0x7ff0fc06d570 2024-05-09T10:50:56.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.634+0000 7ff10369e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0fc0d0db0 con 0x7ff0fc06eb30 2024-05-09T10:50:56.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.634+0000 7ff10369e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc06f9b0 con 0x7ff0fc15ddc0 2024-05-09T10:50:56.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.634+0000 7ff10143a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff0fc06eb30 0x7ff0fc0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:56.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.634+0000 7ff10143a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff0fc06eb30 0x7ff0fc0b4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35564/0 (socket says 172.21.15.98:35564) 2024-05-09T10:50:56.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.634+0000 7ff10143a640 1 -- 172.21.15.98:0/292004651 learned_addr learned my addr 172.21.15.98:0/292004651 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:56.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.634+0000 7ff10143a640 1 -- 172.21.15.98:0/292004651 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0fc15ddc0 msgr2=0x7ff0fc15e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:56.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.634+0000 7ff10143a640 1 --2- 172.21.15.98:0/292004651 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0fc15ddc0 0x7ff0fc15e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.634+0000 7ff10143a640 1 -- 172.21.15.98:0/292004651 >> v1:172.21.15.105:6789/0 conn(0x7ff0fc06d570 legacy=0x7ff0fc06ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:56.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.634+0000 7ff10143a640 1 -- 172.21.15.98:0/292004651 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0fc06f6d0 con 0x7ff0fc06eb30 2024-05-09T10:50:56.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.635+0000 7ff10143a640 1 --2- 172.21.15.98:0/292004651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0fc06eb30 0x7ff0fc0b4000 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0d0f30 tx=0x7ff0f802a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f12c3abd58f5e4f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:56.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.635+0000 7ff0f3fff640 1 -- 172.21.15.98:0/292004651 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8037070 con 0x7ff0fc06eb30 2024-05-09T10:50:56.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.635+0000 7ff0f3fff640 1 -- 172.21.15.98:0/292004651 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f8032440 con 0x7ff0fc06eb30 2024-05-09T10:50:56.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.635+0000 7ff0f3fff640 1 -- 172.21.15.98:0/292004651 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8032810 con 0x7ff0fc06eb30 2024-05-09T10:50:56.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.635+0000 7ff10369e640 1 -- 172.21.15.98:0/292004651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0fc06eb30 msgr2=0x7ff0fc0b4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:56.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.635+0000 7ff10369e640 1 --2- 172.21.15.98:0/292004651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0fc06eb30 0x7ff0fc0b4000 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0d0f30 tx=0x7ff0f802a160 comp rx=0 tx=0).stop 2024-05-09T10:50:56.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.635+0000 7ff10369e640 1 -- 172.21.15.98:0/292004651 shutdown_connections 2024-05-09T10:50:56.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.635+0000 7ff10369e640 1 --2- 172.21.15.98:0/292004651 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0fc15ddc0 0x7ff0fc15e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.635+0000 7ff10369e640 1 --2- 172.21.15.98:0/292004651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0fc06eb30 0x7ff0fc0b4000 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.635+0000 7ff10369e640 1 -- 172.21.15.98:0/292004651 >> 172.21.15.98:0/292004651 conn(0x7ff0fc0b34e0 msgr2=0x7ff0fc0b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:56.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.635+0000 7ff10369e640 1 -- 172.21.15.98:0/292004651 shutdown_connections 2024-05-09T10:50:56.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.635+0000 7ff10369e640 1 -- 172.21.15.98:0/292004651 wait complete. 2024-05-09T10:50:56.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.637+0000 7ff10369e640 1 Processor -- start 2024-05-09T10:50:56.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.637+0000 7ff10369e640 1 -- start start 2024-05-09T10:50:56.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.637+0000 7ff10369e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0fc06d570 0x7ff0fc157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:56.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.637+0000 7ff10369e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0fc06eb30 0x7ff0fc15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:56.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.637+0000 7ff10369e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0fc15ddc0 0x7ff0fc0b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:56.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.637+0000 7ff10369e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc06bea0 con 0x7ff0fc06eb30 2024-05-09T10:50:56.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.637+0000 7ff10369e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc15e790 con 0x7ff0fc15ddc0 2024-05-09T10:50:56.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.637+0000 7ff10369e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc06c020 con 0x7ff0fc06d570 2024-05-09T10:50:56.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.637+0000 7ff100c39640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0fc06eb30 0x7ff0fc15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:56.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.637+0000 7ff100c39640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0fc06eb30 0x7ff0fc15ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35568/0 (socket says 172.21.15.98:35568) 2024-05-09T10:50:56.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.637+0000 7ff100c39640 1 -- 172.21.15.98:0/3010180701 learned_addr learned my addr 172.21.15.98:0/3010180701 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:56.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.638+0000 7ff100c39640 1 -- 172.21.15.98:0/3010180701 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0fc06d570 msgr2=0x7ff0fc157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:56.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.638+0000 7ff100c39640 1 --2- 172.21.15.98:0/3010180701 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0fc06d570 0x7ff0fc157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.638+0000 7ff101c3b640 1 --2- 172.21.15.98:0/3010180701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0fc15ddc0 0x7ff0fc0b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:56.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.638+0000 7ff100c39640 1 -- 172.21.15.98:0/3010180701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0fc15ddc0 msgr2=0x7ff0fc0b01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:56.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.638+0000 7ff100c39640 1 --2- 172.21.15.98:0/3010180701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0fc15ddc0 0x7ff0fc0b01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.638+0000 7ff100c39640 1 -- 172.21.15.98:0/3010180701 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0fc0c5430 con 0x7ff0fc06eb30 2024-05-09T10:50:56.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.638+0000 7ff101c3b640 1 --2- 172.21.15.98:0/3010180701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0fc15ddc0 0x7ff0fc0b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:56.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.638+0000 7ff100c39640 1 --2- 172.21.15.98:0/3010180701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0fc06eb30 0x7ff0fc15ecb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00c440 tx=0x7ff0ec00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:56.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.638+0000 7ff0f1ffb640 1 -- 172.21.15.98:0/3010180701 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec017070 con 0x7ff0fc06eb30 2024-05-09T10:50:56.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.638+0000 7ff0f1ffb640 1 -- 172.21.15.98:0/3010180701 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ec0035f0 con 0x7ff0fc06eb30 2024-05-09T10:50:56.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.638+0000 7ff0f1ffb640 1 -- 172.21.15.98:0/3010180701 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec0039e0 con 0x7ff0fc06eb30 2024-05-09T10:50:56.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.638+0000 7ff10369e640 1 -- 172.21.15.98:0/3010180701 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0fc0c5710 con 0x7ff0fc06eb30 2024-05-09T10:50:56.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.638+0000 7ff10369e640 1 -- 172.21.15.98:0/3010180701 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0fc0c7dc0 con 0x7ff0fc06eb30 2024-05-09T10:50:56.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.639+0000 7ff10369e640 1 -- 172.21.15.98:0/3010180701 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0fc0c7fc0 con 0x7ff0fc06eb30 2024-05-09T10:50:56.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.639+0000 7ff0f1ffb640 1 -- 172.21.15.98:0/3010180701 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff0ec01a070 con 0x7ff0fc06eb30 2024-05-09T10:50:56.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.640+0000 7ff0f1ffb640 1 --2- 172.21.15.98:0/3010180701 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff0e006b500 0x7ff0e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:56.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.640+0000 7ff0f1ffb640 1 -- 172.21.15.98:0/3010180701 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff0ec0894a0 con 0x7ff0fc06eb30 2024-05-09T10:50:56.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.640+0000 7ff10143a640 1 --2- 172.21.15.98:0/3010180701 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff0e006b500 0x7ff0e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:56.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.640+0000 7ff10143a640 1 --2- 172.21.15.98:0/3010180701 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff0e006b500 0x7ff0e006d9c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8034000 tx=0x7ff0f802c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:56.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.644+0000 7ff0f1ffb640 1 -- 172.21.15.98:0/3010180701 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff0fc0c7fc0 con 0x7ff0fc06eb30 2024-05-09T10:50:56.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.814+0000 7ff10369e640 1 -- 172.21.15.98:0/3010180701 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0fc06fde0 con 0x7ff0fc06eb30 2024-05-09T10:50:56.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.816+0000 7ff0f1ffb640 1 -- 172.21.15.98:0/3010180701 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16605 (secure 0 0 0) 0x7ff0ec0583b0 con 0x7ff0fc06eb30 2024-05-09T10:50:56.810 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:50:56.811 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":20,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:50:21.176419+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:50:17.323444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T10:50:18.482315+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:50:56.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.817+0000 7ff10369e640 1 -- 172.21.15.98:0/3010180701 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff0e006b500 msgr2=0x7ff0e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:56.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.817+0000 7ff10369e640 1 --2- 172.21.15.98:0/3010180701 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff0e006b500 0x7ff0e006d9c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8034000 tx=0x7ff0f802c000 comp rx=0 tx=0).stop 2024-05-09T10:50:56.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.818+0000 7ff10369e640 1 -- 172.21.15.98:0/3010180701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0fc06eb30 msgr2=0x7ff0fc15ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:56.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.818+0000 7ff10369e640 1 --2- 172.21.15.98:0/3010180701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0fc06eb30 0x7ff0fc15ecb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00c440 tx=0x7ff0ec00c900 comp rx=0 tx=0).stop 2024-05-09T10:50:56.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.818+0000 7ff10369e640 1 -- 172.21.15.98:0/3010180701 shutdown_connections 2024-05-09T10:50:56.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.818+0000 7ff10369e640 1 --2- 172.21.15.98:0/3010180701 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff0e006b500 0x7ff0e006d9c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.818+0000 7ff10369e640 1 --2- 172.21.15.98:0/3010180701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0fc15ddc0 0x7ff0fc0b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.818+0000 7ff10369e640 1 --2- 172.21.15.98:0/3010180701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0fc06eb30 0x7ff0fc15ecb0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.818+0000 7ff10369e640 1 --2- 172.21.15.98:0/3010180701 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0fc06d570 0x7ff0fc157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.818+0000 7ff10369e640 1 -- 172.21.15.98:0/3010180701 >> 172.21.15.98:0/3010180701 conn(0x7ff0fc0b34e0 msgr2=0x7ff0fc06ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:56.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.818+0000 7ff10369e640 1 -- 172.21.15.98:0/3010180701 shutdown_connections 2024-05-09T10:50:56.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.818+0000 7ff10369e640 1 -- 172.21.15.98:0/3010180701 wait complete. 2024-05-09T10:50:56.823 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T10:50:56.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.930+0000 7ff80ae4d640 1 Processor -- start 2024-05-09T10:50:56.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.930+0000 7ff80ae4d640 1 -- start start 2024-05-09T10:50:56.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.930+0000 7ff80ae4d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff80415de40 0x7ff8040d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:56.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.930+0000 7ff80ae4d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff8040cf250 0x7ff8040cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:56.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.930+0000 7ff80ae4d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80415e220 con 0x7ff8040d0470 2024-05-09T10:50:56.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.930+0000 7ff80ae4d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8040d1790 con 0x7ff8040cf250 2024-05-09T10:50:56.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.930+0000 7ff80ae4d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8040d19e0 con 0x7ff80415de40 2024-05-09T10:50:56.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.930+0000 7ff803fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff8040cf250 0x7ff8040cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:56.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.930+0000 7ff803fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff8040cf250 0x7ff8040cf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35570/0 (socket says 172.21.15.98:35570) 2024-05-09T10:50:56.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.930+0000 7ff803fff640 1 -- 172.21.15.98:0/1965584608 learned_addr learned my addr 172.21.15.98:0/1965584608 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:56.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.930+0000 7ff808be9640 1 --2- 172.21.15.98:0/1965584608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff80415de40 0x7ff8040d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:56.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.930+0000 7ff803fff640 1 -- 172.21.15.98:0/1965584608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff80415de40 msgr2=0x7ff8040d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:56.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.930+0000 7ff803fff640 1 --2- 172.21.15.98:0/1965584608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff80415de40 0x7ff8040d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.930+0000 7ff803fff640 1 -- 172.21.15.98:0/1965584608 >> v1:172.21.15.105:6789/0 conn(0x7ff8040d0470 legacy=0x7ff8040b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:56.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.930+0000 7ff803fff640 1 -- 172.21.15.98:0/1965584608 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8040d1cc0 con 0x7ff8040cf250 2024-05-09T10:50:56.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.931+0000 7ff803fff640 1 --2- 172.21.15.98:0/1965584608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8040cf250 0x7ff8040cf6b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009870 tx=0x7ff7f002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47294cdcb4465cd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:56.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.931+0000 7ff8037fe640 1 -- 172.21.15.98:0/1965584608 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0037070 con 0x7ff8040cf250 2024-05-09T10:50:56.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.931+0000 7ff8037fe640 1 -- 172.21.15.98:0/1965584608 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f0032440 con 0x7ff8040cf250 2024-05-09T10:50:56.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.931+0000 7ff8037fe640 1 -- 172.21.15.98:0/1965584608 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0032810 con 0x7ff8040cf250 2024-05-09T10:50:56.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.932+0000 7ff80ae4d640 1 -- 172.21.15.98:0/1965584608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8040cf250 msgr2=0x7ff8040cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:56.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.932+0000 7ff80ae4d640 1 --2- 172.21.15.98:0/1965584608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8040cf250 0x7ff8040cf6b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009870 tx=0x7ff7f002a210 comp rx=0 tx=0).stop 2024-05-09T10:50:56.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.932+0000 7ff80ae4d640 1 -- 172.21.15.98:0/1965584608 shutdown_connections 2024-05-09T10:50:56.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.932+0000 7ff80ae4d640 1 --2- 172.21.15.98:0/1965584608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8040cf250 0x7ff8040cf6b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.932+0000 7ff80ae4d640 1 --2- 172.21.15.98:0/1965584608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff80415de40 0x7ff8040d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.932+0000 7ff80ae4d640 1 -- 172.21.15.98:0/1965584608 >> 172.21.15.98:0/1965584608 conn(0x7ff80409ce90 msgr2=0x7ff8040d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:56.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.932+0000 7ff80ae4d640 1 -- 172.21.15.98:0/1965584608 shutdown_connections 2024-05-09T10:50:56.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.932+0000 7ff80ae4d640 1 -- 172.21.15.98:0/1965584608 wait complete. 2024-05-09T10:50:56.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.933+0000 7ff80ae4d640 1 Processor -- start 2024-05-09T10:50:56.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.933+0000 7ff80ae4d640 1 -- start start 2024-05-09T10:50:56.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.934+0000 7ff80ae4d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8040cf250 0x7ff8041575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:56.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.934+0000 7ff80ae4d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8040d0470 0x7ff804150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:56.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.934+0000 7ff80ae4d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff80415de40 0x7ff804150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:56.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.934+0000 7ff80ae4d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8040d4680 con 0x7ff80415de40 2024-05-09T10:50:56.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.934+0000 7ff80ae4d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8040d2280 con 0x7ff8040d0470 2024-05-09T10:50:56.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.934+0000 7ff80ae4d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8040d4800 con 0x7ff8040cf250 2024-05-09T10:50:56.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.934+0000 7ff8093ea640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff80415de40 0x7ff804150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:56.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.934+0000 7ff8093ea640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff80415de40 0x7ff804150da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35586/0 (socket says 172.21.15.98:35586) 2024-05-09T10:50:56.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.934+0000 7ff8093ea640 1 -- 172.21.15.98:0/3212219261 learned_addr learned my addr 172.21.15.98:0/3212219261 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:56.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.934+0000 7ff808be9640 1 --2- 172.21.15.98:0/3212219261 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8040cf250 0x7ff8041575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:56.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.934+0000 7ff803fff640 1 --2- 172.21.15.98:0/3212219261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8040d0470 0x7ff804150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:56.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.934+0000 7ff8093ea640 1 -- 172.21.15.98:0/3212219261 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8040cf250 msgr2=0x7ff8041575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:56.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.934+0000 7ff8093ea640 1 --2- 172.21.15.98:0/3212219261 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8040cf250 0x7ff8041575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.934+0000 7ff8093ea640 1 -- 172.21.15.98:0/3212219261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8040d0470 msgr2=0x7ff804150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:56.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.934+0000 7ff8093ea640 1 --2- 172.21.15.98:0/3212219261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8040d0470 0x7ff804150860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:56.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.934+0000 7ff8093ea640 1 -- 172.21.15.98:0/3212219261 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff804150670 con 0x7ff80415de40 2024-05-09T10:50:56.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.934+0000 7ff8093ea640 1 --2- 172.21.15.98:0/3212219261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff80415de40 0x7ff804150da0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00c7b0 tx=0x7ff7fc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:56.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.934+0000 7ff8017fa640 1 -- 172.21.15.98:0/3212219261 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc004b60 con 0x7ff80415de40 2024-05-09T10:50:56.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.934+0000 7ff8017fa640 1 -- 172.21.15.98:0/3212219261 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7fc004d00 con 0x7ff80415de40 2024-05-09T10:50:56.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.934+0000 7ff80ae4d640 1 -- 172.21.15.98:0/3212219261 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff804157c30 con 0x7ff80415de40 2024-05-09T10:50:56.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.935+0000 7ff8017fa640 1 -- 172.21.15.98:0/3212219261 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc00aa90 con 0x7ff80415de40 2024-05-09T10:50:56.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.935+0000 7ff80ae4d640 1 -- 172.21.15.98:0/3212219261 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8041581f0 con 0x7ff80415de40 2024-05-09T10:50:56.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.935+0000 7ff80ae4d640 1 -- 172.21.15.98:0/3212219261 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff804069cb0 con 0x7ff80415de40 2024-05-09T10:50:56.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.936+0000 7ff8017fa640 1 -- 172.21.15.98:0/3212219261 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff7fc00ac30 con 0x7ff80415de40 2024-05-09T10:50:56.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.936+0000 7ff8017fa640 1 --2- 172.21.15.98:0/3212219261 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff7e806b4d0 0x7ff7e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:56.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.937+0000 7ff8017fa640 1 -- 172.21.15.98:0/3212219261 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7fc08a440 con 0x7ff80415de40 2024-05-09T10:50:56.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.937+0000 7ff808be9640 1 --2- 172.21.15.98:0/3212219261 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff7e806b4d0 0x7ff7e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:56.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.937+0000 7ff808be9640 1 --2- 172.21.15.98:0/3212219261 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff7e806b4d0 0x7ff7e806d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800b7c0 tx=0x7ff7f8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:56.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:56.941+0000 7ff8017fa640 1 -- 172.21.15.98:0/3212219261 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff7fc020070 con 0x7ff80415de40 2024-05-09T10:50:57.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.149+0000 7ff80ae4d640 1 -- 172.21.15.98:0/3212219261 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff8040d42d0 con 0x7ff80415de40 2024-05-09T10:50:57.143 INFO:tasks.ceph.mon.a.smithi098.stderr:2024-05-09T10:50:57.150+0000 7fbec99f0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T10:50:57.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.151+0000 7ff8017fa640 1 -- 172.21.15.98:0/3212219261 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff7fc059350 con 0x7ff80415de40 2024-05-09T10:50:57.145 INFO:teuthology.orchestra.run.smithi098.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T10:50:57.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.154+0000 7ff80ae4d640 1 -- 172.21.15.98:0/3212219261 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff7e806b4d0 msgr2=0x7ff7e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:57.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.154+0000 7ff80ae4d640 1 --2- 172.21.15.98:0/3212219261 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff7e806b4d0 0x7ff7e806d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800b7c0 tx=0x7ff7f8008040 comp rx=0 tx=0).stop 2024-05-09T10:50:57.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.154+0000 7ff80ae4d640 1 -- 172.21.15.98:0/3212219261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff80415de40 msgr2=0x7ff804150da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:57.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.154+0000 7ff80ae4d640 1 --2- 172.21.15.98:0/3212219261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff80415de40 0x7ff804150da0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00c7b0 tx=0x7ff7fc00cc70 comp rx=0 tx=0).stop 2024-05-09T10:50:57.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.154+0000 7ff80ae4d640 1 -- 172.21.15.98:0/3212219261 shutdown_connections 2024-05-09T10:50:57.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.154+0000 7ff80ae4d640 1 --2- 172.21.15.98:0/3212219261 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff7e806b4d0 0x7ff7e806d990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:57.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.154+0000 7ff80ae4d640 1 --2- 172.21.15.98:0/3212219261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff80415de40 0x7ff804150da0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:57.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.154+0000 7ff80ae4d640 1 --2- 172.21.15.98:0/3212219261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8040d0470 0x7ff804150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:57.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.154+0000 7ff80ae4d640 1 --2- 172.21.15.98:0/3212219261 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff8040cf250 0x7ff8041575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:57.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.154+0000 7ff80ae4d640 1 -- 172.21.15.98:0/3212219261 >> 172.21.15.98:0/3212219261 conn(0x7ff80409ce90 msgr2=0x7ff8040b7570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:57.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.155+0000 7ff80ae4d640 1 -- 172.21.15.98:0/3212219261 shutdown_connections 2024-05-09T10:50:57.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.155+0000 7ff80ae4d640 1 -- 172.21.15.98:0/3212219261 wait complete. 2024-05-09T10:50:57.151 INFO:tasks.ceph.mds.d.smithi105.stderr: -6662> 2024-05-09T10:50:23.931+0000 7f7576662640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T10:50:57.159 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T10:50:57.184 INFO:tasks.ceph.mds.d.smithi105.stdout:starting mds.d at 2024-05-09T10:50:57.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.268+0000 7f1856110640 1 Processor -- start 2024-05-09T10:50:57.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.268+0000 7f1856110640 1 -- start start 2024-05-09T10:50:57.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.269+0000 7f1856110640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f18500720d0 0x7f18500af720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:57.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.269+0000 7f1856110640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f185006abe0 0x7f185006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:57.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.269+0000 7f1856110640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1850072690 con 0x7f185015db90 2024-05-09T10:50:57.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.269+0000 7f1856110640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18500afc60 con 0x7f185006abe0 2024-05-09T10:50:57.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.269+0000 7f1856110640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18500b5d70 con 0x7f18500720d0 2024-05-09T10:50:57.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.269+0000 7f184effd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f18500720d0 0x7f18500af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:57.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.269+0000 7f184ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f185006abe0 0x7f185006aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:57.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.269+0000 7f184ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f185006abe0 0x7f185006aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.98:52512/0 (socket says 172.21.15.98:52512) 2024-05-09T10:50:57.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.269+0000 7f184effd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f18500720d0 0x7f18500af720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:42320/0 (socket says 172.21.15.98:42320) 2024-05-09T10:50:57.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.269+0000 7f184ffff640 1 -- 172.21.15.98:0/404065951 learned_addr learned my addr 172.21.15.98:0/404065951 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:57.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.269+0000 7f184ffff640 1 -- 172.21.15.98:0/404065951 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f18500720d0 msgr2=0x7f18500af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:57.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.269+0000 7f184ffff640 1 --2- 172.21.15.98:0/404065951 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f18500720d0 0x7f18500af720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:57.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.269+0000 7f184ffff640 1 -- 172.21.15.98:0/404065951 >> v1:172.21.15.105:6789/0 conn(0x7f185015db90 legacy=0x7f185006d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:57.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.269+0000 7f184ffff640 1 -- 172.21.15.98:0/404065951 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185006b530 con 0x7f185006abe0 2024-05-09T10:50:57.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.270+0000 7f184ffff640 1 --2- 172.21.15.98:0/404065951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f185006abe0 0x7f185006aff0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1848009870 tx=0x7f184802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aa9c64a766c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:57.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.270+0000 7f184e7fc640 1 -- 172.21.15.98:0/404065951 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1848037070 con 0x7f185006abe0 2024-05-09T10:50:57.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.270+0000 7f184e7fc640 1 -- 172.21.15.98:0/404065951 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1848032440 con 0x7f185006abe0 2024-05-09T10:50:57.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.270+0000 7f184e7fc640 1 -- 172.21.15.98:0/404065951 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1848032810 con 0x7f185006abe0 2024-05-09T10:50:57.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.271+0000 7f1856110640 1 -- 172.21.15.98:0/404065951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f185006abe0 msgr2=0x7f185006aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:57.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.271+0000 7f1856110640 1 --2- 172.21.15.98:0/404065951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f185006abe0 0x7f185006aff0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1848009870 tx=0x7f184802a1c0 comp rx=0 tx=0).stop 2024-05-09T10:50:57.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.271+0000 7f1856110640 1 -- 172.21.15.98:0/404065951 shutdown_connections 2024-05-09T10:50:57.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.271+0000 7f1856110640 1 --2- 172.21.15.98:0/404065951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f185006abe0 0x7f185006aff0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:57.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.271+0000 7f1856110640 1 --2- 172.21.15.98:0/404065951 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f18500720d0 0x7f18500af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:57.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.271+0000 7f1856110640 1 -- 172.21.15.98:0/404065951 >> 172.21.15.98:0/404065951 conn(0x7f1850066fd0 msgr2=0x7f185006bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:57.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.271+0000 7f1856110640 1 -- 172.21.15.98:0/404065951 shutdown_connections 2024-05-09T10:50:57.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.271+0000 7f1856110640 1 -- 172.21.15.98:0/404065951 wait complete. 2024-05-09T10:50:57.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.272+0000 7f1856110640 1 Processor -- start 2024-05-09T10:50:57.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.272+0000 7f1856110640 1 -- start start 2024-05-09T10:50:57.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.273+0000 7f1856110640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f185006abe0 0x7f185014b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:57.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.273+0000 7f1856110640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18500720d0 0x7f18501f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:57.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.273+0000 7f1856110640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f185015db90 0x7f18501f7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:57.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.273+0000 7f1856110640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18500d0670 con 0x7f18500720d0 2024-05-09T10:50:57.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.273+0000 7f1856110640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18500b5a90 con 0x7f185015db90 2024-05-09T10:50:57.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.273+0000 7f1856110640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f185015df70 con 0x7f185006abe0 2024-05-09T10:50:57.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.273+0000 7f184effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18500720d0 0x7f18501f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:57.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.273+0000 7f184effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18500720d0 0x7f18501f6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35596/0 (socket says 172.21.15.98:35596) 2024-05-09T10:50:57.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.273+0000 7f184effd640 1 -- 172.21.15.98:0/2872886846 learned_addr learned my addr 172.21.15.98:0/2872886846 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:57.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.273+0000 7f184f7fe640 1 --2- 172.21.15.98:0/2872886846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f185006abe0 0x7f185014b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:57.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.273+0000 7f184effd640 1 -- 172.21.15.98:0/2872886846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f185006abe0 msgr2=0x7f185014b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:57.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.273+0000 7f184effd640 1 --2- 172.21.15.98:0/2872886846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f185006abe0 0x7f185014b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:57.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.273+0000 7f184ffff640 1 --2- 172.21.15.98:0/2872886846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f185015db90 0x7f18501f7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:57.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.273+0000 7f184effd640 1 -- 172.21.15.98:0/2872886846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f185015db90 msgr2=0x7f18501f7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:57.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.273+0000 7f184effd640 1 --2- 172.21.15.98:0/2872886846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f185015db90 0x7f18501f7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:57.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.273+0000 7f184effd640 1 -- 172.21.15.98:0/2872886846 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1850157c80 con 0x7f18500720d0 2024-05-09T10:50:57.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.274+0000 7f184effd640 1 --2- 172.21.15.98:0/2872886846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18500720d0 0x7f18501f6f00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f184400e9b0 tx=0x7f184400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:57.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.274+0000 7f1837fff640 1 -- 172.21.15.98:0/2872886846 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1844017070 con 0x7f18500720d0 2024-05-09T10:50:57.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.274+0000 7f1837fff640 1 -- 172.21.15.98:0/2872886846 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f184400c9a0 con 0x7f18500720d0 2024-05-09T10:50:57.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.274+0000 7f1856110640 1 -- 172.21.15.98:0/2872886846 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1850157f60 con 0x7f18500720d0 2024-05-09T10:50:57.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.274+0000 7f1837fff640 1 -- 172.21.15.98:0/2872886846 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184400cd90 con 0x7f18500720d0 2024-05-09T10:50:57.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.274+0000 7f1856110640 1 -- 172.21.15.98:0/2872886846 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1850153520 con 0x7f18500720d0 2024-05-09T10:50:57.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.275+0000 7f1856110640 1 -- 172.21.15.98:0/2872886846 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1850153030 con 0x7f18500720d0 2024-05-09T10:50:57.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.275+0000 7f1837fff640 1 -- 172.21.15.98:0/2872886846 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f184401a070 con 0x7f18500720d0 2024-05-09T10:50:57.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.276+0000 7f1837fff640 1 --2- 172.21.15.98:0/2872886846 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f183c06b4d0 0x7f183c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:57.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.276+0000 7f184f7fe640 1 --2- 172.21.15.98:0/2872886846 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f183c06b4d0 0x7f183c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:57.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.276+0000 7f1837fff640 1 -- 172.21.15.98:0/2872886846 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f184408a730 con 0x7f18500720d0 2024-05-09T10:50:57.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.276+0000 7f184f7fe640 1 --2- 172.21.15.98:0/2872886846 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f183c06b4d0 0x7f183c06d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1840007fd0 tx=0x7f1840009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:57.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.280+0000 7f1837fff640 1 -- 172.21.15.98:0/2872886846 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1844059440 con 0x7f18500720d0 2024-05-09T10:50:57.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:57.489+0000 7f1856110640 1 -- 172.21.15.98:0/2872886846 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1850072dd0 con 0x7f18500720d0 2024-05-09T10:50:58.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.153+0000 7f1837fff640 1 -- 172.21.15.98:0/2872886846 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1844058d40 con 0x7f18500720d0 2024-05-09T10:50:58.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.155+0000 7f1856110640 1 -- 172.21.15.98:0/2872886846 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f183c06b4d0 msgr2=0x7f183c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:58.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.155+0000 7f1856110640 1 --2- 172.21.15.98:0/2872886846 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f183c06b4d0 0x7f183c06d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1840007fd0 tx=0x7f1840009040 comp rx=0 tx=0).stop 2024-05-09T10:50:58.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.155+0000 7f1856110640 1 -- 172.21.15.98:0/2872886846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18500720d0 msgr2=0x7f18501f6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:58.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.155+0000 7f1856110640 1 --2- 172.21.15.98:0/2872886846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18500720d0 0x7f18501f6f00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f184400e9b0 tx=0x7f184400ee70 comp rx=0 tx=0).stop 2024-05-09T10:50:58.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.156+0000 7f1856110640 1 -- 172.21.15.98:0/2872886846 shutdown_connections 2024-05-09T10:50:58.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.156+0000 7f1856110640 1 --2- 172.21.15.98:0/2872886846 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f183c06b4d0 0x7f183c06d990 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:58.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.156+0000 7f1856110640 1 --2- 172.21.15.98:0/2872886846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f185015db90 0x7f18501f7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:58.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.156+0000 7f1856110640 1 --2- 172.21.15.98:0/2872886846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18500720d0 0x7f18501f6f00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:58.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.156+0000 7f1856110640 1 --2- 172.21.15.98:0/2872886846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f185006abe0 0x7f185014b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:58.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.156+0000 7f1856110640 1 -- 172.21.15.98:0/2872886846 >> 172.21.15.98:0/2872886846 conn(0x7f1850066fd0 msgr2=0x7f18500ad880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:58.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.156+0000 7f1856110640 1 -- 172.21.15.98:0/2872886846 shutdown_connections 2024-05-09T10:50:58.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.156+0000 7f1856110640 1 -- 172.21.15.98:0/2872886846 wait complete. 2024-05-09T10:50:58.163 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T10:50:58.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.278+0000 7faab8142640 1 Processor -- start 2024-05-09T10:50:58.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.278+0000 7faab8142640 1 -- start start 2024-05-09T10:50:58.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.279+0000 7faab8142640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faab006e4b0 0x7faab015fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:58.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.279+0000 7faab8142640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab006ea70 con 0x7faab015f140 2024-05-09T10:50:58.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.279+0000 7faab8142640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab006ed50 con 0x7faab0160320 2024-05-09T10:50:58.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.279+0000 7faab8142640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab015f520 con 0x7faab006e4b0 2024-05-09T10:50:58.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.279+0000 7faab5ede640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7faab015f140 0x7faab00af9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38874/0 (socket says 172.21.15.98:38874) 2024-05-09T10:50:58.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.279+0000 7faab5ede640 1 -- 172.21.15.98:0/523901614 learned_addr learned my addr 172.21.15.98:0/523901614 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:58.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.279+0000 7faab4edc640 1 -- 172.21.15.98:0/523901614 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 569693859 0 0) 0x7faab006ea70 con 0x7faab015f140 2024-05-09T10:50:58.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.279+0000 7faab4edc640 1 -- 172.21.15.98:0/523901614 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa9c003520 con 0x7faab015f140 2024-05-09T10:50:58.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.279+0000 7faab56dd640 1 --2- 172.21.15.98:0/523901614 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faab006e4b0 0x7faab015fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:58.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.279+0000 7faab4edc640 1 -- 172.21.15.98:0/523901614 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 50101855 0 0) 0x7faa9c003520 con 0x7faab015f140 2024-05-09T10:50:58.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.279+0000 7faab4edc640 1 -- 172.21.15.98:0/523901614 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faab006e4b0 msgr2=0x7faab015fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:58.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.280+0000 7faab4edc640 1 --2- 172.21.15.98:0/523901614 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faab006e4b0 0x7faab015fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:58.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.280+0000 7faab4edc640 1 -- 172.21.15.98:0/523901614 >> v1:172.21.15.105:6789/0 conn(0x7faab0160320 legacy=0x7faab00d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:58.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.280+0000 7faab4edc640 1 -- 172.21.15.98:0/523901614 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab006f030 con 0x7faab015f140 2024-05-09T10:50:58.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.280+0000 7faab4edc640 1 -- 172.21.15.98:0/523901614 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7faaac001670 con 0x7faab015f140 2024-05-09T10:50:58.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.280+0000 7faab56dd640 1 --2- 172.21.15.98:0/523901614 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faab006e4b0 0x7faab015fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:50:58.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.280+0000 7faab4edc640 1 -- 172.21.15.98:0/523901614 >> v1:172.21.15.98:6789/0 conn(0x7faab015f140 legacy=0x7faab00af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:58.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.280+0000 7faab4edc640 1 --2- 172.21.15.98:0/523901614 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faab006e4b0 0x7faa9c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:58.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.280+0000 7faab4edc640 1 --2- 172.21.15.98:0/523901614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa9c003fd0 0x7faa9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:58.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.280+0000 7faab4edc640 1 --2- 172.21.15.98:0/523901614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa9c004d40 0x7faa9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:58.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.280+0000 7faab4edc640 1 -- 172.21.15.98:0/523901614 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c00baf0 con 0x7faa9c004d40 2024-05-09T10:50:58.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.280+0000 7faab4edc640 1 -- 172.21.15.98:0/523901614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c00bca0 con 0x7faa9c003fd0 2024-05-09T10:50:58.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.280+0000 7faab4edc640 1 -- 172.21.15.98:0/523901614 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c00bea0 con 0x7faab006e4b0 2024-05-09T10:50:58.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.280+0000 7faab66df640 1 --2- 172.21.15.98:0/523901614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa9c004d40 0x7faa9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:58.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.280+0000 7faab5ede640 1 --2- 172.21.15.98:0/523901614 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faab006e4b0 0x7faa9c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:58.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.280+0000 7faab56dd640 1 --2- 172.21.15.98:0/523901614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa9c003fd0 0x7faa9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:58.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.280+0000 7faab66df640 1 -- 172.21.15.98:0/523901614 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faab006e4b0 msgr2=0x7faa9c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:58.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.280+0000 7faab66df640 1 --2- 172.21.15.98:0/523901614 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faab006e4b0 0x7faa9c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:58.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.280+0000 7faab66df640 1 -- 172.21.15.98:0/523901614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa9c003fd0 msgr2=0x7faa9c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:58.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.280+0000 7faab66df640 1 --2- 172.21.15.98:0/523901614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa9c003fd0 0x7faa9c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:58.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.280+0000 7faab66df640 1 -- 172.21.15.98:0/523901614 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9c003520 con 0x7faa9c004d40 2024-05-09T10:50:58.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.281+0000 7faab66df640 1 --2- 172.21.15.98:0/523901614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa9c004d40 0x7faa9c0051c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7faaa8001e50 tx=0x7faa9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73905a984617f85e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:58.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.281+0000 7faab4edc640 1 -- 172.21.15.98:0/523901614 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa8001a70 con 0x7faa9c004d40 2024-05-09T10:50:58.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.281+0000 7faab4edc640 1 -- 172.21.15.98:0/523901614 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa8003970 con 0x7faa9c004d40 2024-05-09T10:50:58.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.281+0000 7faab4edc640 1 -- 172.21.15.98:0/523901614 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa8003d60 con 0x7faa9c004d40 2024-05-09T10:50:58.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.281+0000 7faab8142640 1 -- 172.21.15.98:0/523901614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa9c004d40 msgr2=0x7faa9c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:58.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.281+0000 7faab8142640 1 --2- 172.21.15.98:0/523901614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa9c004d40 0x7faa9c0051c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7faaa8001e50 tx=0x7faa9c005830 comp rx=0 tx=0).stop 2024-05-09T10:50:58.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.281+0000 7faab66df640 1 -- 172.21.15.98:0/523901614 reap_dead start 2024-05-09T10:50:58.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.281+0000 7faab8142640 1 -- 172.21.15.98:0/523901614 shutdown_connections 2024-05-09T10:50:58.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.281+0000 7faab8142640 1 -- 172.21.15.98:0/523901614 >> 172.21.15.98:0/523901614 conn(0x7faab0066fd0 msgr2=0x7faab006bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:58.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.281+0000 7faab8142640 1 -- 172.21.15.98:0/523901614 shutdown_connections 2024-05-09T10:50:58.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.281+0000 7faab8142640 1 -- 172.21.15.98:0/523901614 wait complete. 2024-05-09T10:50:58.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.283+0000 7faab8142640 1 Processor -- start 2024-05-09T10:50:58.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.283+0000 7faab8142640 1 -- start start 2024-05-09T10:50:58.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.283+0000 7faab8142640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa9c004d40 0x7faab013fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:58.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.283+0000 7faab8142640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faab0159000 0x7faab0159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:58.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.283+0000 7faab8142640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faab0142520 0x7faab01429a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:58.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.283+0000 7faab8142640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab015ff60 con 0x7faab0142520 2024-05-09T10:50:58.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.283+0000 7faab8142640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab015fde0 con 0x7faa9c004d40 2024-05-09T10:50:58.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.283+0000 7faab8142640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab01600e0 con 0x7faab0159000 2024-05-09T10:50:58.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.283+0000 7faab66df640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faab0142520 0x7faab01429a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:58.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.283+0000 7faab66df640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faab0142520 0x7faab01429a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35610/0 (socket says 172.21.15.98:35610) 2024-05-09T10:50:58.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.284+0000 7faab66df640 1 -- 172.21.15.98:0/3244607493 learned_addr learned my addr 172.21.15.98:0/3244607493 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:58.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.284+0000 7faab5ede640 1 --2- 172.21.15.98:0/3244607493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa9c004d40 0x7faab013fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:58.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.284+0000 7faab56dd640 1 --2- 172.21.15.98:0/3244607493 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faab0159000 0x7faab0159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:58.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.284+0000 7faab66df640 1 -- 172.21.15.98:0/3244607493 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faab0159000 msgr2=0x7faab0159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:58.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.284+0000 7faab66df640 1 --2- 172.21.15.98:0/3244607493 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faab0159000 0x7faab0159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:58.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.284+0000 7faab66df640 1 -- 172.21.15.98:0/3244607493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa9c004d40 msgr2=0x7faab013fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:58.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.284+0000 7faab66df640 1 --2- 172.21.15.98:0/3244607493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa9c004d40 0x7faab013fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:58.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.284+0000 7faab66df640 1 -- 172.21.15.98:0/3244607493 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab01432b0 con 0x7faab0142520 2024-05-09T10:50:58.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.284+0000 7faab5ede640 1 --2- 172.21.15.98:0/3244607493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa9c004d40 0x7faab013fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:58.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.284+0000 7faab56dd640 1 --2- 172.21.15.98:0/3244607493 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faab0159000 0x7faab0159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:58.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.284+0000 7faab66df640 1 --2- 172.21.15.98:0/3244607493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faab0142520 0x7faab01429a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7faaa8001f40 tx=0x7faaa80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:58.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.284+0000 7faaa67fc640 1 -- 172.21.15.98:0/3244607493 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa8041070 con 0x7faab0142520 2024-05-09T10:50:58.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.284+0000 7faaa67fc640 1 -- 172.21.15.98:0/3244607493 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa8006d60 con 0x7faab0142520 2024-05-09T10:50:58.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.284+0000 7faaa67fc640 1 -- 172.21.15.98:0/3244607493 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa8007db0 con 0x7faab0142520 2024-05-09T10:50:58.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.284+0000 7faab8142640 1 -- 172.21.15.98:0/3244607493 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab00ad990 con 0x7faab0142520 2024-05-09T10:50:58.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.284+0000 7faab8142640 1 -- 172.21.15.98:0/3244607493 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab00adf50 con 0x7faab0142520 2024-05-09T10:50:58.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.285+0000 7faab8142640 1 -- 172.21.15.98:0/3244607493 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab0072dd0 con 0x7faab0142520 2024-05-09T10:50:58.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.286+0000 7faaa67fc640 1 -- 172.21.15.98:0/3244607493 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faaa8044070 con 0x7faab0142520 2024-05-09T10:50:58.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.286+0000 7faaa67fc640 1 --2- 172.21.15.98:0/3244607493 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7faa9406b4d0 0x7faa9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:58.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.286+0000 7faaa67fc640 1 -- 172.21.15.98:0/3244607493 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5795+0+0 (secure 0 0 0) 0x7faaa80a9390 con 0x7faab0142520 2024-05-09T10:50:58.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.286+0000 7faab5ede640 1 --2- 172.21.15.98:0/3244607493 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7faa9406b4d0 0x7faa9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:58.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.286+0000 7faab5ede640 1 --2- 172.21.15.98:0/3244607493 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7faa9406b4d0 0x7faa9406d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faab0159c30 tx=0x7faaac009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:58.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.290+0000 7faaa67fc640 1 -- 172.21.15.98:0/3244607493 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faaa80782a0 con 0x7faab0142520 2024-05-09T10:50:58.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:58.463+0000 7faab8142640 1 -- 172.21.15.98:0/3244607493 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faab004f000 con 0x7faab0142520 2024-05-09T10:50:59.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.352+0000 7faaa67fc640 1 -- 172.21.15.98:0/3244607493 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v22) v1 ==== 181+0+0 (secure 0 0 0) 0x7faaa8077ba0 con 0x7faab0142520 2024-05-09T10:50:59.345 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' removed 2024-05-09T10:50:59.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.357+0000 7faab8142640 1 -- 172.21.15.98:0/3244607493 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7faa9406b4d0 msgr2=0x7faa9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:59.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.357+0000 7faab8142640 1 --2- 172.21.15.98:0/3244607493 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7faa9406b4d0 0x7faa9406d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faab0159c30 tx=0x7faaac009040 comp rx=0 tx=0).stop 2024-05-09T10:50:59.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.357+0000 7faab8142640 1 -- 172.21.15.98:0/3244607493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faab0142520 msgr2=0x7faab01429a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:59.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.357+0000 7faab8142640 1 --2- 172.21.15.98:0/3244607493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faab0142520 0x7faab01429a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7faaa8001f40 tx=0x7faaa80014a0 comp rx=0 tx=0).stop 2024-05-09T10:50:59.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.357+0000 7faab8142640 1 -- 172.21.15.98:0/3244607493 shutdown_connections 2024-05-09T10:50:59.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.357+0000 7faab8142640 1 --2- 172.21.15.98:0/3244607493 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7faa9406b4d0 0x7faa9406d990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:59.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.357+0000 7faab8142640 1 --2- 172.21.15.98:0/3244607493 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faab0142520 0x7faab01429a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:59.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.357+0000 7faab8142640 1 --2- 172.21.15.98:0/3244607493 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faab0159000 0x7faab0159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:59.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.357+0000 7faab8142640 1 --2- 172.21.15.98:0/3244607493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa9c004d40 0x7faab013fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:59.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.357+0000 7faab8142640 1 -- 172.21.15.98:0/3244607493 >> 172.21.15.98:0/3244607493 conn(0x7faab0066fd0 msgr2=0x7faab00b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:59.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.357+0000 7faab8142640 1 -- 172.21.15.98:0/3244607493 shutdown_connections 2024-05-09T10:50:59.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.357+0000 7faab8142640 1 -- 172.21.15.98:0/3244607493 wait complete. 2024-05-09T10:50:59.365 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T10:50:59.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.480+0000 7f8de91c4640 1 Processor -- start 2024-05-09T10:50:59.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.480+0000 7f8de91c4640 1 -- start start 2024-05-09T10:50:59.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.481+0000 7f8de91c4640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8de415bc80 0x7f8de40cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:59.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.481+0000 7f8de91c4640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8de415c060 0x7f8de40cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:59.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.481+0000 7f8de91c4640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8de40ce380 con 0x7f8de40cd5d0 2024-05-09T10:50:59.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.481+0000 7f8de91c4640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8de40ce660 con 0x7f8de415bc80 2024-05-09T10:50:59.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.481+0000 7f8de91c4640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de40ce940 con 0x7f8de415c060 2024-05-09T10:50:59.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.481+0000 7f8de359e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f8de40cd5d0 0x7f8de415f9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38884/0 (socket says 172.21.15.98:38884) 2024-05-09T10:50:59.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.481+0000 7f8de359e640 1 -- 172.21.15.98:0/379589126 learned_addr learned my addr 172.21.15.98:0/379589126 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:50:59.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.481+0000 7f8ddbfff640 1 --2- 172.21.15.98:0/379589126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8de415c060 0x7f8de40cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:59.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.481+0000 7f8de2d9d640 1 --2- 172.21.15.98:0/379589126 >> v2:172.21.15.105:3300/0 conn(0x7f8de415bc80 0x7f8de40cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:59.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.481+0000 7f8de259c640 1 -- 172.21.15.98:0/379589126 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 498126301 0 0) 0x7f8de40ce380 con 0x7f8de40cd5d0 2024-05-09T10:50:59.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.481+0000 7f8de259c640 1 -- 172.21.15.98:0/379589126 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dd0003520 con 0x7f8de40cd5d0 2024-05-09T10:50:59.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.481+0000 7f8de259c640 1 -- 172.21.15.98:0/379589126 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2127256994 0 0) 0x7f8dd0003520 con 0x7f8de40cd5d0 2024-05-09T10:50:59.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.481+0000 7f8de259c640 1 -- 172.21.15.98:0/379589126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8de415c060 msgr2=0x7f8de40cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:59.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.481+0000 7f8de259c640 1 --2- 172.21.15.98:0/379589126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8de415c060 0x7f8de40cff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:59.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.481+0000 7f8de259c640 1 -- 172.21.15.98:0/379589126 >> v2:172.21.15.105:3300/0 conn(0x7f8de415bc80 msgr2=0x7f8de40cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:59.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.481+0000 7f8de259c640 1 --2- 172.21.15.98:0/379589126 >> v2:172.21.15.105:3300/0 conn(0x7f8de415bc80 0x7f8de40cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:59.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.481+0000 7f8de259c640 1 -- 172.21.15.98:0/379589126 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de40cec20 con 0x7f8de40cd5d0 2024-05-09T10:50:59.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.481+0000 7f8de259c640 1 -- 172.21.15.98:0/379589126 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f8ddc0019f0 con 0x7f8de40cd5d0 2024-05-09T10:50:59.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.481+0000 7f8ddbfff640 1 --2- 172.21.15.98:0/379589126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8de415c060 0x7f8de40cff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:50:59.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.481+0000 7f8de259c640 1 -- 172.21.15.98:0/379589126 >> v1:172.21.15.98:6789/0 conn(0x7f8de40cd5d0 legacy=0x7f8de415f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:59.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.481+0000 7f8de259c640 1 --2- 172.21.15.98:0/379589126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dd00039b0 0x7f8dd0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:59.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.482+0000 7f8de259c640 1 --2- 172.21.15.98:0/379589126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8de415c060 0x7f8dd0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:59.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.482+0000 7f8de259c640 1 --2- 172.21.15.98:0/379589126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dd0004d40 0x7f8dd00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:59.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.482+0000 7f8de259c640 1 -- 172.21.15.98:0/379589126 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de40ce660 con 0x7f8dd0004d40 2024-05-09T10:50:59.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.482+0000 7f8de259c640 1 -- 172.21.15.98:0/379589126 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de40ce940 con 0x7f8dd00039b0 2024-05-09T10:50:59.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.482+0000 7f8de259c640 1 -- 172.21.15.98:0/379589126 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd000b9c0 con 0x7f8de415c060 2024-05-09T10:50:59.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.482+0000 7f8de359e640 1 --2- 172.21.15.98:0/379589126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dd0004d40 0x7f8dd00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:59.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.482+0000 7f8de2d9d640 1 --2- 172.21.15.98:0/379589126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dd00039b0 0x7f8dd0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:59.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.482+0000 7f8ddbfff640 1 --2- 172.21.15.98:0/379589126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8de415c060 0x7f8dd0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:59.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.482+0000 7f8de359e640 1 -- 172.21.15.98:0/379589126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8de415c060 msgr2=0x7f8dd0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:59.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.482+0000 7f8de359e640 1 --2- 172.21.15.98:0/379589126 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8de415c060 0x7f8dd0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:59.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.482+0000 7f8de359e640 1 -- 172.21.15.98:0/379589126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dd00039b0 msgr2=0x7f8dd0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:50:59.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.482+0000 7f8de359e640 1 --2- 172.21.15.98:0/379589126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dd00039b0 0x7f8dd0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:59.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.482+0000 7f8de359e640 1 -- 172.21.15.98:0/379589126 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd0003520 con 0x7f8dd0004d40 2024-05-09T10:50:59.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.483+0000 7f8de359e640 1 --2- 172.21.15.98:0/379589126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dd0004d40 0x7f8dd00051c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00a870 tx=0x7f8dd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6129611c42a9d70 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:59.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.483+0000 7f8de259c640 1 -- 172.21.15.98:0/379589126 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc00a4c0 con 0x7f8dd0004d40 2024-05-09T10:50:59.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.483+0000 7f8de259c640 1 -- 172.21.15.98:0/379589126 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ddc02f070 con 0x7f8dd0004d40 2024-05-09T10:50:59.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.483+0000 7f8de259c640 1 -- 172.21.15.98:0/379589126 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc02bce0 con 0x7f8dd0004d40 2024-05-09T10:50:59.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.483+0000 7f8de91c4640 1 -- 172.21.15.98:0/379589126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dd0004d40 msgr2=0x7f8dd00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:59.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.483+0000 7f8de91c4640 1 --2- 172.21.15.98:0/379589126 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8dd0004d40 0x7f8dd00051c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00a870 tx=0x7f8dd0005830 comp rx=0 tx=0).stop 2024-05-09T10:50:59.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.483+0000 7f8de359e640 1 -- 172.21.15.98:0/379589126 reap_dead start 2024-05-09T10:50:59.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.483+0000 7f8de91c4640 1 -- 172.21.15.98:0/379589126 shutdown_connections 2024-05-09T10:50:59.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.483+0000 7f8de91c4640 1 -- 172.21.15.98:0/379589126 >> 172.21.15.98:0/379589126 conn(0x7f8de40d1970 msgr2=0x7f8de40d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:50:59.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.483+0000 7f8de91c4640 1 -- 172.21.15.98:0/379589126 shutdown_connections 2024-05-09T10:50:59.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.483+0000 7f8de91c4640 1 -- 172.21.15.98:0/379589126 wait complete. 2024-05-09T10:50:59.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.485+0000 7f8de91c4640 1 Processor -- start 2024-05-09T10:50:59.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.485+0000 7f8de91c4640 1 -- start start 2024-05-09T10:50:59.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.485+0000 7f8de91c4640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8dd0004d40 0x7f8de40ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:59.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.485+0000 7f8de91c4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8de4154150 0x7f8de41545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:59.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.485+0000 7f8de91c4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8de4152c40 0x7f8de41530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:59.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.485+0000 7f8de91c4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de40ce120 con 0x7f8de4154150 2024-05-09T10:50:59.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.485+0000 7f8de91c4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de40cdfa0 con 0x7f8de4152c40 2024-05-09T10:50:59.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.485+0000 7f8de91c4640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de40b1ef0 con 0x7f8dd0004d40 2024-05-09T10:50:59.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.486+0000 7f8ddbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8de4154150 0x7f8de41545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:59.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.486+0000 7f8ddbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8de4154150 0x7f8de41545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35624/0 (socket says 172.21.15.98:35624) 2024-05-09T10:50:59.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.486+0000 7f8ddbfff640 1 -- 172.21.15.98:0/160920515 learned_addr learned my addr 172.21.15.98:0/160920515 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:50:59.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.486+0000 7f8de2d9d640 1 --2- 172.21.15.98:0/160920515 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8dd0004d40 0x7f8de40ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:59.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.486+0000 7f8de359e640 1 --2- 172.21.15.98:0/160920515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8de4152c40 0x7f8de41530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:59.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.486+0000 7f8ddbfff640 1 -- 172.21.15.98:0/160920515 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8dd0004d40 msgr2=0x7f8de40ca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:59.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.486+0000 7f8ddbfff640 1 --2- 172.21.15.98:0/160920515 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8dd0004d40 0x7f8de40ca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:59.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.486+0000 7f8ddbfff640 1 -- 172.21.15.98:0/160920515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8de4152c40 msgr2=0x7f8de41530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:50:59.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.486+0000 7f8ddbfff640 1 --2- 172.21.15.98:0/160920515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8de4152c40 0x7f8de41530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:50:59.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.486+0000 7f8ddbfff640 1 -- 172.21.15.98:0/160920515 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de415c750 con 0x7f8de4154150 2024-05-09T10:50:59.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.486+0000 7f8de2d9d640 1 --2- 172.21.15.98:0/160920515 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8dd0004d40 0x7f8de40ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:50:59.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.486+0000 7f8ddbfff640 1 --2- 172.21.15.98:0/160920515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8de4154150 0x7f8de41545b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc011a00 tx=0x7f8dcc011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:59.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.486+0000 7f8ddb7fe640 1 -- 172.21.15.98:0/160920515 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc00faf0 con 0x7f8de4154150 2024-05-09T10:50:59.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.486+0000 7f8ddb7fe640 1 -- 172.21.15.98:0/160920515 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dcc00fc90 con 0x7f8de4154150 2024-05-09T10:50:59.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.486+0000 7f8de91c4640 1 -- 172.21.15.98:0/160920515 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de415ca30 con 0x7f8de4154150 2024-05-09T10:50:59.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.486+0000 7f8ddb7fe640 1 -- 172.21.15.98:0/160920515 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc004030 con 0x7f8de4154150 2024-05-09T10:50:59.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.486+0000 7f8de91c4640 1 -- 172.21.15.98:0/160920515 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de415cff0 con 0x7f8de4154150 2024-05-09T10:50:59.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.487+0000 7f8de91c4640 1 -- 172.21.15.98:0/160920515 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de406ba50 con 0x7f8de4154150 2024-05-09T10:50:59.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.488+0000 7f8ddb7fe640 1 -- 172.21.15.98:0/160920515 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8dcc012040 con 0x7f8de4154150 2024-05-09T10:50:59.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.488+0000 7f8ddb7fe640 1 --2- 172.21.15.98:0/160920515 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8dc406b4d0 0x7f8dc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:50:59.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.488+0000 7f8ddb7fe640 1 -- 172.21.15.98:0/160920515 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f8dcc08a4e0 con 0x7f8de4154150 2024-05-09T10:50:59.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.488+0000 7f8de2d9d640 1 --2- 172.21.15.98:0/160920515 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8dc406b4d0 0x7f8dc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:50:59.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.489+0000 7f8de2d9d640 1 --2- 172.21.15.98:0/160920515 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8dc406b4d0 0x7f8dc406d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4009ab0 tx=0x7f8dd4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:50:59.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.492+0000 7f8ddb7fe640 1 -- 172.21.15.98:0/160920515 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8dcc019070 con 0x7f8de4154150 2024-05-09T10:50:59.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:50:59.670+0000 7f8de91c4640 1 -- 172.21.15.98:0/160920515 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8de415a9e0 con 0x7f8de4154150 2024-05-09T10:51:00.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.357+0000 7f8ddb7fe640 1 -- 172.21.15.98:0/160920515 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8dcc0594c0 con 0x7f8de4154150 2024-05-09T10:51:00.351 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' removed 2024-05-09T10:51:00.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.362+0000 7f8de91c4640 1 -- 172.21.15.98:0/160920515 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8dc406b4d0 msgr2=0x7f8dc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:00.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.362+0000 7f8de91c4640 1 --2- 172.21.15.98:0/160920515 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8dc406b4d0 0x7f8dc406d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4009ab0 tx=0x7f8dd4008040 comp rx=0 tx=0).stop 2024-05-09T10:51:00.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.362+0000 7f8de91c4640 1 -- 172.21.15.98:0/160920515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8de4154150 msgr2=0x7f8de41545b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:00.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.362+0000 7f8de91c4640 1 --2- 172.21.15.98:0/160920515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8de4154150 0x7f8de41545b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc011a00 tx=0x7f8dcc011ec0 comp rx=0 tx=0).stop 2024-05-09T10:51:00.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.362+0000 7f8de91c4640 1 -- 172.21.15.98:0/160920515 shutdown_connections 2024-05-09T10:51:00.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.362+0000 7f8de91c4640 1 --2- 172.21.15.98:0/160920515 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8dc406b4d0 0x7f8dc406d990 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:00.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.362+0000 7f8de91c4640 1 --2- 172.21.15.98:0/160920515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8de4152c40 0x7f8de41530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:00.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.362+0000 7f8de91c4640 1 --2- 172.21.15.98:0/160920515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8de4154150 0x7f8de41545b0 secure :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc011a00 tx=0x7f8dcc011ec0 comp rx=0 tx=0).stop 2024-05-09T10:51:00.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.362+0000 7f8de91c4640 1 --2- 172.21.15.98:0/160920515 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8dd0004d40 0x7f8de40ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:00.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.362+0000 7f8de91c4640 1 -- 172.21.15.98:0/160920515 >> 172.21.15.98:0/160920515 conn(0x7f8de40d1970 msgr2=0x7f8de40aeaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:00.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.363+0000 7f8de91c4640 1 -- 172.21.15.98:0/160920515 shutdown_connections 2024-05-09T10:51:00.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.363+0000 7f8de91c4640 1 -- 172.21.15.98:0/160920515 wait complete. 2024-05-09T10:51:00.372 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-09T10:51:00.372 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-09T10:51:00.372 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-09T10:51:00.372 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T10:51:00.372 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-09T10:51:00.372 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-09T10:51:00.373 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-09T10:51:00.373 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T10:51:00.373 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-09T10:51:00.373 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-09T10:51:00.374 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-09T10:51:00.374 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T10:51:00.374 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-09T10:51:00.374 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-09T10:51:00.374 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-09T10:51:00.374 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T10:51:00.383 INFO:tasks.ceph.mds.a:Stopped 2024-05-09T10:51:00.383 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-09T10:51:00.384 INFO:tasks.ceph.mds.c:Stopped 2024-05-09T10:51:00.384 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-09T10:51:00.385 INFO:tasks.ceph.mds.b:Stopped 2024-05-09T10:51:00.385 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-09T10:51:00.386 INFO:tasks.ceph.mds.d:Stopped 2024-05-09T10:51:00.386 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-09T10:51:00.426 INFO:tasks.ceph.mds.a:Started 2024-05-09T10:51:00.426 DEBUG:teuthology.parallel:result is None 2024-05-09T10:51:00.427 INFO:tasks.ceph.mds.c:Started 2024-05-09T10:51:00.427 DEBUG:teuthology.parallel:result is None 2024-05-09T10:51:00.428 INFO:tasks.ceph.mds.b:Started 2024-05-09T10:51:00.428 DEBUG:teuthology.parallel:result is None 2024-05-09T10:51:00.429 INFO:tasks.ceph.mds.d:Started 2024-05-09T10:51:00.429 DEBUG:teuthology.parallel:result is None 2024-05-09T10:51:00.429 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-09T10:51:00.507 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2024-05-09T10:51:00.508 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2024-05-09T10:51:00.512 INFO:tasks.ceph.mds.d.smithi105.stdout:starting mds.d at 2024-05-09T10:51:00.512 INFO:tasks.ceph.mds.b.smithi105.stdout:starting mds.b at 2024-05-09T10:51:00.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.548+0000 7f103a1f9640 1 Processor -- start 2024-05-09T10:51:00.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.548+0000 7f103a1f9640 1 -- start start 2024-05-09T10:51:00.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.549+0000 7f103a1f9640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f103415b9e0 0x7f10340d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:00.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.549+0000 7f103a1f9640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f10340af6e0 0x7f103406abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:00.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.549+0000 7f103a1f9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f103406b120 0x7f10340713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:00.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.549+0000 7f103a1f9640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f103406f830 con 0x7f10340af6e0 2024-05-09T10:51:00.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.549+0000 7f103a1f9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f103406fdb0 con 0x7f103406b120 2024-05-09T10:51:00.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.549+0000 7f103a1f9640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1034160210 con 0x7f103415b9e0 2024-05-09T10:51:00.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.549+0000 7f1032ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f10340af6e0 0x7f103406abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:00.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.549+0000 7f1032ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f10340af6e0 0x7f103406abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35674/0 (socket says 172.21.15.98:35674) 2024-05-09T10:51:00.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.549+0000 7f1032ffd640 1 -- 172.21.15.98:0/2374502330 learned_addr learned my addr 172.21.15.98:0/2374502330 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:00.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.549+0000 7f1033fff640 1 --2- 172.21.15.98:0/2374502330 >> v2:172.21.15.105:3300/0 conn(0x7f103406b120 0x7f10340713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:00.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.549+0000 7f10337fe640 1 --2- 172.21.15.98:0/2374502330 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f103415b9e0 0x7f10340d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:00.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.549+0000 7f1032ffd640 1 -- 172.21.15.98:0/2374502330 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f103415b9e0 msgr2=0x7f10340d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:00.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.549+0000 7f1032ffd640 1 --2- 172.21.15.98:0/2374502330 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f103415b9e0 0x7f10340d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:00.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.549+0000 7f1032ffd640 1 -- 172.21.15.98:0/2374502330 >> v2:172.21.15.105:3300/0 conn(0x7f103406b120 msgr2=0x7f10340713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:00.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.549+0000 7f1032ffd640 1 --2- 172.21.15.98:0/2374502330 >> v2:172.21.15.105:3300/0 conn(0x7f103406b120 0x7f10340713f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:00.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.549+0000 7f1032ffd640 1 -- 172.21.15.98:0/2374502330 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103406fad0 con 0x7f10340af6e0 2024-05-09T10:51:00.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.550+0000 7f1032ffd640 1 --2- 172.21.15.98:0/2374502330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10340af6e0 0x7f103406abe0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f10340ce0d0 tx=0x7f102802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0fd0845248a36 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:00.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.550+0000 7f10327fc640 1 -- 172.21.15.98:0/2374502330 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028037070 con 0x7f10340af6e0 2024-05-09T10:51:00.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.550+0000 7f10327fc640 1 -- 172.21.15.98:0/2374502330 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1028032440 con 0x7f10340af6e0 2024-05-09T10:51:00.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.550+0000 7f10327fc640 1 -- 172.21.15.98:0/2374502330 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028032810 con 0x7f10340af6e0 2024-05-09T10:51:00.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.551+0000 7f103a1f9640 1 -- 172.21.15.98:0/2374502330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10340af6e0 msgr2=0x7f103406abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:00.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.551+0000 7f103a1f9640 1 --2- 172.21.15.98:0/2374502330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10340af6e0 0x7f103406abe0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f10340ce0d0 tx=0x7f102802a1e0 comp rx=0 tx=0).stop 2024-05-09T10:51:00.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.551+0000 7f103a1f9640 1 -- 172.21.15.98:0/2374502330 shutdown_connections 2024-05-09T10:51:00.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.551+0000 7f103a1f9640 1 --2- 172.21.15.98:0/2374502330 >> v2:172.21.15.105:3300/0 conn(0x7f103406b120 0x7f10340713f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:00.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.551+0000 7f103a1f9640 1 --2- 172.21.15.98:0/2374502330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10340af6e0 0x7f103406abe0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:00.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.551+0000 7f103a1f9640 1 --2- 172.21.15.98:0/2374502330 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f103415b9e0 0x7f10340d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:00.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.551+0000 7f103a1f9640 1 -- 172.21.15.98:0/2374502330 >> 172.21.15.98:0/2374502330 conn(0x7f10340b24c0 msgr2=0x7f10340b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:00.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.551+0000 7f103a1f9640 1 -- 172.21.15.98:0/2374502330 shutdown_connections 2024-05-09T10:51:00.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.551+0000 7f103a1f9640 1 -- 172.21.15.98:0/2374502330 wait complete. 2024-05-09T10:51:00.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.552+0000 7f103a1f9640 1 Processor -- start 2024-05-09T10:51:00.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.552+0000 7f103a1f9640 1 -- start start 2024-05-09T10:51:00.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.553+0000 7f103a1f9640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f103406b120 0x7f103414b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:00.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.553+0000 7f103a1f9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10340af6e0 0x7f103415e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:00.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.553+0000 7f103a1f9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f103415b9e0 0x7f103415c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:00.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.553+0000 7f103a1f9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103406d7c0 con 0x7f103415b9e0 2024-05-09T10:51:00.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.553+0000 7f103a1f9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103406d640 con 0x7f10340af6e0 2024-05-09T10:51:00.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.553+0000 7f103a1f9640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103415bdc0 con 0x7f103406b120 2024-05-09T10:51:00.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.553+0000 7f10337fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f103406b120 0x7f103414b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:00.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.553+0000 7f1033fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f103415b9e0 0x7f103415c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:00.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.553+0000 7f10337fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f103406b120 0x7f103414b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:42398/0 (socket says 172.21.15.98:42398) 2024-05-09T10:51:00.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.553+0000 7f10337fe640 1 -- 172.21.15.98:0/1671659856 learned_addr learned my addr 172.21.15.98:0/1671659856 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:00.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.553+0000 7f1032ffd640 1 --2- 172.21.15.98:0/1671659856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10340af6e0 0x7f103415e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:00.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.553+0000 7f1033fff640 1 -- 172.21.15.98:0/1671659856 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f103406b120 msgr2=0x7f103414b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:00.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.553+0000 7f1033fff640 1 --2- 172.21.15.98:0/1671659856 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f103406b120 0x7f103414b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:00.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.553+0000 7f1033fff640 1 -- 172.21.15.98:0/1671659856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10340af6e0 msgr2=0x7f103415e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:00.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.553+0000 7f1033fff640 1 --2- 172.21.15.98:0/1671659856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10340af6e0 0x7f103415e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:00.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.553+0000 7f1033fff640 1 -- 172.21.15.98:0/1671659856 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10340b38c0 con 0x7f103415b9e0 2024-05-09T10:51:00.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.553+0000 7f10337fe640 1 --2- 172.21.15.98:0/1671659856 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f103406b120 0x7f103414b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:51:00.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.553+0000 7f1032ffd640 1 --2- 172.21.15.98:0/1671659856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10340af6e0 0x7f103415e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:51:00.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.553+0000 7f1033fff640 1 --2- 172.21.15.98:0/1671659856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f103415b9e0 0x7f103415c160 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f102c009d90 tx=0x7f102c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:00.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.554+0000 7f101bfff640 1 -- 172.21.15.98:0/1671659856 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c018020 con 0x7f103415b9e0 2024-05-09T10:51:00.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.554+0000 7f101bfff640 1 -- 172.21.15.98:0/1671659856 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102c014480 con 0x7f103415b9e0 2024-05-09T10:51:00.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.554+0000 7f101bfff640 1 -- 172.21.15.98:0/1671659856 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c014870 con 0x7f103415b9e0 2024-05-09T10:51:00.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.554+0000 7f103a1f9640 1 -- 172.21.15.98:0/1671659856 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10340b3ba0 con 0x7f103415b9e0 2024-05-09T10:51:00.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.554+0000 7f103a1f9640 1 -- 172.21.15.98:0/1671659856 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10340b4160 con 0x7f103415b9e0 2024-05-09T10:51:00.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.555+0000 7f103a1f9640 1 -- 172.21.15.98:0/1671659856 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1034069cb0 con 0x7f103415b9e0 2024-05-09T10:51:00.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.556+0000 7f101bfff640 1 -- 172.21.15.98:0/1671659856 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f102c005d10 con 0x7f103415b9e0 2024-05-09T10:51:00.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.556+0000 7f101bfff640 1 --2- 172.21.15.98:0/1671659856 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f102006b4d0 0x7f102006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:00.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.556+0000 7f101bfff640 1 -- 172.21.15.98:0/1671659856 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f102c08ab40 con 0x7f103415b9e0 2024-05-09T10:51:00.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.556+0000 7f10337fe640 1 --2- 172.21.15.98:0/1671659856 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f102006b4d0 0x7f102006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:00.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.556+0000 7f10337fe640 1 --2- 172.21.15.98:0/1671659856 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f102006b4d0 0x7f102006d990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1024009ed0 tx=0x7f1024009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:00.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.560+0000 7f101bfff640 1 -- 172.21.15.98:0/1671659856 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f102c022030 con 0x7f103415b9e0 2024-05-09T10:51:00.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:00.743+0000 7f103a1f9640 1 -- 172.21.15.98:0/1671659856 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1034152140 con 0x7f103415b9e0 2024-05-09T10:51:01.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.369+0000 7f101bfff640 1 -- 172.21.15.98:0/1671659856 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v25) v1 ==== 97+0+0 (secure 0 0 0) 0x7f102c010040 con 0x7f103415b9e0 2024-05-09T10:51:01.362 INFO:teuthology.orchestra.run.smithi098.stderr: removed all blocklist entries 2024-05-09T10:51:01.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.371+0000 7f103a1f9640 1 -- 172.21.15.98:0/1671659856 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f102006b4d0 msgr2=0x7f102006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:01.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.371+0000 7f103a1f9640 1 --2- 172.21.15.98:0/1671659856 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f102006b4d0 0x7f102006d990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1024009ed0 tx=0x7f1024009190 comp rx=0 tx=0).stop 2024-05-09T10:51:01.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.371+0000 7f103a1f9640 1 -- 172.21.15.98:0/1671659856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f103415b9e0 msgr2=0x7f103415c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:01.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.371+0000 7f103a1f9640 1 --2- 172.21.15.98:0/1671659856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f103415b9e0 0x7f103415c160 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f102c009d90 tx=0x7f102c00c490 comp rx=0 tx=0).stop 2024-05-09T10:51:01.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.371+0000 7f103a1f9640 1 -- 172.21.15.98:0/1671659856 shutdown_connections 2024-05-09T10:51:01.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.371+0000 7f103a1f9640 1 --2- 172.21.15.98:0/1671659856 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f102006b4d0 0x7f102006d990 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:01.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.371+0000 7f103a1f9640 1 --2- 172.21.15.98:0/1671659856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f103415b9e0 0x7f103415c160 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:01.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.371+0000 7f103a1f9640 1 --2- 172.21.15.98:0/1671659856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10340af6e0 0x7f103415e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:01.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.371+0000 7f103a1f9640 1 --2- 172.21.15.98:0/1671659856 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f103406b120 0x7f103414b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:01.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.371+0000 7f103a1f9640 1 -- 172.21.15.98:0/1671659856 >> 172.21.15.98:0/1671659856 conn(0x7f10340b24c0 msgr2=0x7f103415f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:01.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.371+0000 7f103a1f9640 1 -- 172.21.15.98:0/1671659856 shutdown_connections 2024-05-09T10:51:01.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.371+0000 7f103a1f9640 1 -- 172.21.15.98:0/1671659856 wait complete. 2024-05-09T10:51:01.377 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-09T10:51:01.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.485+0000 7fe2a8122640 1 Processor -- start 2024-05-09T10:51:01.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.486+0000 7fe2a8122640 1 -- start start 2024-05-09T10:51:01.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.487+0000 7fe2a8122640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe2a015df70 0x7fe2a00713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:01.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.487+0000 7fe2a8122640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe2a0071910 0x7fe2a006abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:01.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.487+0000 7fe2a8122640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2a00cdb30 con 0x7fe2a015db90 2024-05-09T10:51:01.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.487+0000 7fe2a8122640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2a0071f40 con 0x7fe2a0071910 2024-05-09T10:51:01.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.487+0000 7fe2a8122640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2a0072220 con 0x7fe2a015df70 2024-05-09T10:51:01.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.487+0000 7fe2a66bf640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe2a0071910 0x7fe2a006abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:01.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.487+0000 7fe2a5ebe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe2a015db90 0x7fe2a006d430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38912/0 (socket says 172.21.15.98:38912) 2024-05-09T10:51:01.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.487+0000 7fe2a5ebe640 1 -- 172.21.15.98:0/846142189 learned_addr learned my addr 172.21.15.98:0/846142189 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:01.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.487+0000 7fe2a56bd640 1 --2- 172.21.15.98:0/846142189 >> v2:172.21.15.105:3300/0 conn(0x7fe2a015df70 0x7fe2a00713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:01.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.487+0000 7fe2a66bf640 1 -- 172.21.15.98:0/846142189 >> v2:172.21.15.105:3300/0 conn(0x7fe2a015df70 msgr2=0x7fe2a00713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:01.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.487+0000 7fe2a66bf640 1 --2- 172.21.15.98:0/846142189 >> v2:172.21.15.105:3300/0 conn(0x7fe2a015df70 0x7fe2a00713d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:01.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.487+0000 7fe2a66bf640 1 -- 172.21.15.98:0/846142189 >> v1:172.21.15.98:6789/0 conn(0x7fe2a015db90 legacy=0x7fe2a006d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:01.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.487+0000 7fe2a66bf640 1 -- 172.21.15.98:0/846142189 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a0072500 con 0x7fe2a0071910 2024-05-09T10:51:01.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.487+0000 7fe2a4ebc640 1 -- 172.21.15.98:0/846142189 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 195826500 0 0) 0x7fe298001110 con 0x7fe2a015db90 2024-05-09T10:51:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.488+0000 7fe2a66bf640 1 --2- 172.21.15.98:0/846142189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2a0071910 0x7fe2a006abe0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00b5e40 tx=0x7fe29002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b45bfd1f3560eb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.488+0000 7fe2a4ebc640 1 -- 172.21.15.98:0/846142189 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe290007740 con 0x7fe2a0071910 2024-05-09T10:51:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.488+0000 7fe2a4ebc640 1 -- 172.21.15.98:0/846142189 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2900078e0 con 0x7fe2a0071910 2024-05-09T10:51:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.488+0000 7fe2a4ebc640 1 -- 172.21.15.98:0/846142189 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe290007cb0 con 0x7fe2a0071910 2024-05-09T10:51:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.488+0000 7fe2a8122640 1 -- 172.21.15.98:0/846142189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2a0071910 msgr2=0x7fe2a006abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.488+0000 7fe2a8122640 1 --2- 172.21.15.98:0/846142189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2a0071910 0x7fe2a006abe0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00b5e40 tx=0x7fe29002a5f0 comp rx=0 tx=0).stop 2024-05-09T10:51:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.488+0000 7fe2a8122640 1 -- 172.21.15.98:0/846142189 shutdown_connections 2024-05-09T10:51:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.488+0000 7fe2a8122640 1 --2- 172.21.15.98:0/846142189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2a0071910 0x7fe2a006abe0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.488+0000 7fe2a8122640 1 --2- 172.21.15.98:0/846142189 >> v2:172.21.15.105:3300/0 conn(0x7fe2a015df70 0x7fe2a00713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:01.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.488+0000 7fe2a8122640 1 -- 172.21.15.98:0/846142189 >> 172.21.15.98:0/846142189 conn(0x7fe2a0066f20 msgr2=0x7fe2a006bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:01.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.488+0000 7fe2a8122640 1 -- 172.21.15.98:0/846142189 shutdown_connections 2024-05-09T10:51:01.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.488+0000 7fe2a8122640 1 -- 172.21.15.98:0/846142189 wait complete. 2024-05-09T10:51:01.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.490+0000 7fe2a8122640 1 Processor -- start 2024-05-09T10:51:01.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.490+0000 7fe2a8122640 1 -- start start 2024-05-09T10:51:01.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.490+0000 7fe2a8122640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2a0071910 0x7fe2a00b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:01.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.490+0000 7fe2a8122640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2a015db90 0x7fe2a00cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:01.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.490+0000 7fe2a8122640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2a015df70 0x7fe2a00b4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:01.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.490+0000 7fe2a8122640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a006b120 con 0x7fe2a015df70 2024-05-09T10:51:01.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.490+0000 7fe2a8122640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a00cefd0 con 0x7fe2a0071910 2024-05-09T10:51:01.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.490+0000 7fe2a8122640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a006b2a0 con 0x7fe2a015db90 2024-05-09T10:51:01.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.490+0000 7fe2a66bf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2a015df70 0x7fe2a00b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:01.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.490+0000 7fe2a66bf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2a015df70 0x7fe2a00b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35710/0 (socket says 172.21.15.98:35710) 2024-05-09T10:51:01.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.491+0000 7fe2a66bf640 1 -- 172.21.15.98:0/3358941650 learned_addr learned my addr 172.21.15.98:0/3358941650 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:01.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.491+0000 7fe2a5ebe640 1 --2- 172.21.15.98:0/3358941650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2a0071910 0x7fe2a00b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:01.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.491+0000 7fe2a56bd640 1 --2- 172.21.15.98:0/3358941650 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2a015db90 0x7fe2a00cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:01.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.491+0000 7fe2a66bf640 1 -- 172.21.15.98:0/3358941650 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2a015db90 msgr2=0x7fe2a00cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:01.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.491+0000 7fe2a66bf640 1 --2- 172.21.15.98:0/3358941650 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2a015db90 0x7fe2a00cf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:01.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.491+0000 7fe2a66bf640 1 -- 172.21.15.98:0/3358941650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2a0071910 msgr2=0x7fe2a00b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:01.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.491+0000 7fe2a66bf640 1 --2- 172.21.15.98:0/3358941650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2a0071910 0x7fe2a00b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:01.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.491+0000 7fe2a66bf640 1 -- 172.21.15.98:0/3358941650 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a00b26f0 con 0x7fe2a015df70 2024-05-09T10:51:01.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.491+0000 7fe2a5ebe640 1 --2- 172.21.15.98:0/3358941650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2a0071910 0x7fe2a00b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:01.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.491+0000 7fe2a56bd640 1 --2- 172.21.15.98:0/3358941650 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2a015db90 0x7fe2a00cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:01.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.491+0000 7fe2a66bf640 1 --2- 172.21.15.98:0/3358941650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2a015df70 0x7fe2a00b4330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe29002abe0 tx=0x7fe290004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:01.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.491+0000 7fe29e7fc640 1 -- 172.21.15.98:0/3358941650 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe290040070 con 0x7fe2a015df70 2024-05-09T10:51:01.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.491+0000 7fe29e7fc640 1 -- 172.21.15.98:0/3358941650 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe290031bc0 con 0x7fe2a015df70 2024-05-09T10:51:01.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.491+0000 7fe2a8122640 1 -- 172.21.15.98:0/3358941650 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a00b29d0 con 0x7fe2a015df70 2024-05-09T10:51:01.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.491+0000 7fe29e7fc640 1 -- 172.21.15.98:0/3358941650 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe290045460 con 0x7fe2a015df70 2024-05-09T10:51:01.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.491+0000 7fe2a8122640 1 -- 172.21.15.98:0/3358941650 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a00b2f90 con 0x7fe2a015df70 2024-05-09T10:51:01.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.492+0000 7fe2a8122640 1 -- 172.21.15.98:0/3358941650 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2a00b3480 con 0x7fe2a015df70 2024-05-09T10:51:01.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.493+0000 7fe29e7fc640 1 -- 172.21.15.98:0/3358941650 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe290039040 con 0x7fe2a015df70 2024-05-09T10:51:01.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.493+0000 7fe29e7fc640 1 --2- 172.21.15.98:0/3358941650 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe28c06b4d0 0x7fe28c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:01.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.493+0000 7fe29e7fc640 1 -- 172.21.15.98:0/3358941650 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe2900ab950 con 0x7fe2a015df70 2024-05-09T10:51:01.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.493+0000 7fe2a5ebe640 1 --2- 172.21.15.98:0/3358941650 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe28c06b4d0 0x7fe28c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:01.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.494+0000 7fe2a5ebe640 1 --2- 172.21.15.98:0/3358941650 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe28c06b4d0 0x7fe28c06d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe298000f60 tx=0x7fe298007a10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:01.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.497+0000 7fe29e7fc640 1 -- 172.21.15.98:0/3358941650 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe29007ab20 con 0x7fe2a015df70 2024-05-09T10:51:01.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.734+0000 7fe2a8122640 1 -- 172.21.15.98:0/3358941650 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe2a0069cb0 con 0x7fe2a015df70 2024-05-09T10:51:01.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.735+0000 7fe29e7fc640 1 -- 172.21.15.98:0/3358941650 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fe29007a420 con 0x7fe2a015df70 2024-05-09T10:51:01.728 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:01.728 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-05-09T10:51:01.728 INFO:teuthology.orchestra.run.smithi098.stdout: "auth_dump": [ 2024-05-09T10:51:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.a", 2024-05-09T10:51:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAxqjxmKekgIhAAWaiiDMySfjchssEwwBC7OQ==", 2024-05-09T10:51:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-05-09T10:51:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-05-09T10:51:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-05-09T10:51:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-05-09T10:51:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.b", 2024-05-09T10:51:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAxqjxmesW1LRAA4vCFcVQ7pP4hB32YyDeMKw==", 2024-05-09T10:51:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-05-09T10:51:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-05-09T10:51:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-05-09T10:51:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-05-09T10:51:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.c", 2024-05-09T10:51:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAxqjxmM7wWKRAA/BXGwXIJscgJYJjaSg+BqA==", 2024-05-09T10:51:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-05-09T10:51:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-05-09T10:51:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-05-09T10:51:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-05-09T10:51:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.d", 2024-05-09T10:51:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAxqjxmRHGyNBAAvcEknZ4L7zuRE3KOo8fpnA==", 2024-05-09T10:51:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-05-09T10:51:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-05-09T10:51:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-05-09T10:51:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-05-09T10:51:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.0", 2024-05-09T10:51:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQA1qjxmUPtjGRAAfaVYD/ym9Ih1QbbgDvDJWg==", 2024-05-09T10:51:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-05-09T10:51:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-05-09T10:51:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-05-09T10:51:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.1", 2024-05-09T10:51:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQA4qjxmbM9hFxAACR1PQvRZOQ4nRXzL+3FGog==", 2024-05-09T10:51:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-05-09T10:51:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-05-09T10:51:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-05-09T10:51:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.2", 2024-05-09T10:51:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQA7qjxm/Q93FRAAKgbFv84TwLs6cwe91IN0Rg==", 2024-05-09T10:51:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-05-09T10:51:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-05-09T10:51:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-05-09T10:51:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.3", 2024-05-09T10:51:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQA+qjxmACv9ExAACD1053vaJzS13heB2SrByQ==", 2024-05-09T10:51:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-05-09T10:51:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-05-09T10:51:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-05-09T10:51:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.4", 2024-05-09T10:51:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQBEqjxmaLBEHBAAfQJfAs+6TcDM1KOfzdIlKA==", 2024-05-09T10:51:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-05-09T10:51:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-05-09T10:51:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-05-09T10:51:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.5", 2024-05-09T10:51:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQBHqjxmAE0BGxAA0LIjdnt3rsNCXKHtr4400Q==", 2024-05-09T10:51:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-05-09T10:51:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-05-09T10:51:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-05-09T10:51:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.6", 2024-05-09T10:51:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQBKqjxmNjsRGRAAsX+aMYgavwhrMhLBL6bMFw==", 2024-05-09T10:51:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-05-09T10:51:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-05-09T10:51:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-05-09T10:51:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.7", 2024-05-09T10:51:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQBNqjxmc+MjFxAAtI4Tx9jbZgy8hHOAFbJSAA==", 2024-05-09T10:51:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-05-09T10:51:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-05-09T10:51:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-05-09T10:51:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.0", 2024-05-09T10:51:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAxqjxmT+bPOBAA4L/A/EK0lI7oydvPJ27lMA==", 2024-05-09T10:51:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-05-09T10:51:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-05-09T10:51:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-05-09T10:51:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-05-09T10:51:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.1", 2024-05-09T10:51:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAyqjxmhz/zBRAAH8iWmULG5e1d7ZuQoCtIZg==", 2024-05-09T10:51:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-05-09T10:51:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-05-09T10:51:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-05-09T10:51:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-05-09T10:51:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.2", 2024-05-09T10:51:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAyqjxmbyLKAhAAQpXTBuUvHOibVF0c9c0pOw==", 2024-05-09T10:51:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-05-09T10:51:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-05-09T10:51:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-05-09T10:51:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-05-09T10:51:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.3", 2024-05-09T10:51:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAyqjxmq0h4CxAAAaZzvfbsvhfYfs01IBWbKw==", 2024-05-09T10:51:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-05-09T10:51:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-05-09T10:51:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-05-09T10:51:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-05-09T10:51:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.admin", 2024-05-09T10:51:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAxqjxmntbxBhAAhWTWTOmF3S1Qz3q18s2NPA==", 2024-05-09T10:51:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-05-09T10:51:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-05-09T10:51:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-05-09T10:51:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-05-09T10:51:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-mds", 2024-05-09T10:51:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQBTqjxmB68ZCxAAtD9kshb7ImV39ONbDWLWUQ==", 2024-05-09T10:51:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-mds" 2024-05-09T10:51:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-mgr", 2024-05-09T10:51:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQBTqjxmUrcZCxAAJ7uTnwQQ8Z5/4w7zZquLbg==", 2024-05-09T10:51:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-09T10:51:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-osd", 2024-05-09T10:51:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQBTqjxmAL4ZCxAAw18dwpKPezNfglQrjaU4sg==", 2024-05-09T10:51:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-osd" 2024-05-09T10:51:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-rbd", 2024-05-09T10:51:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQBTqjxmjsQZCxAA655NtGNvH4YV5yyIgCV7tA==", 2024-05-09T10:51:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-09T10:51:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-09T10:51:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQBTqjxmHssZCxAA2LopiXbMryeFNO5SMXRMyA==", 2024-05-09T10:51:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-09T10:51:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-rgw", 2024-05-09T10:51:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQBTqjxmA9IZCxAAch3bBYJRFyNP9dDZFQ+glg==", 2024-05-09T10:51:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-09T10:51:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.x", 2024-05-09T10:51:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAxqjxmDF77HhAAzZTxVNfOjYpxuVCXDJ/Xzg==", 2024-05-09T10:51:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-05-09T10:51:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-05-09T10:51:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-05-09T10:51:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.y", 2024-05-09T10:51:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQAxqjxmiTuEGxAAc7n/z9PkjoC3HrAV8BKplw==", 2024-05-09T10:51:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-05-09T10:51:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-05-09T10:51:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-05-09T10:51:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-05-09T10:51:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-05-09T10:51:01.743 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-05-09T10:51:01.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.737+0000 7fe2a8122640 1 -- 172.21.15.98:0/3358941650 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe28c06b4d0 msgr2=0x7fe28c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:01.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.737+0000 7fe2a8122640 1 --2- 172.21.15.98:0/3358941650 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe28c06b4d0 0x7fe28c06d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe298000f60 tx=0x7fe298007a10 comp rx=0 tx=0).stop 2024-05-09T10:51:01.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.737+0000 7fe2a8122640 1 -- 172.21.15.98:0/3358941650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2a015df70 msgr2=0x7fe2a00b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:01.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.737+0000 7fe2a8122640 1 --2- 172.21.15.98:0/3358941650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2a015df70 0x7fe2a00b4330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe29002abe0 tx=0x7fe290004a40 comp rx=0 tx=0).stop 2024-05-09T10:51:01.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.737+0000 7fe2a8122640 1 -- 172.21.15.98:0/3358941650 shutdown_connections 2024-05-09T10:51:01.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.737+0000 7fe2a8122640 1 --2- 172.21.15.98:0/3358941650 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe28c06b4d0 0x7fe28c06d990 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:01.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.737+0000 7fe2a8122640 1 --2- 172.21.15.98:0/3358941650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe2a015df70 0x7fe2a00b4330 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:01.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.737+0000 7fe2a8122640 1 --2- 172.21.15.98:0/3358941650 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2a015db90 0x7fe2a00cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:01.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.737+0000 7fe2a8122640 1 --2- 172.21.15.98:0/3358941650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2a0071910 0x7fe2a00b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:01.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.737+0000 7fe2a8122640 1 -- 172.21.15.98:0/3358941650 >> 172.21.15.98:0/3358941650 conn(0x7fe2a0066f20 msgr2=0x7fe2a00ccf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:01.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.737+0000 7fe2a8122640 1 -- 172.21.15.98:0/3358941650 shutdown_connections 2024-05-09T10:51:01.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.737+0000 7fe2a8122640 1 -- 172.21.15.98:0/3358941650 wait complete. 2024-05-09T10:51:01.746 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-09T10:51:01.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.853+0000 7f386d63f640 1 Processor -- start 2024-05-09T10:51:01.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.853+0000 7f386d63f640 1 -- start start 2024-05-09T10:51:01.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.854+0000 7f386d63f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f386815de40 0x7f38680d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:01.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.854+0000 7f386d63f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f386815e220 con 0x7f38680cf250 2024-05-09T10:51:01.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.854+0000 7f386d63f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38680d19e0 con 0x7f38680d0b80 2024-05-09T10:51:01.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.854+0000 7f386d63f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38680d1790 con 0x7f386815de40 2024-05-09T10:51:01.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.854+0000 7f385e7fc640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f38680cf250 0x7f38680d0470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38920/0 (socket says 172.21.15.98:38920) 2024-05-09T10:51:01.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.854+0000 7f385e7fc640 1 -- 172.21.15.98:0/3598798717 learned_addr learned my addr 172.21.15.98:0/3598798717 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:01.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.854+0000 7f38667fc640 1 -- 172.21.15.98:0/3598798717 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126622114 0 0) 0x7f386815e220 con 0x7f38680cf250 2024-05-09T10:51:01.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.854+0000 7f38667fc640 1 -- 172.21.15.98:0/3598798717 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3854003520 con 0x7f38680cf250 2024-05-09T10:51:01.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.854+0000 7f3866ffd640 1 --2- 172.21.15.98:0/3598798717 >> v2:172.21.15.105:3300/0 conn(0x7f386815de40 0x7f38680d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:01.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.855+0000 7f38667fc640 1 -- 172.21.15.98:0/3598798717 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 891537118 0 0) 0x7f3854003520 con 0x7f38680cf250 2024-05-09T10:51:01.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.855+0000 7f38667fc640 1 -- 172.21.15.98:0/3598798717 >> v2:172.21.15.105:3300/0 conn(0x7f386815de40 msgr2=0x7f38680d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:01.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.855+0000 7f38667fc640 1 --2- 172.21.15.98:0/3598798717 >> v2:172.21.15.105:3300/0 conn(0x7f386815de40 0x7f38680d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:01.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.855+0000 7f38667fc640 1 -- 172.21.15.98:0/3598798717 >> v1:172.21.15.105:6789/0 conn(0x7f38680d0b80 legacy=0x7f38680b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:01.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.855+0000 7f38667fc640 1 -- 172.21.15.98:0/3598798717 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38680d1cc0 con 0x7f38680cf250 2024-05-09T10:51:01.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.855+0000 7f38667fc640 1 -- 172.21.15.98:0/3598798717 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f3850001670 con 0x7f38680cf250 2024-05-09T10:51:01.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.855+0000 7f38667fc640 1 -- 172.21.15.98:0/3598798717 >> v1:172.21.15.98:6789/0 conn(0x7f38680cf250 legacy=0x7f38680d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:01.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.855+0000 7f38667fc640 1 --2- 172.21.15.98:0/3598798717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38540039b0 0x7f3854003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:01.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.855+0000 7f38667fc640 1 --2- 172.21.15.98:0/3598798717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3854004800 0x7f3854004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:01.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.855+0000 7f38667fc640 1 --2- 172.21.15.98:0/3598798717 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38540051c0 0x7f385400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:01.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.855+0000 7f38667fc640 1 -- 172.21.15.98:0/3598798717 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38680d1790 con 0x7f38540039b0 2024-05-09T10:51:01.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.855+0000 7f38667fc640 1 -- 172.21.15.98:0/3598798717 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385400c060 con 0x7f3854004800 2024-05-09T10:51:01.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.855+0000 7f38667fc640 1 -- 172.21.15.98:0/3598798717 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385400c260 con 0x7f38540051c0 2024-05-09T10:51:01.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.855+0000 7f3866ffd640 1 --2- 172.21.15.98:0/3598798717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38540039b0 0x7f3854003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:01.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.855+0000 7f385e7fc640 1 --2- 172.21.15.98:0/3598798717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3854004800 0x7f3854004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:01.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.855+0000 7f38677fe640 1 --2- 172.21.15.98:0/3598798717 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38540051c0 0x7f385400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:01.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.855+0000 7f3866ffd640 1 -- 172.21.15.98:0/3598798717 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38540051c0 msgr2=0x7f385400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:01.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.855+0000 7f3866ffd640 1 --2- 172.21.15.98:0/3598798717 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38540051c0 0x7f385400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:01.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.855+0000 7f3866ffd640 1 -- 172.21.15.98:0/3598798717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3854004800 msgr2=0x7f3854004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:01.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.855+0000 7f3866ffd640 1 --2- 172.21.15.98:0/3598798717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3854004800 0x7f3854004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:01.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.855+0000 7f3866ffd640 1 -- 172.21.15.98:0/3598798717 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3854003520 con 0x7f38540039b0 2024-05-09T10:51:01.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.855+0000 7f38677fe640 1 -- 172.21.15.98:0/3598798717 reap_dead start 2024-05-09T10:51:01.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.856+0000 7f3866ffd640 1 --2- 172.21.15.98:0/3598798717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38540039b0 0x7f3854003dd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3858014f00 tx=0x7f3854005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25debf21ee5ce1a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:01.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.856+0000 7f38667fc640 1 -- 172.21.15.98:0/3598798717 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385803b070 con 0x7f38540039b0 2024-05-09T10:51:01.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.856+0000 7f38667fc640 1 -- 172.21.15.98:0/3598798717 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3858037670 con 0x7f38540039b0 2024-05-09T10:51:01.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.856+0000 7f38667fc640 1 -- 172.21.15.98:0/3598798717 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3858014b50 con 0x7f38540039b0 2024-05-09T10:51:01.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.856+0000 7f386d63f640 1 -- 172.21.15.98:0/3598798717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38540039b0 msgr2=0x7f3854003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:01.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.856+0000 7f386d63f640 1 --2- 172.21.15.98:0/3598798717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38540039b0 0x7f3854003dd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3858014f00 tx=0x7f3854005830 comp rx=0 tx=0).stop 2024-05-09T10:51:01.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.856+0000 7f386d63f640 1 -- 172.21.15.98:0/3598798717 shutdown_connections 2024-05-09T10:51:01.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.856+0000 7f386d63f640 1 --2- 172.21.15.98:0/3598798717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38540039b0 0x7f3854003dd0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:01.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.856+0000 7f386d63f640 1 -- 172.21.15.98:0/3598798717 >> 172.21.15.98:0/3598798717 conn(0x7f386809ce90 msgr2=0x7f38680d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:01.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.857+0000 7f386d63f640 1 -- 172.21.15.98:0/3598798717 shutdown_connections 2024-05-09T10:51:01.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.857+0000 7f386d63f640 1 -- 172.21.15.98:0/3598798717 wait complete. 2024-05-09T10:51:01.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.858+0000 7f386d63f640 1 Processor -- start 2024-05-09T10:51:01.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.858+0000 7f386d63f640 1 -- start start 2024-05-09T10:51:01.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.858+0000 7f386d63f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38540039b0 0x7f3868156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:01.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.858+0000 7f386d63f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f386815e430 0x7f3868157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:01.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.859+0000 7f386d63f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f386815e8b0 0x7f38680c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:01.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.859+0000 7f386d63f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38680d4680 con 0x7f386815e8b0 2024-05-09T10:51:01.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.859+0000 7f386d63f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38680d2280 con 0x7f386815e430 2024-05-09T10:51:01.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.859+0000 7f386d63f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38680d4800 con 0x7f38540039b0 2024-05-09T10:51:01.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.859+0000 7f38677fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f386815e8b0 0x7f38680c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:01.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.859+0000 7f38677fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f386815e8b0 0x7f38680c7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35734/0 (socket says 172.21.15.98:35734) 2024-05-09T10:51:01.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.859+0000 7f38677fe640 1 -- 172.21.15.98:0/3994850618 learned_addr learned my addr 172.21.15.98:0/3994850618 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:01.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.859+0000 7f385e7fc640 1 --2- 172.21.15.98:0/3994850618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f386815e430 0x7f3868157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:01.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.859+0000 7f3866ffd640 1 --2- 172.21.15.98:0/3994850618 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38540039b0 0x7f3868156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:01.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.859+0000 7f38677fe640 1 -- 172.21.15.98:0/3994850618 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38540039b0 msgr2=0x7f3868156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:01.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.859+0000 7f38677fe640 1 --2- 172.21.15.98:0/3994850618 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38540039b0 0x7f3868156560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:01.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.859+0000 7f38677fe640 1 -- 172.21.15.98:0/3994850618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f386815e430 msgr2=0x7f3868157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:01.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.859+0000 7f38677fe640 1 --2- 172.21.15.98:0/3994850618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f386815e430 0x7f3868157eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:01.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.859+0000 7f38677fe640 1 -- 172.21.15.98:0/3994850618 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38680cb3b0 con 0x7f386815e8b0 2024-05-09T10:51:01.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.859+0000 7f385e7fc640 1 --2- 172.21.15.98:0/3994850618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f386815e430 0x7f3868157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:01.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.859+0000 7f3866ffd640 1 --2- 172.21.15.98:0/3994850618 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38540039b0 0x7f3868156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:01.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.859+0000 7f38677fe640 1 --2- 172.21.15.98:0/3994850618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f386815e8b0 0x7f38680c7b40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f386806d960 tx=0x7f386000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:01.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.859+0000 7f385ffff640 1 -- 172.21.15.98:0/3994850618 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3860003970 con 0x7f386815e8b0 2024-05-09T10:51:01.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.859+0000 7f385ffff640 1 -- 172.21.15.98:0/3994850618 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3860018070 con 0x7f386815e8b0 2024-05-09T10:51:01.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.859+0000 7f385ffff640 1 -- 172.21.15.98:0/3994850618 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3860003d70 con 0x7f386815e8b0 2024-05-09T10:51:01.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.859+0000 7f386d63f640 1 -- 172.21.15.98:0/3994850618 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38680cb690 con 0x7f386815e8b0 2024-05-09T10:51:01.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.859+0000 7f386d63f640 1 -- 172.21.15.98:0/3994850618 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38680cbc50 con 0x7f386815e8b0 2024-05-09T10:51:01.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.860+0000 7f386d63f640 1 -- 172.21.15.98:0/3994850618 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3868069cb0 con 0x7f386815e8b0 2024-05-09T10:51:01.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.861+0000 7f385ffff640 1 -- 172.21.15.98:0/3994850618 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f38600050d0 con 0x7f386815e8b0 2024-05-09T10:51:01.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.861+0000 7f385ffff640 1 --2- 172.21.15.98:0/3994850618 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f384806b4d0 0x7f384806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:01.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.861+0000 7f385ffff640 1 -- 172.21.15.98:0/3994850618 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3860089ba0 con 0x7f386815e8b0 2024-05-09T10:51:01.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.861+0000 7f3866ffd640 1 --2- 172.21.15.98:0/3994850618 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f384806b4d0 0x7f384806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:01.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.862+0000 7f3866ffd640 1 --2- 172.21.15.98:0/3994850618 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f384806b4d0 0x7f384806d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f385803d000 tx=0x7f3858008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:01.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:01.865+0000 7f385ffff640 1 -- 172.21.15.98:0/3994850618 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3860058cd0 con 0x7f386815e8b0 2024-05-09T10:51:02.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.099+0000 7f386d63f640 1 -- 172.21.15.98:0/3994850618 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f386815d660 con 0x7f386815e8b0 2024-05-09T10:51:02.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.101+0000 7f385ffff640 1 -- 172.21.15.98:0/3994850618 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7f38600585d0 con 0x7f386815e8b0 2024-05-09T10:51:02.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.102+0000 7f386d63f640 1 -- 172.21.15.98:0/3994850618 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f384806b4d0 msgr2=0x7f384806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:02.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.102+0000 7f386d63f640 1 --2- 172.21.15.98:0/3994850618 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f384806b4d0 0x7f384806d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f385803d000 tx=0x7f3858008030 comp rx=0 tx=0).stop 2024-05-09T10:51:02.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.102+0000 7f386d63f640 1 -- 172.21.15.98:0/3994850618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f386815e8b0 msgr2=0x7f38680c7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:02.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.102+0000 7f386d63f640 1 --2- 172.21.15.98:0/3994850618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f386815e8b0 0x7f38680c7b40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f386806d960 tx=0x7f386000ec10 comp rx=0 tx=0).stop 2024-05-09T10:51:02.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.103+0000 7f386d63f640 1 -- 172.21.15.98:0/3994850618 shutdown_connections 2024-05-09T10:51:02.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.103+0000 7f386d63f640 1 --2- 172.21.15.98:0/3994850618 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f384806b4d0 0x7f384806d990 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.103+0000 7f386d63f640 1 --2- 172.21.15.98:0/3994850618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f386815e8b0 0x7f38680c7b40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.103+0000 7f386d63f640 1 --2- 172.21.15.98:0/3994850618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f386815e430 0x7f3868157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.103+0000 7f386d63f640 1 --2- 172.21.15.98:0/3994850618 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f38540039b0 0x7f3868156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.103+0000 7f386d63f640 1 -- 172.21.15.98:0/3994850618 >> 172.21.15.98:0/3994850618 conn(0x7f386809ce90 msgr2=0x7f38680cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:02.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.103+0000 7f386d63f640 1 -- 172.21.15.98:0/3994850618 shutdown_connections 2024-05-09T10:51:02.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.103+0000 7f386d63f640 1 -- 172.21.15.98:0/3994850618 wait complete. 2024-05-09T10:51:02.107 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-09T10:51:02.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.215+0000 7f0230676640 1 Processor -- start 2024-05-09T10:51:02.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.215+0000 7f0230676640 1 -- start start 2024-05-09T10:51:02.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.216+0000 7f0230676640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f02280ceb70 0x7f02280b4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:02.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.216+0000 7f0230676640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f02280b4d40 0x7f02280cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:02.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.216+0000 7f0230676640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02280cc4c0 con 0x7f022815bdc0 2024-05-09T10:51:02.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.216+0000 7f0230676640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f022815c1a0 con 0x7f02280ceb70 2024-05-09T10:51:02.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.216+0000 7f0230676640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02280cf3c0 con 0x7f02280b4d40 2024-05-09T10:51:02.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.216+0000 7f022dc11640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f02280ceb70 0x7f02280b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:02.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.216+0000 7f022e412640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f022815bdc0 0x7f02280cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38928/0 (socket says 172.21.15.98:38928) 2024-05-09T10:51:02.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.216+0000 7f022e412640 1 -- 172.21.15.98:0/908628132 learned_addr learned my addr 172.21.15.98:0/908628132 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:02.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.216+0000 7f022dc11640 1 -- 172.21.15.98:0/908628132 >> v2:172.21.15.105:3300/0 conn(0x7f02280b4d40 msgr2=0x7f02280cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:02.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.216+0000 7f022ec13640 1 --2- 172.21.15.98:0/908628132 >> v2:172.21.15.105:3300/0 conn(0x7f02280b4d40 0x7f02280cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:02.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.216+0000 7f022d410640 1 -- 172.21.15.98:0/908628132 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 61732405 0 0) 0x7f02280cc4c0 con 0x7f022815bdc0 2024-05-09T10:51:02.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.216+0000 7f022dc11640 1 --2- 172.21.15.98:0/908628132 >> v2:172.21.15.105:3300/0 conn(0x7f02280b4d40 0x7f02280cfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.216+0000 7f022dc11640 1 -- 172.21.15.98:0/908628132 >> v1:172.21.15.98:6789/0 conn(0x7f022815bdc0 legacy=0x7f02280cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:02.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.216+0000 7f022dc11640 1 -- 172.21.15.98:0/908628132 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02280cf0e0 con 0x7f02280ceb70 2024-05-09T10:51:02.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.216+0000 7f022ec13640 1 --2- 172.21.15.98:0/908628132 >> v2:172.21.15.105:3300/0 conn(0x7f02280b4d40 0x7f02280cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:02.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.217+0000 7f022dc11640 1 --2- 172.21.15.98:0/908628132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02280ceb70 0x7f02280b4800 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0228071a20 tx=0x7f021802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9edc305f1869489a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:02.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.217+0000 7f022d410640 1 -- 172.21.15.98:0/908628132 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0218037070 con 0x7f02280ceb70 2024-05-09T10:51:02.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.217+0000 7f022d410640 1 -- 172.21.15.98:0/908628132 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0218032440 con 0x7f02280ceb70 2024-05-09T10:51:02.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.217+0000 7f022d410640 1 -- 172.21.15.98:0/908628132 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0218032810 con 0x7f02280ceb70 2024-05-09T10:51:02.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.218+0000 7f0230676640 1 -- 172.21.15.98:0/908628132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02280ceb70 msgr2=0x7f02280b4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:02.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.218+0000 7f0230676640 1 --2- 172.21.15.98:0/908628132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02280ceb70 0x7f02280b4800 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0228071a20 tx=0x7f021802a1a0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.218+0000 7f0230676640 1 -- 172.21.15.98:0/908628132 shutdown_connections 2024-05-09T10:51:02.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.218+0000 7f0230676640 1 --2- 172.21.15.98:0/908628132 >> v2:172.21.15.105:3300/0 conn(0x7f02280b4d40 0x7f02280cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.218+0000 7f0230676640 1 --2- 172.21.15.98:0/908628132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02280ceb70 0x7f02280b4800 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.218+0000 7f0230676640 1 -- 172.21.15.98:0/908628132 >> 172.21.15.98:0/908628132 conn(0x7f02280b34e0 msgr2=0x7f02280b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:02.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.218+0000 7f0230676640 1 -- 172.21.15.98:0/908628132 shutdown_connections 2024-05-09T10:51:02.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.218+0000 7f0230676640 1 -- 172.21.15.98:0/908628132 wait complete. 2024-05-09T10:51:02.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.219+0000 7f0230676640 1 Processor -- start 2024-05-09T10:51:02.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.219+0000 7f0230676640 1 -- start start 2024-05-09T10:51:02.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.220+0000 7f0230676640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02280b4d40 0x7f022809cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:02.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.220+0000 7f0230676640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02280ceb70 0x7f022814cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:02.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.220+0000 7f0230676640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f022815bdc0 0x7f02280c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:02.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.220+0000 7f0230676640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02280ccd20 con 0x7f022815bdc0 2024-05-09T10:51:02.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.220+0000 7f0230676640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02280cca40 con 0x7f02280ceb70 2024-05-09T10:51:02.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.220+0000 7f0230676640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02280b5190 con 0x7f02280b4d40 2024-05-09T10:51:02.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.220+0000 7f022ec13640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f022815bdc0 0x7f02280c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:02.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.220+0000 7f022ec13640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f022815bdc0 0x7f02280c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35748/0 (socket says 172.21.15.98:35748) 2024-05-09T10:51:02.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.220+0000 7f022ec13640 1 -- 172.21.15.98:0/3462735768 learned_addr learned my addr 172.21.15.98:0/3462735768 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:02.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.220+0000 7f022e412640 1 --2- 172.21.15.98:0/3462735768 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02280b4d40 0x7f022809cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:02.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.220+0000 7f022dc11640 1 --2- 172.21.15.98:0/3462735768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02280ceb70 0x7f022814cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:02.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.220+0000 7f022ec13640 1 -- 172.21.15.98:0/3462735768 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02280b4d40 msgr2=0x7f022809cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:02.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.220+0000 7f022ec13640 1 --2- 172.21.15.98:0/3462735768 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02280b4d40 0x7f022809cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.220+0000 7f022ec13640 1 -- 172.21.15.98:0/3462735768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02280ceb70 msgr2=0x7f022814cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:02.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.220+0000 7f022ec13640 1 --2- 172.21.15.98:0/3462735768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02280ceb70 0x7f022814cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.220+0000 7f022ec13640 1 -- 172.21.15.98:0/3462735768 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022809d0f0 con 0x7f022815bdc0 2024-05-09T10:51:02.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.220+0000 7f022ec13640 1 --2- 172.21.15.98:0/3462735768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f022815bdc0 0x7f02280c8a80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f022000cf20 tx=0x7f022000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:02.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.220+0000 7f021effd640 1 -- 172.21.15.98:0/3462735768 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022001a070 con 0x7f022815bdc0 2024-05-09T10:51:02.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.220+0000 7f021effd640 1 -- 172.21.15.98:0/3462735768 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f022000d840 con 0x7f022815bdc0 2024-05-09T10:51:02.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.220+0000 7f0230676640 1 -- 172.21.15.98:0/3462735768 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02280c9380 con 0x7f022815bdc0 2024-05-09T10:51:02.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.220+0000 7f021effd640 1 -- 172.21.15.98:0/3462735768 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022000dc30 con 0x7f022815bdc0 2024-05-09T10:51:02.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.220+0000 7f0230676640 1 -- 172.21.15.98:0/3462735768 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02280c9940 con 0x7f022815bdc0 2024-05-09T10:51:02.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.221+0000 7f0230676640 1 -- 172.21.15.98:0/3462735768 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0228069cb0 con 0x7f022815bdc0 2024-05-09T10:51:02.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.222+0000 7f021effd640 1 -- 172.21.15.98:0/3462735768 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f022001d070 con 0x7f022815bdc0 2024-05-09T10:51:02.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.222+0000 7f021effd640 1 --2- 172.21.15.98:0/3462735768 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f020c06b4d0 0x7f020c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:02.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.222+0000 7f021effd640 1 -- 172.21.15.98:0/3462735768 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f022008a750 con 0x7f022815bdc0 2024-05-09T10:51:02.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.222+0000 7f022e412640 1 --2- 172.21.15.98:0/3462735768 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f020c06b4d0 0x7f020c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:02.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.223+0000 7f022e412640 1 --2- 172.21.15.98:0/3462735768 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f020c06b4d0 0x7f020c06d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0224007fd0 tx=0x7f0224007de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:02.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.227+0000 7f021effd640 1 -- 172.21.15.98:0/3462735768 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0220059240 con 0x7f022815bdc0 2024-05-09T10:51:02.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.466+0000 7f0230676640 1 -- 172.21.15.98:0/3462735768 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f02280b5990 con 0x7f022815bdc0 2024-05-09T10:51:02.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.468+0000 7f021effd640 1 -- 172.21.15.98:0/3462735768 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f022005d2e0 con 0x7f022815bdc0 2024-05-09T10:51:02.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.470+0000 7f0230676640 1 -- 172.21.15.98:0/3462735768 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f020c06b4d0 msgr2=0x7f020c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:02.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.470+0000 7f0230676640 1 --2- 172.21.15.98:0/3462735768 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f020c06b4d0 0x7f020c06d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0224007fd0 tx=0x7f0224007de0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.470+0000 7f0230676640 1 -- 172.21.15.98:0/3462735768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f022815bdc0 msgr2=0x7f02280c8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:02.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.470+0000 7f0230676640 1 --2- 172.21.15.98:0/3462735768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f022815bdc0 0x7f02280c8a80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f022000cf20 tx=0x7f022000a7d0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.470+0000 7f0230676640 1 -- 172.21.15.98:0/3462735768 shutdown_connections 2024-05-09T10:51:02.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.470+0000 7f0230676640 1 --2- 172.21.15.98:0/3462735768 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f020c06b4d0 0x7f020c06d990 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.471+0000 7f0230676640 1 --2- 172.21.15.98:0/3462735768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f022815bdc0 0x7f02280c8a80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.471+0000 7f0230676640 1 --2- 172.21.15.98:0/3462735768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02280ceb70 0x7f022814cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.471+0000 7f0230676640 1 --2- 172.21.15.98:0/3462735768 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02280b4d40 0x7f022809cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.471+0000 7f0230676640 1 -- 172.21.15.98:0/3462735768 >> 172.21.15.98:0/3462735768 conn(0x7f02280b34e0 msgr2=0x7f022806fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:02.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.471+0000 7f0230676640 1 -- 172.21.15.98:0/3462735768 shutdown_connections 2024-05-09T10:51:02.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.471+0000 7f0230676640 1 -- 172.21.15.98:0/3462735768 wait complete. 2024-05-09T10:51:02.475 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-09T10:51:02.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.583+0000 7fb3456fc640 1 Processor -- start 2024-05-09T10:51:02.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.583+0000 7fb3456fc640 1 -- start start 2024-05-09T10:51:02.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.584+0000 7fb3456fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb34015de40 0x7fb3400d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:02.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.584+0000 7fb3456fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb3400cf250 0x7fb3400d0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:02.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.584+0000 7fb3456fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3400cf6e0 0x7fb3400d09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:02.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.584+0000 7fb3456fc640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3400d1790 con 0x7fb34015de40 2024-05-09T10:51:02.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.584+0000 7fb3456fc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb34015e220 con 0x7fb3400cf250 2024-05-09T10:51:02.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.584+0000 7fb3456fc640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3400d19e0 con 0x7fb3400cf6e0 2024-05-09T10:51:02.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.584+0000 7fb33effd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb34015de40 0x7fb3400d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:02.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.584+0000 7fb33effd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb34015de40 0x7fb3400d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35764/0 (socket says 172.21.15.98:35764) 2024-05-09T10:51:02.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.584+0000 7fb33effd640 1 -- 172.21.15.98:0/744257062 learned_addr learned my addr 172.21.15.98:0/744257062 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:02.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.584+0000 7fb337fff640 1 --2- 172.21.15.98:0/744257062 >> v2:172.21.15.105:3300/0 conn(0x7fb3400cf250 0x7fb3400d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:02.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.584+0000 7fb33f7fe640 1 --2- 172.21.15.98:0/744257062 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3400cf6e0 0x7fb3400d09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:02.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.584+0000 7fb33effd640 1 -- 172.21.15.98:0/744257062 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3400cf6e0 msgr2=0x7fb3400d09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:02.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.584+0000 7fb33effd640 1 --2- 172.21.15.98:0/744257062 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3400cf6e0 0x7fb3400d09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.584+0000 7fb33effd640 1 -- 172.21.15.98:0/744257062 >> v2:172.21.15.105:3300/0 conn(0x7fb3400cf250 msgr2=0x7fb3400d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:02.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.584+0000 7fb33effd640 1 --2- 172.21.15.98:0/744257062 >> v2:172.21.15.105:3300/0 conn(0x7fb3400cf250 0x7fb3400d0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.584+0000 7fb33effd640 1 -- 172.21.15.98:0/744257062 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3400d1cc0 con 0x7fb34015de40 2024-05-09T10:51:02.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.585+0000 7fb33effd640 1 --2- 172.21.15.98:0/744257062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb34015de40 0x7fb3400d27a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb3400d3a40 tx=0x7fb33002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439440f7661a7850 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:02.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.585+0000 7fb33e7fc640 1 -- 172.21.15.98:0/744257062 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330037070 con 0x7fb34015de40 2024-05-09T10:51:02.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.585+0000 7fb33e7fc640 1 -- 172.21.15.98:0/744257062 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb330032440 con 0x7fb34015de40 2024-05-09T10:51:02.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.585+0000 7fb33e7fc640 1 -- 172.21.15.98:0/744257062 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330032810 con 0x7fb34015de40 2024-05-09T10:51:02.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.586+0000 7fb3456fc640 1 -- 172.21.15.98:0/744257062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb34015de40 msgr2=0x7fb3400d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:02.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.586+0000 7fb3456fc640 1 --2- 172.21.15.98:0/744257062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb34015de40 0x7fb3400d27a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb3400d3a40 tx=0x7fb33002a1e0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.586+0000 7fb3456fc640 1 -- 172.21.15.98:0/744257062 shutdown_connections 2024-05-09T10:51:02.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.586+0000 7fb3456fc640 1 --2- 172.21.15.98:0/744257062 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3400cf6e0 0x7fb3400d09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.586+0000 7fb3456fc640 1 --2- 172.21.15.98:0/744257062 >> v2:172.21.15.105:3300/0 conn(0x7fb3400cf250 0x7fb3400d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.586+0000 7fb3456fc640 1 --2- 172.21.15.98:0/744257062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb34015de40 0x7fb3400d27a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.586+0000 7fb3456fc640 1 -- 172.21.15.98:0/744257062 >> 172.21.15.98:0/744257062 conn(0x7fb34009ce90 msgr2=0x7fb3400d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:02.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.586+0000 7fb3456fc640 1 -- 172.21.15.98:0/744257062 shutdown_connections 2024-05-09T10:51:02.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.586+0000 7fb3456fc640 1 -- 172.21.15.98:0/744257062 wait complete. 2024-05-09T10:51:02.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.587+0000 7fb3456fc640 1 Processor -- start 2024-05-09T10:51:02.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.587+0000 7fb3456fc640 1 -- start start 2024-05-09T10:51:02.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.587+0000 7fb3456fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3400cf250 0x7fb3400cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:02.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.588+0000 7fb3456fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3400cf6e0 0x7fb3400b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:02.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.588+0000 7fb3456fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb34015de40 0x7fb340154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:02.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.588+0000 7fb3456fc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3400d4680 con 0x7fb3400cf250 2024-05-09T10:51:02.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.588+0000 7fb3456fc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3400d2280 con 0x7fb34015de40 2024-05-09T10:51:02.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.588+0000 7fb3456fc640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3400d4800 con 0x7fb3400cf6e0 2024-05-09T10:51:02.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.588+0000 7fb33effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3400cf250 0x7fb3400cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:02.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.588+0000 7fb33effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3400cf250 0x7fb3400cb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35774/0 (socket says 172.21.15.98:35774) 2024-05-09T10:51:02.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.588+0000 7fb33effd640 1 -- 172.21.15.98:0/1717270510 learned_addr learned my addr 172.21.15.98:0/1717270510 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:02.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.588+0000 7fb337fff640 1 --2- 172.21.15.98:0/1717270510 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3400cf6e0 0x7fb3400b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:02.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.588+0000 7fb33f7fe640 1 --2- 172.21.15.98:0/1717270510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb34015de40 0x7fb340154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:02.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.588+0000 7fb33effd640 1 -- 172.21.15.98:0/1717270510 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3400cf6e0 msgr2=0x7fb3400b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:02.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.588+0000 7fb33effd640 1 --2- 172.21.15.98:0/1717270510 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3400cf6e0 0x7fb3400b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.588+0000 7fb33effd640 1 -- 172.21.15.98:0/1717270510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb34015de40 msgr2=0x7fb340154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:02.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.588+0000 7fb33effd640 1 --2- 172.21.15.98:0/1717270510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb34015de40 0x7fb340154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.588+0000 7fb33effd640 1 -- 172.21.15.98:0/1717270510 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3400b4380 con 0x7fb3400cf250 2024-05-09T10:51:02.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.588+0000 7fb33effd640 1 --2- 172.21.15.98:0/1717270510 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3400cf250 0x7fb3400cb960 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb330000c00 tx=0x7fb3300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:02.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.588+0000 7fb3377fe640 1 -- 172.21.15.98:0/1717270510 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330041020 con 0x7fb3400cf250 2024-05-09T10:51:02.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.588+0000 7fb3377fe640 1 -- 172.21.15.98:0/1717270510 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3300306c0 con 0x7fb3400cf250 2024-05-09T10:51:02.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.588+0000 7fb3456fc640 1 -- 172.21.15.98:0/1717270510 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3400b0920 con 0x7fb3400cf250 2024-05-09T10:51:02.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.588+0000 7fb3377fe640 1 -- 172.21.15.98:0/1717270510 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330037040 con 0x7fb3400cf250 2024-05-09T10:51:02.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.588+0000 7fb3456fc640 1 -- 172.21.15.98:0/1717270510 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3400b0ee0 con 0x7fb3400cf250 2024-05-09T10:51:02.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.589+0000 7fb3456fc640 1 -- 172.21.15.98:0/1717270510 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb340069cb0 con 0x7fb3400cf250 2024-05-09T10:51:02.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.590+0000 7fb3377fe640 1 -- 172.21.15.98:0/1717270510 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb330044020 con 0x7fb3400cf250 2024-05-09T10:51:02.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.590+0000 7fb3377fe640 1 --2- 172.21.15.98:0/1717270510 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb32006b4d0 0x7fb32006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:02.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.590+0000 7fb3377fe640 1 -- 172.21.15.98:0/1717270510 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb3300aa500 con 0x7fb3400cf250 2024-05-09T10:51:02.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.590+0000 7fb337fff640 1 --2- 172.21.15.98:0/1717270510 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb32006b4d0 0x7fb32006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:02.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.591+0000 7fb337fff640 1 --2- 172.21.15.98:0/1717270510 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb32006b4d0 0x7fb32006d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb3401557f0 tx=0x7fb328008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:02.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.594+0000 7fb3377fe640 1 -- 172.21.15.98:0/1717270510 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb33004b030 con 0x7fb3400cf250 2024-05-09T10:51:02.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.843+0000 7fb3456fc640 1 -- 172.21.15.98:0/1717270510 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fb3400d42d0 con 0x7fb3400cf250 2024-05-09T10:51:02.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.845+0000 7fb3377fe640 1 -- 172.21.15.98:0/1717270510 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb3400b4380 con 0x7fb3400cf250 2024-05-09T10:51:02.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.847+0000 7fb3456fc640 1 -- 172.21.15.98:0/1717270510 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb32006b4d0 msgr2=0x7fb32006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:02.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.847+0000 7fb3456fc640 1 --2- 172.21.15.98:0/1717270510 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb32006b4d0 0x7fb32006d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb3401557f0 tx=0x7fb328008040 comp rx=0 tx=0).stop 2024-05-09T10:51:02.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.847+0000 7fb3456fc640 1 -- 172.21.15.98:0/1717270510 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3400cf250 msgr2=0x7fb3400cb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:02.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.847+0000 7fb3456fc640 1 --2- 172.21.15.98:0/1717270510 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3400cf250 0x7fb3400cb960 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb330000c00 tx=0x7fb3300046c0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.847+0000 7fb3456fc640 1 -- 172.21.15.98:0/1717270510 shutdown_connections 2024-05-09T10:51:02.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.847+0000 7fb3456fc640 1 --2- 172.21.15.98:0/1717270510 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb32006b4d0 0x7fb32006d990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.847+0000 7fb3456fc640 1 --2- 172.21.15.98:0/1717270510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb34015de40 0x7fb340154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.847+0000 7fb3456fc640 1 --2- 172.21.15.98:0/1717270510 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3400cf6e0 0x7fb3400b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.847+0000 7fb3456fc640 1 --2- 172.21.15.98:0/1717270510 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3400cf250 0x7fb3400cb960 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.847+0000 7fb3456fc640 1 -- 172.21.15.98:0/1717270510 >> 172.21.15.98:0/1717270510 conn(0x7fb34009ce90 msgr2=0x7fb3400d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:02.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.847+0000 7fb3456fc640 1 -- 172.21.15.98:0/1717270510 shutdown_connections 2024-05-09T10:51:02.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.847+0000 7fb3456fc640 1 -- 172.21.15.98:0/1717270510 wait complete. 2024-05-09T10:51:02.852 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-09T10:51:02.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.959+0000 7ff2bb83a640 1 Processor -- start 2024-05-09T10:51:02.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.959+0000 7ff2bb83a640 1 -- start start 2024-05-09T10:51:02.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.960+0000 7ff2bb83a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff2b40afba0 0x7ff2b4071c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:02.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.960+0000 7ff2bb83a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b415ebf0 con 0x7ff2b4147470 2024-05-09T10:51:02.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.960+0000 7ff2bb83a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b415eed0 con 0x7ff2b40721d0 2024-05-09T10:51:02.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.960+0000 7ff2bb83a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b4071800 con 0x7ff2b40afba0 2024-05-09T10:51:02.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.960+0000 7ff2b95d6640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff2b4147470 0x7ff2b415e280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38932/0 (socket says 172.21.15.98:38932) 2024-05-09T10:51:02.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.960+0000 7ff2b8dd5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff2b40afba0 0x7ff2b4071c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:02.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.960+0000 7ff2b95d6640 1 -- 172.21.15.98:0/3624861442 learned_addr learned my addr 172.21.15.98:0/3624861442 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:02.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.961+0000 7ff2b8dd5640 1 -- 172.21.15.98:0/3624861442 >> v1:172.21.15.105:6789/0 conn(0x7ff2b40721d0 legacy=0x7ff2b406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:02.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.961+0000 7ff2b8dd5640 1 -- 172.21.15.98:0/3624861442 >> v1:172.21.15.98:6789/0 conn(0x7ff2b4147470 legacy=0x7ff2b415e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:02.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.961+0000 7ff2b8dd5640 1 -- 172.21.15.98:0/3624861442 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b415f1b0 con 0x7ff2b40afba0 2024-05-09T10:51:02.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.961+0000 7ff2b8dd5640 1 --2- 172.21.15.98:0/3624861442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2b40afba0 0x7ff2b4071c90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4071980 tx=0x7ff2a802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8661d029e5cdbde1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:02.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.962+0000 7ff2b3fff640 1 -- 172.21.15.98:0/3624861442 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8037070 con 0x7ff2b40afba0 2024-05-09T10:51:02.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.962+0000 7ff2b3fff640 1 -- 172.21.15.98:0/3624861442 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a8032440 con 0x7ff2b40afba0 2024-05-09T10:51:02.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.962+0000 7ff2b3fff640 1 -- 172.21.15.98:0/3624861442 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8032810 con 0x7ff2b40afba0 2024-05-09T10:51:02.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.962+0000 7ff2bb83a640 1 -- 172.21.15.98:0/3624861442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2b40afba0 msgr2=0x7ff2b4071c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:02.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.962+0000 7ff2bb83a640 1 --2- 172.21.15.98:0/3624861442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2b40afba0 0x7ff2b4071c90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4071980 tx=0x7ff2a802a1c0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.962+0000 7ff2bb83a640 1 -- 172.21.15.98:0/3624861442 shutdown_connections 2024-05-09T10:51:02.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.962+0000 7ff2bb83a640 1 --2- 172.21.15.98:0/3624861442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2b40afba0 0x7ff2b4071c90 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.962+0000 7ff2bb83a640 1 -- 172.21.15.98:0/3624861442 >> 172.21.15.98:0/3624861442 conn(0x7ff2b40b24c0 msgr2=0x7ff2b40b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:02.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.962+0000 7ff2bb83a640 1 -- 172.21.15.98:0/3624861442 shutdown_connections 2024-05-09T10:51:02.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.962+0000 7ff2bb83a640 1 -- 172.21.15.98:0/3624861442 wait complete. 2024-05-09T10:51:02.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.963+0000 7ff2bb83a640 1 Processor -- start 2024-05-09T10:51:02.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.963+0000 7ff2bb83a640 1 -- start start 2024-05-09T10:51:02.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.964+0000 7ff2bb83a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2b40afba0 0x7ff2b415d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:02.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.964+0000 7ff2bb83a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2b4147470 0x7ff2b4135c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:02.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.964+0000 7ff2bb83a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2b4136150 0x7ff2b40b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:02.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.964+0000 7ff2bb83a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b4147850 con 0x7ff2b4147470 2024-05-09T10:51:02.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.964+0000 7ff2bb83a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b415e990 con 0x7ff2b40afba0 2024-05-09T10:51:02.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.964+0000 7ff2bb83a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b41479d0 con 0x7ff2b4136150 2024-05-09T10:51:02.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.964+0000 7ff2b8dd5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2b4147470 0x7ff2b4135c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:02.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.964+0000 7ff2b8dd5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2b4147470 0x7ff2b4135c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35798/0 (socket says 172.21.15.98:35798) 2024-05-09T10:51:02.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.964+0000 7ff2b8dd5640 1 -- 172.21.15.98:0/276079849 learned_addr learned my addr 172.21.15.98:0/276079849 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:02.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.964+0000 7ff2b8dd5640 1 -- 172.21.15.98:0/276079849 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2b4136150 msgr2=0x7ff2b40b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:02.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.964+0000 7ff2b9dd7640 1 --2- 172.21.15.98:0/276079849 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2b4136150 0x7ff2b40b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:02.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.964+0000 7ff2b95d6640 1 --2- 172.21.15.98:0/276079849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2b40afba0 0x7ff2b415d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:02.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.964+0000 7ff2b8dd5640 1 --2- 172.21.15.98:0/276079849 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2b4136150 0x7ff2b40b49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.964+0000 7ff2b8dd5640 1 -- 172.21.15.98:0/276079849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2b40afba0 msgr2=0x7ff2b415d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:02.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.964+0000 7ff2b8dd5640 1 --2- 172.21.15.98:0/276079849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2b40afba0 0x7ff2b415d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:02.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.964+0000 7ff2b8dd5640 1 -- 172.21.15.98:0/276079849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b40b56d0 con 0x7ff2b4147470 2024-05-09T10:51:02.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.965+0000 7ff2b8dd5640 1 --2- 172.21.15.98:0/276079849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2b4147470 0x7ff2b4135c10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff2a802a200 tx=0x7ff2a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:02.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.965+0000 7ff2b1ffb640 1 -- 172.21.15.98:0/276079849 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8041070 con 0x7ff2b4147470 2024-05-09T10:51:02.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.965+0000 7ff2b1ffb640 1 -- 172.21.15.98:0/276079849 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a803c890 con 0x7ff2b4147470 2024-05-09T10:51:02.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.965+0000 7ff2bb83a640 1 -- 172.21.15.98:0/276079849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b40b53f0 con 0x7ff2b4147470 2024-05-09T10:51:02.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.965+0000 7ff2b1ffb640 1 -- 172.21.15.98:0/276079849 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8037040 con 0x7ff2b4147470 2024-05-09T10:51:02.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.965+0000 7ff2bb83a640 1 -- 172.21.15.98:0/276079849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b40c5b20 con 0x7ff2b4147470 2024-05-09T10:51:02.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.966+0000 7ff2bb83a640 1 -- 172.21.15.98:0/276079849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b4069cb0 con 0x7ff2b4147470 2024-05-09T10:51:02.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.967+0000 7ff2b1ffb640 1 -- 172.21.15.98:0/276079849 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff2a8044070 con 0x7ff2b4147470 2024-05-09T10:51:02.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.967+0000 7ff2b1ffb640 1 --2- 172.21.15.98:0/276079849 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff2a006b4d0 0x7ff2a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:02.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.967+0000 7ff2b1ffb640 1 -- 172.21.15.98:0/276079849 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff2a80aac70 con 0x7ff2b4147470 2024-05-09T10:51:02.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.967+0000 7ff2b95d6640 1 --2- 172.21.15.98:0/276079849 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff2a006b4d0 0x7ff2a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:02.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.967+0000 7ff2b95d6640 1 --2- 172.21.15.98:0/276079849 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff2a006b4d0 0x7ff2a006d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40012e0 tx=0x7ff2a4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:02.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:02.971+0000 7ff2b1ffb640 1 -- 172.21.15.98:0/276079849 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff2a8079ff0 con 0x7ff2b4147470 2024-05-09T10:51:03.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.205+0000 7ff2bb83a640 1 -- 172.21.15.98:0/276079849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7ff2b40b5a00 con 0x7ff2b4147470 2024-05-09T10:51:03.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.207+0000 7ff2b1ffb640 1 -- 172.21.15.98:0/276079849 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff2a80798f0 con 0x7ff2b4147470 2024-05-09T10:51:03.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.209+0000 7ff2bb83a640 1 -- 172.21.15.98:0/276079849 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff2a006b4d0 msgr2=0x7ff2a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:03.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.209+0000 7ff2bb83a640 1 --2- 172.21.15.98:0/276079849 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff2a006b4d0 0x7ff2a006d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40012e0 tx=0x7ff2a4009040 comp rx=0 tx=0).stop 2024-05-09T10:51:03.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.209+0000 7ff2bb83a640 1 -- 172.21.15.98:0/276079849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2b4147470 msgr2=0x7ff2b4135c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:03.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.209+0000 7ff2bb83a640 1 --2- 172.21.15.98:0/276079849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2b4147470 0x7ff2b4135c10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff2a802a200 tx=0x7ff2a80046c0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.209+0000 7ff2bb83a640 1 -- 172.21.15.98:0/276079849 shutdown_connections 2024-05-09T10:51:03.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.209+0000 7ff2bb83a640 1 --2- 172.21.15.98:0/276079849 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff2a006b4d0 0x7ff2a006d990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.209+0000 7ff2bb83a640 1 --2- 172.21.15.98:0/276079849 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2b4136150 0x7ff2b40b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.209+0000 7ff2bb83a640 1 --2- 172.21.15.98:0/276079849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2b4147470 0x7ff2b4135c10 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.209+0000 7ff2bb83a640 1 --2- 172.21.15.98:0/276079849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2b40afba0 0x7ff2b415d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.209+0000 7ff2bb83a640 1 -- 172.21.15.98:0/276079849 >> 172.21.15.98:0/276079849 conn(0x7ff2b40b24c0 msgr2=0x7ff2b415f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:03.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.209+0000 7ff2bb83a640 1 -- 172.21.15.98:0/276079849 shutdown_connections 2024-05-09T10:51:03.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.209+0000 7ff2bb83a640 1 -- 172.21.15.98:0/276079849 wait complete. 2024-05-09T10:51:03.214 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-09T10:51:03.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.322+0000 7fad551d5640 1 Processor -- start 2024-05-09T10:51:03.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.322+0000 7fad551d5640 1 -- start start 2024-05-09T10:51:03.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.323+0000 7fad551d5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fad5015dce0 0x7fad5006bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:03.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.323+0000 7fad551d5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad500d0c70 con 0x7fad5015e0c0 2024-05-09T10:51:03.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.323+0000 7fad551d5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad500d0f50 con 0x7fad5006cac0 2024-05-09T10:51:03.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.323+0000 7fad551d5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad500b4350 con 0x7fad5015dce0 2024-05-09T10:51:03.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.323+0000 7fad4659c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fad5015e0c0 0x7fad5006c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38946/0 (socket says 172.21.15.98:38946) 2024-05-09T10:51:03.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.323+0000 7fad4659c640 1 -- 172.21.15.98:0/4096784504 learned_addr learned my addr 172.21.15.98:0/4096784504 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:03.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.323+0000 7fad4ed9d640 1 --2- 172.21.15.98:0/4096784504 >> v2:172.21.15.105:3300/0 conn(0x7fad5015dce0 0x7fad5006bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:03.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.323+0000 7fad4e59c640 1 -- 172.21.15.98:0/4096784504 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1422112362 0 0) 0x7fad500d0c70 con 0x7fad5015e0c0 2024-05-09T10:51:03.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.323+0000 7fad4e59c640 1 -- 172.21.15.98:0/4096784504 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad3c003520 con 0x7fad5015e0c0 2024-05-09T10:51:03.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.323+0000 7fad4e59c640 1 -- 172.21.15.98:0/4096784504 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2019658964 0 0) 0x7fad3c003520 con 0x7fad5015e0c0 2024-05-09T10:51:03.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.323+0000 7fad4e59c640 1 -- 172.21.15.98:0/4096784504 >> v2:172.21.15.105:3300/0 conn(0x7fad5015dce0 msgr2=0x7fad5006bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:03.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.323+0000 7fad4e59c640 1 --2- 172.21.15.98:0/4096784504 >> v2:172.21.15.105:3300/0 conn(0x7fad5015dce0 0x7fad5006bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.323+0000 7fad4e59c640 1 -- 172.21.15.98:0/4096784504 >> v1:172.21.15.105:6789/0 conn(0x7fad5006cac0 legacy=0x7fad500b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:03.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.323+0000 7fad4e59c640 1 -- 172.21.15.98:0/4096784504 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad500b0fd0 con 0x7fad5015e0c0 2024-05-09T10:51:03.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.323+0000 7fad4e59c640 1 -- 172.21.15.98:0/4096784504 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7fad38001670 con 0x7fad5015e0c0 2024-05-09T10:51:03.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.323+0000 7fad4ed9d640 1 --2- 172.21.15.98:0/4096784504 >> v2:172.21.15.105:3300/0 conn(0x7fad5015dce0 0x7fad5006bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:51:03.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.323+0000 7fad4e59c640 1 -- 172.21.15.98:0/4096784504 >> v1:172.21.15.98:6789/0 conn(0x7fad5015e0c0 legacy=0x7fad5006c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:03.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.323+0000 7fad4e59c640 1 --2- 172.21.15.98:0/4096784504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad3c0039b0 0x7fad3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:03.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.323+0000 7fad4e59c640 1 --2- 172.21.15.98:0/4096784504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad3c004800 0x7fad3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:03.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.323+0000 7fad4e59c640 1 --2- 172.21.15.98:0/4096784504 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad3c0051c0 0x7fad3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:03.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.324+0000 7fad4e59c640 1 -- 172.21.15.98:0/4096784504 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c00c040 con 0x7fad3c004800 2024-05-09T10:51:03.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.324+0000 7fad4e59c640 1 -- 172.21.15.98:0/4096784504 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c00c1f0 con 0x7fad3c0039b0 2024-05-09T10:51:03.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.324+0000 7fad4e59c640 1 -- 172.21.15.98:0/4096784504 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c00c3f0 con 0x7fad3c0051c0 2024-05-09T10:51:03.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.324+0000 7fad4659c640 1 --2- 172.21.15.98:0/4096784504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad3c004800 0x7fad3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:03.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.324+0000 7fad4ed9d640 1 --2- 172.21.15.98:0/4096784504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad3c0039b0 0x7fad3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:03.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.324+0000 7fad4f59e640 1 --2- 172.21.15.98:0/4096784504 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad3c0051c0 0x7fad3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:03.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.324+0000 7fad4659c640 1 -- 172.21.15.98:0/4096784504 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad3c0051c0 msgr2=0x7fad3c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:03.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.324+0000 7fad4659c640 1 --2- 172.21.15.98:0/4096784504 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad3c0051c0 0x7fad3c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.324+0000 7fad4659c640 1 -- 172.21.15.98:0/4096784504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad3c0039b0 msgr2=0x7fad3c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:03.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.324+0000 7fad4659c640 1 --2- 172.21.15.98:0/4096784504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad3c0039b0 0x7fad3c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.324+0000 7fad4659c640 1 -- 172.21.15.98:0/4096784504 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3c003520 con 0x7fad3c004800 2024-05-09T10:51:03.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.324+0000 7fad4f59e640 1 -- 172.21.15.98:0/4096784504 reap_dead start 2024-05-09T10:51:03.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.325+0000 7fad4659c640 1 --2- 172.21.15.98:0/4096784504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad3c004800 0x7fad3c004c80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fad3800c210 tx=0x7fad3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db432cab414a1d79 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:03.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.325+0000 7fad4e59c640 1 -- 172.21.15.98:0/4096784504 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad38007cf0 con 0x7fad3c004800 2024-05-09T10:51:03.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.325+0000 7fad4e59c640 1 -- 172.21.15.98:0/4096784504 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad38005af0 con 0x7fad3c004800 2024-05-09T10:51:03.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.325+0000 7fad4e59c640 1 -- 172.21.15.98:0/4096784504 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad38007070 con 0x7fad3c004800 2024-05-09T10:51:03.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.325+0000 7fad551d5640 1 -- 172.21.15.98:0/4096784504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad3c004800 msgr2=0x7fad3c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:03.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.325+0000 7fad551d5640 1 --2- 172.21.15.98:0/4096784504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad3c004800 0x7fad3c004c80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fad3800c210 tx=0x7fad3c005830 comp rx=0 tx=0).stop 2024-05-09T10:51:03.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.325+0000 7fad551d5640 1 -- 172.21.15.98:0/4096784504 shutdown_connections 2024-05-09T10:51:03.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.325+0000 7fad551d5640 1 --2- 172.21.15.98:0/4096784504 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad3c004800 0x7fad3c004c80 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.325+0000 7fad551d5640 1 -- 172.21.15.98:0/4096784504 >> 172.21.15.98:0/4096784504 conn(0x7fad500b34e0 msgr2=0x7fad500b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:03.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.325+0000 7fad551d5640 1 -- 172.21.15.98:0/4096784504 shutdown_connections 2024-05-09T10:51:03.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.325+0000 7fad551d5640 1 -- 172.21.15.98:0/4096784504 wait complete. 2024-05-09T10:51:03.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.326+0000 7fad551d5640 1 Processor -- start 2024-05-09T10:51:03.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.327+0000 7fad551d5640 1 -- start start 2024-05-09T10:51:03.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.327+0000 7fad551d5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad3c004800 0x7fad5014b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:03.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.327+0000 7fad551d5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad500b5710 0x7fad500b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:03.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.327+0000 7fad551d5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad501e8920 0x7fad501e8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:03.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.327+0000 7fad551d5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad500d0700 con 0x7fad500b5710 2024-05-09T10:51:03.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.327+0000 7fad551d5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad500b1590 con 0x7fad3c004800 2024-05-09T10:51:03.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.327+0000 7fad551d5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad500d0880 con 0x7fad501e8920 2024-05-09T10:51:03.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.327+0000 7fad4659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad500b5710 0x7fad500b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:03.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.328+0000 7fad4659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad500b5710 0x7fad500b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35818/0 (socket says 172.21.15.98:35818) 2024-05-09T10:51:03.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.328+0000 7fad4659c640 1 -- 172.21.15.98:0/3858772246 learned_addr learned my addr 172.21.15.98:0/3858772246 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:03.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.328+0000 7fad4ed9d640 1 --2- 172.21.15.98:0/3858772246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad3c004800 0x7fad5014b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:03.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.328+0000 7fad4f59e640 1 --2- 172.21.15.98:0/3858772246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad501e8920 0x7fad501e8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:03.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.328+0000 7fad4659c640 1 -- 172.21.15.98:0/3858772246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad501e8920 msgr2=0x7fad501e8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:03.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.328+0000 7fad4659c640 1 --2- 172.21.15.98:0/3858772246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad501e8920 0x7fad501e8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.328+0000 7fad4659c640 1 -- 172.21.15.98:0/3858772246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad3c004800 msgr2=0x7fad5014b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:03.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.328+0000 7fad4659c640 1 --2- 172.21.15.98:0/3858772246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad3c004800 0x7fad5014b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.328+0000 7fad4659c640 1 -- 172.21.15.98:0/3858772246 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad501f3c20 con 0x7fad500b5710 2024-05-09T10:51:03.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.328+0000 7fad4659c640 1 --2- 172.21.15.98:0/3858772246 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad500b5710 0x7fad500b5b70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fad3800c210 tx=0x7fad500b11a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:03.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.328+0000 7fad47fff640 1 -- 172.21.15.98:0/3858772246 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad38041020 con 0x7fad500b5710 2024-05-09T10:51:03.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.328+0000 7fad47fff640 1 -- 172.21.15.98:0/3858772246 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad38038040 con 0x7fad500b5710 2024-05-09T10:51:03.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.328+0000 7fad551d5640 1 -- 172.21.15.98:0/3858772246 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad501e7880 con 0x7fad500b5710 2024-05-09T10:51:03.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.328+0000 7fad47fff640 1 -- 172.21.15.98:0/3858772246 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad38007290 con 0x7fad500b5710 2024-05-09T10:51:03.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.328+0000 7fad551d5640 1 -- 172.21.15.98:0/3858772246 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad501e7e40 con 0x7fad500b5710 2024-05-09T10:51:03.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.329+0000 7fad551d5640 1 -- 172.21.15.98:0/3858772246 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad50069cb0 con 0x7fad500b5710 2024-05-09T10:51:03.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.329+0000 7fad47fff640 1 -- 172.21.15.98:0/3858772246 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad38044020 con 0x7fad500b5710 2024-05-09T10:51:03.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.330+0000 7fad47fff640 1 --2- 172.21.15.98:0/3858772246 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fad3006b4d0 0x7fad3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:03.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.330+0000 7fad4ed9d640 1 --2- 172.21.15.98:0/3858772246 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fad3006b4d0 0x7fad3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:03.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.330+0000 7fad47fff640 1 -- 172.21.15.98:0/3858772246 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fad380aa0a0 con 0x7fad500b5710 2024-05-09T10:51:03.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.330+0000 7fad4ed9d640 1 --2- 172.21.15.98:0/3858772246 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fad3006b4d0 0x7fad3006d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fad40009dc0 tx=0x7fad40009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:03.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.334+0000 7fad47fff640 1 -- 172.21.15.98:0/3858772246 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad38078c30 con 0x7fad500b5710 2024-05-09T10:51:03.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.572+0000 7fad551d5640 1 -- 172.21.15.98:0/3858772246 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fad500d14f0 con 0x7fad500b5710 2024-05-09T10:51:03.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.574+0000 7fad47fff640 1 -- 172.21.15.98:0/3858772246 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7fad3807ccd0 con 0x7fad500b5710 2024-05-09T10:51:03.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.576+0000 7fad551d5640 1 -- 172.21.15.98:0/3858772246 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fad3006b4d0 msgr2=0x7fad3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:03.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.576+0000 7fad551d5640 1 --2- 172.21.15.98:0/3858772246 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fad3006b4d0 0x7fad3006d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fad40009dc0 tx=0x7fad40009190 comp rx=0 tx=0).stop 2024-05-09T10:51:03.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.576+0000 7fad551d5640 1 -- 172.21.15.98:0/3858772246 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad500b5710 msgr2=0x7fad500b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:03.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.576+0000 7fad551d5640 1 --2- 172.21.15.98:0/3858772246 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad500b5710 0x7fad500b5b70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fad3800c210 tx=0x7fad500b11a0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.576+0000 7fad551d5640 1 -- 172.21.15.98:0/3858772246 shutdown_connections 2024-05-09T10:51:03.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.576+0000 7fad551d5640 1 --2- 172.21.15.98:0/3858772246 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fad3006b4d0 0x7fad3006d990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.576+0000 7fad551d5640 1 --2- 172.21.15.98:0/3858772246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fad501e8920 0x7fad501e8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.576+0000 7fad551d5640 1 --2- 172.21.15.98:0/3858772246 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad500b5710 0x7fad500b5b70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.576+0000 7fad551d5640 1 --2- 172.21.15.98:0/3858772246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad3c004800 0x7fad5014b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.576+0000 7fad551d5640 1 -- 172.21.15.98:0/3858772246 >> 172.21.15.98:0/3858772246 conn(0x7fad500b34e0 msgr2=0x7fad5006fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:03.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.576+0000 7fad551d5640 1 -- 172.21.15.98:0/3858772246 shutdown_connections 2024-05-09T10:51:03.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.576+0000 7fad551d5640 1 -- 172.21.15.98:0/3858772246 wait complete. 2024-05-09T10:51:03.581 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-09T10:51:03.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.688+0000 7ff00cdc1640 1 Processor -- start 2024-05-09T10:51:03.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.688+0000 7ff00cdc1640 1 -- start start 2024-05-09T10:51:03.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.689+0000 7ff00cdc1640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0080d27a0 0x7ff0080d0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:03.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.689+0000 7ff00cdc1640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff0080d09b0 0x7ff0080b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:03.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.689+0000 7ff00cdc1640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff00815e220 con 0x7ff00815de40 2024-05-09T10:51:03.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.689+0000 7ff00cdc1640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0080d1790 con 0x7ff0080d09b0 2024-05-09T10:51:03.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.689+0000 7ff00cdc1640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0080d2c00 con 0x7ff0080d27a0 2024-05-09T10:51:03.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.690+0000 7ff006d9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff0080d09b0 0x7ff0080b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:03.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.690+0000 7ff006d9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff0080d09b0 0x7ff0080b6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48808/0 (socket says 172.21.15.98:48808) 2024-05-09T10:51:03.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.690+0000 7ff006d9d640 1 -- 172.21.15.98:0/2748496448 learned_addr learned my addr 172.21.15.98:0/2748496448 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:03.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.690+0000 7ff005d9b640 1 --2- 172.21.15.98:0/2748496448 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0080d27a0 0x7ff0080d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:03.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.690+0000 7ff006d9d640 1 -- 172.21.15.98:0/2748496448 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0080d27a0 msgr2=0x7ff0080d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:03.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.690+0000 7ff006d9d640 1 --2- 172.21.15.98:0/2748496448 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0080d27a0 0x7ff0080d0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.690+0000 7ff006d9d640 1 -- 172.21.15.98:0/2748496448 >> v1:172.21.15.105:6789/0 conn(0x7ff00815de40 legacy=0x7ff0080cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:03.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.690+0000 7ff006d9d640 1 -- 172.21.15.98:0/2748496448 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0080d19e0 con 0x7ff0080d09b0 2024-05-09T10:51:03.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.691+0000 7ff006d9d640 1 --2- 172.21.15.98:0/2748496448 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0080d09b0 0x7ff0080b6a10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff000009e50 tx=0x7ff00002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa94e694a110920 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:03.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.691+0000 7ff00559a640 1 -- 172.21.15.98:0/2748496448 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff000036070 con 0x7ff0080d09b0 2024-05-09T10:51:03.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.691+0000 7ff00559a640 1 -- 172.21.15.98:0/2748496448 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff000031600 con 0x7ff0080d09b0 2024-05-09T10:51:03.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.691+0000 7ff00559a640 1 -- 172.21.15.98:0/2748496448 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0000319d0 con 0x7ff0080d09b0 2024-05-09T10:51:03.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.691+0000 7ff00cdc1640 1 -- 172.21.15.98:0/2748496448 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0080d09b0 msgr2=0x7ff0080b6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:03.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.691+0000 7ff00cdc1640 1 --2- 172.21.15.98:0/2748496448 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0080d09b0 0x7ff0080b6a10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff000009e50 tx=0x7ff00002a730 comp rx=0 tx=0).stop 2024-05-09T10:51:03.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.691+0000 7ff00cdc1640 1 -- 172.21.15.98:0/2748496448 shutdown_connections 2024-05-09T10:51:03.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.691+0000 7ff00cdc1640 1 --2- 172.21.15.98:0/2748496448 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0080d27a0 0x7ff0080d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.691+0000 7ff00cdc1640 1 --2- 172.21.15.98:0/2748496448 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0080d09b0 0x7ff0080b6a10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.691+0000 7ff00cdc1640 1 -- 172.21.15.98:0/2748496448 >> 172.21.15.98:0/2748496448 conn(0x7ff00809ce90 msgr2=0x7ff0080d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:03.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.691+0000 7ff00cdc1640 1 -- 172.21.15.98:0/2748496448 shutdown_connections 2024-05-09T10:51:03.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.691+0000 7ff00cdc1640 1 -- 172.21.15.98:0/2748496448 wait complete. 2024-05-09T10:51:03.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.693+0000 7ff00cdc1640 1 Processor -- start 2024-05-09T10:51:03.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.693+0000 7ff00cdc1640 1 -- start start 2024-05-09T10:51:03.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.693+0000 7ff00cdc1640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0080d09b0 0x7ff0080ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:03.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.693+0000 7ff00cdc1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0080d27a0 0x7ff0081578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:03.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.693+0000 7ff00cdc1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff00815de40 0x7ff008157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:03.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.693+0000 7ff00cdc1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0080d2280 con 0x7ff0080d27a0 2024-05-09T10:51:03.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.693+0000 7ff00cdc1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0080d1fa0 con 0x7ff00815de40 2024-05-09T10:51:03.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.693+0000 7ff00cdc1640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0080cf960 con 0x7ff0080d09b0 2024-05-09T10:51:03.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.694+0000 7ff005d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0080d27a0 0x7ff0081578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:03.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.694+0000 7ff005d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0080d27a0 0x7ff0081578c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48816/0 (socket says 172.21.15.98:48816) 2024-05-09T10:51:03.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.694+0000 7ff005d9b640 1 -- 172.21.15.98:0/3359673494 learned_addr learned my addr 172.21.15.98:0/3359673494 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:03.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.694+0000 7ff00659c640 1 --2- 172.21.15.98:0/3359673494 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0080d09b0 0x7ff0080ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:03.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.694+0000 7ff006d9d640 1 --2- 172.21.15.98:0/3359673494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff00815de40 0x7ff008157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:03.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.694+0000 7ff005d9b640 1 -- 172.21.15.98:0/3359673494 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0080d09b0 msgr2=0x7ff0080ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:03.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.694+0000 7ff005d9b640 1 --2- 172.21.15.98:0/3359673494 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0080d09b0 0x7ff0080ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.694+0000 7ff005d9b640 1 -- 172.21.15.98:0/3359673494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff00815de40 msgr2=0x7ff008157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:03.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.694+0000 7ff005d9b640 1 --2- 172.21.15.98:0/3359673494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff00815de40 0x7ff008157e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.694+0000 7ff005d9b640 1 -- 172.21.15.98:0/3359673494 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff008158620 con 0x7ff0080d27a0 2024-05-09T10:51:03.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.694+0000 7ff006d9d640 1 --2- 172.21.15.98:0/3359673494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff00815de40 0x7ff008157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:03.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.694+0000 7ff005d9b640 1 --2- 172.21.15.98:0/3359673494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0080d27a0 0x7ff0081578c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7feffc00e9b0 tx=0x7feffc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:03.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.694+0000 7feff6ffd640 1 -- 172.21.15.98:0/3359673494 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc017070 con 0x7ff0080d27a0 2024-05-09T10:51:03.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.694+0000 7feff6ffd640 1 -- 172.21.15.98:0/3359673494 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feffc00c9a0 con 0x7ff0080d27a0 2024-05-09T10:51:03.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.694+0000 7ff00cdc1640 1 -- 172.21.15.98:0/3359673494 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff008158900 con 0x7ff0080d27a0 2024-05-09T10:51:03.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.694+0000 7feff6ffd640 1 -- 172.21.15.98:0/3359673494 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc00cd90 con 0x7ff0080d27a0 2024-05-09T10:51:03.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.694+0000 7ff00cdc1640 1 -- 172.21.15.98:0/3359673494 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0080cad30 con 0x7ff0080d27a0 2024-05-09T10:51:03.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.695+0000 7ff00cdc1640 1 -- 172.21.15.98:0/3359673494 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff008158ad0 con 0x7ff0080d27a0 2024-05-09T10:51:03.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.696+0000 7feff6ffd640 1 -- 172.21.15.98:0/3359673494 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feffc01a070 con 0x7ff0080d27a0 2024-05-09T10:51:03.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.696+0000 7feff6ffd640 1 --2- 172.21.15.98:0/3359673494 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fefe806b4d0 0x7fefe806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:03.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.696+0000 7feff6ffd640 1 -- 172.21.15.98:0/3359673494 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4857+0+0 (secure 0 0 0) 0x7feffc08a1a0 con 0x7ff0080d27a0 2024-05-09T10:51:03.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.696+0000 7ff00659c640 1 --2- 172.21.15.98:0/3359673494 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fefe806b4d0 0x7fefe806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:03.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.696+0000 7ff00659c640 1 --2- 172.21.15.98:0/3359673494 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fefe806b4d0 0x7fefe806d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7feff80016e0 tx=0x7feff8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:03.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.700+0000 7feff6ffd640 1 -- 172.21.15.98:0/3359673494 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feffc058d20 con 0x7ff0080d27a0 2024-05-09T10:51:03.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.937+0000 7ff00cdc1640 1 -- 172.21.15.98:0/3359673494 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7ff008069cb0 con 0x7ff0080d27a0 2024-05-09T10:51:03.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.939+0000 7feff6ffd640 1 -- 172.21.15.98:0/3359673494 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7feffc05cdc0 con 0x7ff0080d27a0 2024-05-09T10:51:03.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.941+0000 7ff00cdc1640 1 -- 172.21.15.98:0/3359673494 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fefe806b4d0 msgr2=0x7fefe806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:03.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.941+0000 7ff00cdc1640 1 --2- 172.21.15.98:0/3359673494 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fefe806b4d0 0x7fefe806d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7feff80016e0 tx=0x7feff8009040 comp rx=0 tx=0).stop 2024-05-09T10:51:03.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.941+0000 7ff00cdc1640 1 -- 172.21.15.98:0/3359673494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0080d27a0 msgr2=0x7ff0081578c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:03.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.941+0000 7ff00cdc1640 1 --2- 172.21.15.98:0/3359673494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0080d27a0 0x7ff0081578c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7feffc00e9b0 tx=0x7feffc00ee70 comp rx=0 tx=0).stop 2024-05-09T10:51:03.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.941+0000 7ff00cdc1640 1 -- 172.21.15.98:0/3359673494 shutdown_connections 2024-05-09T10:51:03.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.941+0000 7ff00cdc1640 1 --2- 172.21.15.98:0/3359673494 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fefe806b4d0 0x7fefe806d990 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.941+0000 7ff00cdc1640 1 --2- 172.21.15.98:0/3359673494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff00815de40 0x7ff008157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.941+0000 7ff00cdc1640 1 --2- 172.21.15.98:0/3359673494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0080d27a0 0x7ff0081578c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.941+0000 7ff00cdc1640 1 --2- 172.21.15.98:0/3359673494 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0080d09b0 0x7ff0080ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:03.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.941+0000 7ff00cdc1640 1 -- 172.21.15.98:0/3359673494 >> 172.21.15.98:0/3359673494 conn(0x7ff00809ce90 msgr2=0x7ff0080ced20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:03.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.941+0000 7ff00cdc1640 1 -- 172.21.15.98:0/3359673494 shutdown_connections 2024-05-09T10:51:03.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:03.941+0000 7ff00cdc1640 1 -- 172.21.15.98:0/3359673494 wait complete. 2024-05-09T10:51:03.945 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:51:04.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.052+0000 7f3c8159b640 1 Processor -- start 2024-05-09T10:51:04.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.052+0000 7f3c8159b640 1 -- start start 2024-05-09T10:51:04.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c8159b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3c7c15de40 0x7f3c7c0d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c8159b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c7c0cf250 0x7f3c7c0cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c8159b640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c7c15e220 con 0x7f3c7c0d0470 2024-05-09T10:51:04.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c8159b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c7c0d1790 con 0x7f3c7c15de40 2024-05-09T10:51:04.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c8159b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c0d19e0 con 0x7f3c7c0cf250 2024-05-09T10:51:04.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c7b7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f3c7c0d0470 0x7f3c7c0b6a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52746/0 (socket says 172.21.15.98:52746) 2024-05-09T10:51:04.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c7b7fe640 1 -- 172.21.15.98:0/1406218181 learned_addr learned my addr 172.21.15.98:0/1406218181 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:04.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c7affd640 1 --2- 172.21.15.98:0/1406218181 >> v2:172.21.15.105:3300/0 conn(0x7f3c7c15de40 0x7f3c7c0d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c73fff640 1 --2- 172.21.15.98:0/1406218181 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c7c0cf250 0x7f3c7c0cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c7a7fc640 1 -- 172.21.15.98:0/1406218181 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3287216295 0 0) 0x7f3c7c15e220 con 0x7f3c7c0d0470 2024-05-09T10:51:04.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c7a7fc640 1 -- 172.21.15.98:0/1406218181 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c68003520 con 0x7f3c7c0d0470 2024-05-09T10:51:04.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c7a7fc640 1 -- 172.21.15.98:0/1406218181 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 728916001 0 0) 0x7f3c68003520 con 0x7f3c7c0d0470 2024-05-09T10:51:04.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c7a7fc640 1 -- 172.21.15.98:0/1406218181 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c7c0cf250 msgr2=0x7f3c7c0cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:04.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c7a7fc640 1 --2- 172.21.15.98:0/1406218181 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c7c0cf250 0x7f3c7c0cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c7a7fc640 1 -- 172.21.15.98:0/1406218181 >> v2:172.21.15.105:3300/0 conn(0x7f3c7c15de40 msgr2=0x7f3c7c0d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:04.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c7a7fc640 1 --2- 172.21.15.98:0/1406218181 >> v2:172.21.15.105:3300/0 conn(0x7f3c7c15de40 0x7f3c7c0d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c7a7fc640 1 -- 172.21.15.98:0/1406218181 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7c0d1cc0 con 0x7f3c7c0d0470 2024-05-09T10:51:04.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c7a7fc640 1 -- 172.21.15.98:0/1406218181 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f3c740019f0 con 0x7f3c7c0d0470 2024-05-09T10:51:04.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c7a7fc640 1 -- 172.21.15.98:0/1406218181 >> v1:172.21.15.98:6789/0 conn(0x7f3c7c0d0470 legacy=0x7f3c7c0b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:04.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c7a7fc640 1 --2- 172.21.15.98:0/1406218181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c680039b0 0x7f3c68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c7a7fc640 1 --2- 172.21.15.98:0/1406218181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c68004800 0x7f3c68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c7a7fc640 1 --2- 172.21.15.98:0/1406218181 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c7c0cf250 0x7f3c680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c7a7fc640 1 -- 172.21.15.98:0/1406218181 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c0d1790 con 0x7f3c680039b0 2024-05-09T10:51:04.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c7a7fc640 1 -- 172.21.15.98:0/1406218181 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c0d19e0 con 0x7f3c68004800 2024-05-09T10:51:04.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.053+0000 7f3c7a7fc640 1 -- 172.21.15.98:0/1406218181 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6800b9c0 con 0x7f3c7c0cf250 2024-05-09T10:51:04.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.054+0000 7f3c7affd640 1 --2- 172.21.15.98:0/1406218181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c680039b0 0x7f3c68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.054+0000 7f3c73fff640 1 --2- 172.21.15.98:0/1406218181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c68004800 0x7f3c68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.054+0000 7f3c7b7fe640 1 --2- 172.21.15.98:0/1406218181 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c7c0cf250 0x7f3c680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.054+0000 7f3c7affd640 1 -- 172.21.15.98:0/1406218181 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c7c0cf250 msgr2=0x7f3c680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:04.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.054+0000 7f3c7affd640 1 --2- 172.21.15.98:0/1406218181 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c7c0cf250 0x7f3c680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.054+0000 7f3c7affd640 1 -- 172.21.15.98:0/1406218181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c68004800 msgr2=0x7f3c68004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:04.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.054+0000 7f3c7affd640 1 --2- 172.21.15.98:0/1406218181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c68004800 0x7f3c68004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.054+0000 7f3c7affd640 1 -- 172.21.15.98:0/1406218181 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c68003520 con 0x7f3c680039b0 2024-05-09T10:51:04.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.054+0000 7f3c73fff640 1 --2- 172.21.15.98:0/1406218181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c68004800 0x7f3c68004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:04.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.054+0000 7f3c7affd640 1 --2- 172.21.15.98:0/1406218181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c680039b0 0x7f3c68003dd0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009ac0 tx=0x7f3c68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893aa7362e874fea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:04.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.055+0000 7f3c7a7fc640 1 -- 172.21.15.98:0/1406218181 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c036070 con 0x7f3c680039b0 2024-05-09T10:51:04.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.055+0000 7f3c7a7fc640 1 -- 172.21.15.98:0/1406218181 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6c031400 con 0x7f3c680039b0 2024-05-09T10:51:04.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.055+0000 7f3c7a7fc640 1 -- 172.21.15.98:0/1406218181 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c009690 con 0x7f3c680039b0 2024-05-09T10:51:04.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.055+0000 7f3c8159b640 1 -- 172.21.15.98:0/1406218181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c680039b0 msgr2=0x7f3c68003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:04.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.055+0000 7f3c8159b640 1 --2- 172.21.15.98:0/1406218181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c680039b0 0x7f3c68003dd0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009ac0 tx=0x7f3c68005830 comp rx=0 tx=0).stop 2024-05-09T10:51:04.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.055+0000 7f3c7b7fe640 1 -- 172.21.15.98:0/1406218181 reap_dead start 2024-05-09T10:51:04.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.055+0000 7f3c8159b640 1 -- 172.21.15.98:0/1406218181 shutdown_connections 2024-05-09T10:51:04.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.055+0000 7f3c8159b640 1 -- 172.21.15.98:0/1406218181 >> 172.21.15.98:0/1406218181 conn(0x7f3c7c09ce90 msgr2=0x7f3c7c0d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:04.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.055+0000 7f3c8159b640 1 -- 172.21.15.98:0/1406218181 shutdown_connections 2024-05-09T10:51:04.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.055+0000 7f3c8159b640 1 -- 172.21.15.98:0/1406218181 wait complete. 2024-05-09T10:51:04.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.056+0000 7f3c8159b640 1 Processor -- start 2024-05-09T10:51:04.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.056+0000 7f3c8159b640 1 -- start start 2024-05-09T10:51:04.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.057+0000 7f3c8159b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c680039b0 0x7f3c7c14df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.057+0000 7f3c8159b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c7c0b43f0 0x7f3c7c0b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.057+0000 7f3c8159b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c7c0b0d30 0x7f3c7c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.057+0000 7f3c8159b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c0d4680 con 0x7f3c7c0b43f0 2024-05-09T10:51:04.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.057+0000 7f3c8159b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c0d2280 con 0x7f3c680039b0 2024-05-09T10:51:04.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.057+0000 7f3c8159b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c0d4800 con 0x7f3c7c0b0d30 2024-05-09T10:51:04.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.057+0000 7f3c73fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c7c0b43f0 0x7f3c7c0b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.057+0000 7f3c73fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c7c0b43f0 0x7f3c7c0b07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48838/0 (socket says 172.21.15.98:48838) 2024-05-09T10:51:04.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.057+0000 7f3c73fff640 1 -- 172.21.15.98:0/2571368193 learned_addr learned my addr 172.21.15.98:0/2571368193 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:04.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.057+0000 7f3c7b7fe640 1 --2- 172.21.15.98:0/2571368193 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c7c0b0d30 0x7f3c7c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.057+0000 7f3c7affd640 1 --2- 172.21.15.98:0/2571368193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c680039b0 0x7f3c7c14df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.058+0000 7f3c73fff640 1 -- 172.21.15.98:0/2571368193 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c7c0b0d30 msgr2=0x7f3c7c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:04.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.058+0000 7f3c73fff640 1 --2- 172.21.15.98:0/2571368193 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c7c0b0d30 0x7f3c7c0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.058+0000 7f3c73fff640 1 -- 172.21.15.98:0/2571368193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c680039b0 msgr2=0x7f3c7c14df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:04.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.058+0000 7f3c73fff640 1 --2- 172.21.15.98:0/2571368193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c680039b0 0x7f3c7c14df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.058+0000 7f3c73fff640 1 -- 172.21.15.98:0/2571368193 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7c0b11d0 con 0x7f3c7c0b43f0 2024-05-09T10:51:04.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.058+0000 7f3c7affd640 1 --2- 172.21.15.98:0/2571368193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c680039b0 0x7f3c7c14df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:04.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.058+0000 7f3c73fff640 1 --2- 172.21.15.98:0/2571368193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c7c0b43f0 0x7f3c7c0b07f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3c640116d0 tx=0x7f3c64011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:04.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.058+0000 7f3c737fe640 1 -- 172.21.15.98:0/2571368193 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6400f9c0 con 0x7f3c7c0b43f0 2024-05-09T10:51:04.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.058+0000 7f3c737fe640 1 -- 172.21.15.98:0/2571368193 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6400fb60 con 0x7f3c7c0b43f0 2024-05-09T10:51:04.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.058+0000 7f3c737fe640 1 -- 172.21.15.98:0/2571368193 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c64015460 con 0x7f3c7c0b43f0 2024-05-09T10:51:04.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.058+0000 7f3c8159b640 1 -- 172.21.15.98:0/2571368193 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c7c0b14b0 con 0x7f3c7c0b43f0 2024-05-09T10:51:04.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.058+0000 7f3c8159b640 1 -- 172.21.15.98:0/2571368193 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7c0b1a70 con 0x7f3c7c0b43f0 2024-05-09T10:51:04.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.059+0000 7f3c8159b640 1 -- 172.21.15.98:0/2571368193 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c7c069cb0 con 0x7f3c7c0b43f0 2024-05-09T10:51:04.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.059+0000 7f3c737fe640 1 -- 172.21.15.98:0/2571368193 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3c64015600 con 0x7f3c7c0b43f0 2024-05-09T10:51:04.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.060+0000 7f3c737fe640 1 --2- 172.21.15.98:0/2571368193 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3c5c06b4d0 0x7f3c5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.060+0000 7f3c737fe640 1 -- 172.21.15.98:0/2571368193 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3c6408a910 con 0x7f3c7c0b43f0 2024-05-09T10:51:04.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.060+0000 7f3c7affd640 1 --2- 172.21.15.98:0/2571368193 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3c5c06b4d0 0x7f3c5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.060+0000 7f3c7affd640 1 --2- 172.21.15.98:0/2571368193 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3c5c06b4d0 0x7f3c5c06d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c033000 tx=0x7f3c6c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:04.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.064+0000 7f3c737fe640 1 -- 172.21.15.98:0/2571368193 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3c64019030 con 0x7f3c7c0b43f0 2024-05-09T10:51:04.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.234+0000 7f3c8159b640 1 -- 172.21.15.98:0/2571368193 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c7c0b1e70 con 0x7f3c7c0b43f0 2024-05-09T10:51:04.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.234+0000 7f3c737fe640 1 -- 172.21.15.98:0/2571368193 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+12891 (secure 0 0 0) 0x7f3c64059ae0 con 0x7f3c7c0b43f0 2024-05-09T10:51:04.228 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:04.228 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":27,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:51:01.382184+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:51:04.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.236+0000 7f3c8159b640 1 -- 172.21.15.98:0/2571368193 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3c5c06b4d0 msgr2=0x7f3c5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:04.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.236+0000 7f3c8159b640 1 --2- 172.21.15.98:0/2571368193 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3c5c06b4d0 0x7f3c5c06d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c033000 tx=0x7f3c6c008030 comp rx=0 tx=0).stop 2024-05-09T10:51:04.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.236+0000 7f3c8159b640 1 -- 172.21.15.98:0/2571368193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c7c0b43f0 msgr2=0x7f3c7c0b07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:04.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.236+0000 7f3c8159b640 1 --2- 172.21.15.98:0/2571368193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c7c0b43f0 0x7f3c7c0b07f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3c640116d0 tx=0x7f3c64011b90 comp rx=0 tx=0).stop 2024-05-09T10:51:04.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.237+0000 7f3c8159b640 1 -- 172.21.15.98:0/2571368193 shutdown_connections 2024-05-09T10:51:04.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.237+0000 7f3c8159b640 1 --2- 172.21.15.98:0/2571368193 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3c5c06b4d0 0x7f3c5c06d990 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.237+0000 7f3c8159b640 1 --2- 172.21.15.98:0/2571368193 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3c7c0b0d30 0x7f3c7c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.237+0000 7f3c8159b640 1 --2- 172.21.15.98:0/2571368193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c7c0b43f0 0x7f3c7c0b07f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.237+0000 7f3c8159b640 1 --2- 172.21.15.98:0/2571368193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3c680039b0 0x7f3c7c14df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.237+0000 7f3c8159b640 1 -- 172.21.15.98:0/2571368193 >> 172.21.15.98:0/2571368193 conn(0x7f3c7c09ce90 msgr2=0x7f3c7c0ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:04.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.237+0000 7f3c8159b640 1 -- 172.21.15.98:0/2571368193 shutdown_connections 2024-05-09T10:51:04.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.237+0000 7f3c8159b640 1 -- 172.21.15.98:0/2571368193 wait complete. 2024-05-09T10:51:04.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T10:50:07.396423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T10:51:04.242 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T10:51:04.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.350+0000 7f92cbb7e640 1 Processor -- start 2024-05-09T10:51:04.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.350+0000 7f92cbb7e640 1 -- start start 2024-05-09T10:51:04.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.351+0000 7f92cbb7e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92c40b4b70 0x7f92c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.351+0000 7f92cbb7e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f92c40b2bf0 0x7f92c40b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.351+0000 7f92cbb7e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c406ded0 con 0x7f92c40cf900 2024-05-09T10:51:04.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.351+0000 7f92cbb7e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c415f320 con 0x7f92c40b2bf0 2024-05-09T10:51:04.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.351+0000 7f92cbb7e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c415e310 con 0x7f92c40b4b70 2024-05-09T10:51:04.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.351+0000 7f92ca11b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f92c40b2bf0 0x7f92c40b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.351+0000 7f92c991a640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f92c40cf900 0x7f92c40cfce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52756/0 (socket says 172.21.15.98:52756) 2024-05-09T10:51:04.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.351+0000 7f92c991a640 1 -- 172.21.15.98:0/557475015 learned_addr learned my addr 172.21.15.98:0/557475015 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:04.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.352+0000 7f92c9119640 1 --2- 172.21.15.98:0/557475015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92c40b4b70 0x7f92c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.352+0000 7f92ca11b640 1 -- 172.21.15.98:0/557475015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92c40b4b70 msgr2=0x7f92c406d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:04.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.352+0000 7f92ca11b640 1 --2- 172.21.15.98:0/557475015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92c40b4b70 0x7f92c406d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.352+0000 7f92c8918640 1 -- 172.21.15.98:0/557475015 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3977093275 0 0) 0x7f92c406ded0 con 0x7f92c40cf900 2024-05-09T10:51:04.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.352+0000 7f92ca11b640 1 -- 172.21.15.98:0/557475015 >> v1:172.21.15.98:6789/0 conn(0x7f92c40cf900 legacy=0x7f92c40cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:04.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.352+0000 7f92ca11b640 1 -- 172.21.15.98:0/557475015 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c415e560 con 0x7f92c40b2bf0 2024-05-09T10:51:04.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.352+0000 7f92c9119640 1 --2- 172.21.15.98:0/557475015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92c40b4b70 0x7f92c406d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:04.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.352+0000 7f92ca11b640 1 --2- 172.21.15.98:0/557475015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c40b2bf0 0x7f92c40b3080 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0097a0 tx=0x7f92bc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8758c4358fbb76 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:04.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.352+0000 7f92c8918640 1 -- 172.21.15.98:0/557475015 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc037070 con 0x7f92c40b2bf0 2024-05-09T10:51:04.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.353+0000 7f92c8918640 1 -- 172.21.15.98:0/557475015 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92bc032440 con 0x7f92c40b2bf0 2024-05-09T10:51:04.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.353+0000 7f92c8918640 1 -- 172.21.15.98:0/557475015 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc032810 con 0x7f92c40b2bf0 2024-05-09T10:51:04.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.353+0000 7f92cbb7e640 1 -- 172.21.15.98:0/557475015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c40b2bf0 msgr2=0x7f92c40b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:04.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.353+0000 7f92cbb7e640 1 --2- 172.21.15.98:0/557475015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c40b2bf0 0x7f92c40b3080 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0097a0 tx=0x7f92bc02a180 comp rx=0 tx=0).stop 2024-05-09T10:51:04.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.353+0000 7f92cbb7e640 1 -- 172.21.15.98:0/557475015 shutdown_connections 2024-05-09T10:51:04.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.353+0000 7f92cbb7e640 1 --2- 172.21.15.98:0/557475015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c40b2bf0 0x7f92c40b3080 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.353+0000 7f92cbb7e640 1 --2- 172.21.15.98:0/557475015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92c40b4b70 0x7f92c406d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.353+0000 7f92cbb7e640 1 -- 172.21.15.98:0/557475015 >> 172.21.15.98:0/557475015 conn(0x7f92c406abe0 msgr2=0x7f92c406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:04.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.353+0000 7f92cbb7e640 1 -- 172.21.15.98:0/557475015 shutdown_connections 2024-05-09T10:51:04.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.353+0000 7f92cbb7e640 1 -- 172.21.15.98:0/557475015 wait complete. 2024-05-09T10:51:04.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.354+0000 7f92cbb7e640 1 Processor -- start 2024-05-09T10:51:04.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.354+0000 7f92cbb7e640 1 -- start start 2024-05-09T10:51:04.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.355+0000 7f92cbb7e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92c40b2bf0 0x7f92c40ced70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.355+0000 7f92cbb7e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c40b4b70 0x7f92c41f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.355+0000 7f92cbb7e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92c40cf900 0x7f92c41f7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.355+0000 7f92cbb7e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c415ee00 con 0x7f92c40b4b70 2024-05-09T10:51:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.355+0000 7f92cbb7e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c415eb20 con 0x7f92c40cf900 2024-05-09T10:51:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.355+0000 7f92cbb7e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c415bba0 con 0x7f92c40b2bf0 2024-05-09T10:51:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.355+0000 7f92c9119640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c40b4b70 0x7f92c41f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.355+0000 7f92c9119640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c40b4b70 0x7f92c41f7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48868/0 (socket says 172.21.15.98:48868) 2024-05-09T10:51:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.355+0000 7f92c9119640 1 -- 172.21.15.98:0/124425240 learned_addr learned my addr 172.21.15.98:0/124425240 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.355+0000 7f92c9119640 1 -- 172.21.15.98:0/124425240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92c40b2bf0 msgr2=0x7f92c40ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.355+0000 7f92c9119640 1 --2- 172.21.15.98:0/124425240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92c40b2bf0 0x7f92c40ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.355+0000 7f92c9119640 1 -- 172.21.15.98:0/124425240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92c40cf900 msgr2=0x7f92c41f7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.355+0000 7f92c9119640 1 --2- 172.21.15.98:0/124425240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92c40cf900 0x7f92c41f7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.355+0000 7f92c9119640 1 -- 172.21.15.98:0/124425240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c40cf2b0 con 0x7f92c40b4b70 2024-05-09T10:51:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.356+0000 7f92c9119640 1 --2- 172.21.15.98:0/124425240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c40b4b70 0x7f92c41f7580 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f92b800d7d0 tx=0x7f92b800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.356+0000 7f92c27fc640 1 -- 172.21.15.98:0/124425240 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b8004030 con 0x7f92c40b4b70 2024-05-09T10:51:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.356+0000 7f92c27fc640 1 -- 172.21.15.98:0/124425240 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b80041d0 con 0x7f92c40b4b70 2024-05-09T10:51:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.356+0000 7f92cbb7e640 1 -- 172.21.15.98:0/124425240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c41572c0 con 0x7f92c40b4b70 2024-05-09T10:51:04.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.356+0000 7f92c27fc640 1 -- 172.21.15.98:0/124425240 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b80096a0 con 0x7f92c40b4b70 2024-05-09T10:51:04.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.356+0000 7f92cbb7e640 1 -- 172.21.15.98:0/124425240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c4157880 con 0x7f92c40b4b70 2024-05-09T10:51:04.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.357+0000 7f92cbb7e640 1 -- 172.21.15.98:0/124425240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c415abd0 con 0x7f92c40b4b70 2024-05-09T10:51:04.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.357+0000 7f92c27fc640 1 -- 172.21.15.98:0/124425240 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f92b8009840 con 0x7f92c40b4b70 2024-05-09T10:51:04.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.358+0000 7f92c27fc640 1 --2- 172.21.15.98:0/124425240 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f92b006b4d0 0x7f92b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.358+0000 7f92c27fc640 1 -- 172.21.15.98:0/124425240 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f92b808a160 con 0x7f92c40b4b70 2024-05-09T10:51:04.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.358+0000 7f92c991a640 1 --2- 172.21.15.98:0/124425240 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f92b006b4d0 0x7f92b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.358+0000 7f92c991a640 1 --2- 172.21.15.98:0/124425240 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f92b006b4d0 0x7f92b006d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f92b4001560 tx=0x7f92b4007ca0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:04.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.362+0000 7f92c27fc640 1 -- 172.21.15.98:0/124425240 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f92b8059330 con 0x7f92c40b4b70 2024-05-09T10:51:04.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.534+0000 7f92cbb7e640 1 -- 172.21.15.98:0/124425240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f92c404f000 con 0x7f92c40b4b70 2024-05-09T10:51:04.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.534+0000 7f92c27fc640 1 -- 172.21.15.98:0/124425240 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f92b8058c30 con 0x7f92c40b4b70 2024-05-09T10:51:04.527 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-05-09T10:51:04.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.536+0000 7f92cbb7e640 1 -- 172.21.15.98:0/124425240 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f92b006b4d0 msgr2=0x7f92b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:04.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.536+0000 7f92cbb7e640 1 --2- 172.21.15.98:0/124425240 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f92b006b4d0 0x7f92b006d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f92b4001560 tx=0x7f92b4007ca0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.536+0000 7f92cbb7e640 1 -- 172.21.15.98:0/124425240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c40b4b70 msgr2=0x7f92c41f7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.536+0000 7f92cbb7e640 1 --2- 172.21.15.98:0/124425240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c40b4b70 0x7f92c41f7580 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f92b800d7d0 tx=0x7f92b800dc90 comp rx=0 tx=0).stop 2024-05-09T10:51:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.536+0000 7f92cbb7e640 1 -- 172.21.15.98:0/124425240 shutdown_connections 2024-05-09T10:51:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.536+0000 7f92cbb7e640 1 --2- 172.21.15.98:0/124425240 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f92b006b4d0 0x7f92b006d990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.536+0000 7f92cbb7e640 1 --2- 172.21.15.98:0/124425240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92c40cf900 0x7f92c41f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.536+0000 7f92cbb7e640 1 --2- 172.21.15.98:0/124425240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92c40b4b70 0x7f92c41f7580 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.536+0000 7f92cbb7e640 1 --2- 172.21.15.98:0/124425240 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92c40b2bf0 0x7f92c40ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.536+0000 7f92cbb7e640 1 -- 172.21.15.98:0/124425240 >> 172.21.15.98:0/124425240 conn(0x7f92c406abe0 msgr2=0x7f92c4159000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.536+0000 7f92cbb7e640 1 -- 172.21.15.98:0/124425240 shutdown_connections 2024-05-09T10:51:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.536+0000 7f92cbb7e640 1 -- 172.21.15.98:0/124425240 wait complete. 2024-05-09T10:51:04.540 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-09T10:51:04.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.647+0000 7fbb7de1f640 1 Processor -- start 2024-05-09T10:51:04.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.647+0000 7fbb7de1f640 1 -- start start 2024-05-09T10:51:04.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.648+0000 7fbb7de1f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbb7806ae50 0x7fbb780cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.648+0000 7fbb7de1f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbb7806b230 0x7fbb7815d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.648+0000 7fbb7de1f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb781627c0 con 0x7fbb7806d780 2024-05-09T10:51:04.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.648+0000 7fbb7de1f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb780cdde0 con 0x7fbb7806ae50 2024-05-09T10:51:04.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.648+0000 7fbb7de1f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb78161a00 con 0x7fbb7806b230 2024-05-09T10:51:04.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.648+0000 7fbb777fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbb7806ae50 0x7fbb780cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.648+0000 7fbb777fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbb7806ae50 0x7fbb780cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48870/0 (socket says 172.21.15.98:48870) 2024-05-09T10:51:04.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.648+0000 7fbb777fe640 1 -- 172.21.15.98:0/997158675 learned_addr learned my addr 172.21.15.98:0/997158675 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:04.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.648+0000 7fbb76ffd640 1 --2- 172.21.15.98:0/997158675 >> v2:172.21.15.105:3300/0 conn(0x7fbb7806b230 0x7fbb7815d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.648+0000 7fbb777fe640 1 -- 172.21.15.98:0/997158675 >> v2:172.21.15.105:3300/0 conn(0x7fbb7806b230 msgr2=0x7fbb7815d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:04.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.648+0000 7fbb777fe640 1 --2- 172.21.15.98:0/997158675 >> v2:172.21.15.105:3300/0 conn(0x7fbb7806b230 0x7fbb7815d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.648+0000 7fbb777fe640 1 -- 172.21.15.98:0/997158675 >> v1:172.21.15.105:6789/0 conn(0x7fbb7806d780 legacy=0x7fbb780d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:04.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.648+0000 7fbb777fe640 1 -- 172.21.15.98:0/997158675 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb78161ce0 con 0x7fbb7806ae50 2024-05-09T10:51:04.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.649+0000 7fbb777fe640 1 --2- 172.21.15.98:0/997158675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb7806ae50 0x7fbb780cd8a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbb7806b6a0 tx=0x7fbb6802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3ade5834ccf23c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:04.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.649+0000 7fbb767fc640 1 -- 172.21.15.98:0/997158675 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb68037070 con 0x7fbb7806ae50 2024-05-09T10:51:04.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.649+0000 7fbb767fc640 1 -- 172.21.15.98:0/997158675 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb68032440 con 0x7fbb7806ae50 2024-05-09T10:51:04.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.649+0000 7fbb767fc640 1 -- 172.21.15.98:0/997158675 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb68032810 con 0x7fbb7806ae50 2024-05-09T10:51:04.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.650+0000 7fbb7de1f640 1 -- 172.21.15.98:0/997158675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb7806ae50 msgr2=0x7fbb780cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:04.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.650+0000 7fbb7de1f640 1 --2- 172.21.15.98:0/997158675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb7806ae50 0x7fbb780cd8a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbb7806b6a0 tx=0x7fbb6802a1e0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.650+0000 7fbb7de1f640 1 -- 172.21.15.98:0/997158675 shutdown_connections 2024-05-09T10:51:04.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.650+0000 7fbb7de1f640 1 --2- 172.21.15.98:0/997158675 >> v2:172.21.15.105:3300/0 conn(0x7fbb7806b230 0x7fbb7815d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.650+0000 7fbb7de1f640 1 --2- 172.21.15.98:0/997158675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb7806ae50 0x7fbb780cd8a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.650+0000 7fbb7de1f640 1 -- 172.21.15.98:0/997158675 >> 172.21.15.98:0/997158675 conn(0x7fbb78066f20 msgr2=0x7fbb7806bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:04.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.650+0000 7fbb7de1f640 1 -- 172.21.15.98:0/997158675 shutdown_connections 2024-05-09T10:51:04.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.650+0000 7fbb7de1f640 1 -- 172.21.15.98:0/997158675 wait complete. 2024-05-09T10:51:04.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.651+0000 7fbb7de1f640 1 Processor -- start 2024-05-09T10:51:04.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.651+0000 7fbb7de1f640 1 -- start start 2024-05-09T10:51:04.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.652+0000 7fbb7de1f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb7806ae50 0x7fbb780b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.652+0000 7fbb7de1f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb7806b230 0x7fbb780b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.652+0000 7fbb7de1f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb7806d780 0x7fbb78148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.652+0000 7fbb7de1f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb780723e0 con 0x7fbb7806b230 2024-05-09T10:51:04.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.652+0000 7fbb7de1f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb781622a0 con 0x7fbb7806ae50 2024-05-09T10:51:04.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.652+0000 7fbb7de1f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7815dcd0 con 0x7fbb7806d780 2024-05-09T10:51:04.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.652+0000 7fbb76ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb7806b230 0x7fbb780b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.652+0000 7fbb76ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb7806b230 0x7fbb780b4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48886/0 (socket says 172.21.15.98:48886) 2024-05-09T10:51:04.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.652+0000 7fbb76ffd640 1 -- 172.21.15.98:0/1317932562 learned_addr learned my addr 172.21.15.98:0/1317932562 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:04.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.652+0000 7fbb777fe640 1 --2- 172.21.15.98:0/1317932562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb7806ae50 0x7fbb780b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.652+0000 7fbb76ffd640 1 -- 172.21.15.98:0/1317932562 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb7806d780 msgr2=0x7fbb78148e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:04.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.652+0000 7fbb77fff640 1 --2- 172.21.15.98:0/1317932562 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb7806d780 0x7fbb78148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.652+0000 7fbb76ffd640 1 --2- 172.21.15.98:0/1317932562 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb7806d780 0x7fbb78148e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.652+0000 7fbb76ffd640 1 -- 172.21.15.98:0/1317932562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb7806ae50 msgr2=0x7fbb780b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:04.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.652+0000 7fbb76ffd640 1 --2- 172.21.15.98:0/1317932562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb7806ae50 0x7fbb780b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.652+0000 7fbb76ffd640 1 -- 172.21.15.98:0/1317932562 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb780ae820 con 0x7fbb7806b230 2024-05-09T10:51:04.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.652+0000 7fbb777fe640 1 --2- 172.21.15.98:0/1317932562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb7806ae50 0x7fbb780b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:04.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.652+0000 7fbb77fff640 1 --2- 172.21.15.98:0/1317932562 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb7806d780 0x7fbb78148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:04.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.652+0000 7fbb76ffd640 1 --2- 172.21.15.98:0/1317932562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb7806b230 0x7fbb780b4070 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009db0 tx=0x7fbb6c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:04.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.653+0000 7fbb5ffff640 1 -- 172.21.15.98:0/1317932562 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c018020 con 0x7fbb7806b230 2024-05-09T10:51:04.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.653+0000 7fbb5ffff640 1 -- 172.21.15.98:0/1317932562 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6c014480 con 0x7fbb7806b230 2024-05-09T10:51:04.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.653+0000 7fbb7de1f640 1 -- 172.21.15.98:0/1317932562 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb780b35f0 con 0x7fbb7806b230 2024-05-09T10:51:04.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.653+0000 7fbb5ffff640 1 -- 172.21.15.98:0/1317932562 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c014850 con 0x7fbb7806b230 2024-05-09T10:51:04.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.653+0000 7fbb7de1f640 1 -- 172.21.15.98:0/1317932562 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb780b28b0 con 0x7fbb7806b230 2024-05-09T10:51:04.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.654+0000 7fbb7de1f640 1 -- 172.21.15.98:0/1317932562 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb78069cb0 con 0x7fbb7806b230 2024-05-09T10:51:04.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.654+0000 7fbb5ffff640 1 -- 172.21.15.98:0/1317932562 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb6c005d10 con 0x7fbb7806b230 2024-05-09T10:51:04.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.654+0000 7fbb5ffff640 1 --2- 172.21.15.98:0/1317932562 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fbb6406b4d0 0x7fbb6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.654+0000 7fbb5ffff640 1 -- 172.21.15.98:0/1317932562 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbb6c08a920 con 0x7fbb7806b230 2024-05-09T10:51:04.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.655+0000 7fbb777fe640 1 --2- 172.21.15.98:0/1317932562 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fbb6406b4d0 0x7fbb6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.655+0000 7fbb777fe640 1 --2- 172.21.15.98:0/1317932562 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fbb6406b4d0 0x7fbb6406d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbb68034000 tx=0x7fbb6802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:04.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.658+0000 7fbb5ffff640 1 -- 172.21.15.98:0/1317932562 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb6c022070 con 0x7fbb7806b230 2024-05-09T10:51:04.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.830+0000 7fbb7de1f640 1 -- 172.21.15.98:0/1317932562 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbb780d05a0 con 0x7fbb7806b230 2024-05-09T10:51:04.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.830+0000 7fbb5ffff640 1 -- 172.21.15.98:0/1317932562 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v27) v1 ==== 84+0+65 (secure 0 0 0) 0x7fbb6c010040 con 0x7fbb7806b230 2024-05-09T10:51:04.824 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:04.824 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-05-09T10:51:04.824 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:04.824 INFO:teuthology.orchestra.run.smithi098.stdout: "poolnum": 1, 2024-05-09T10:51:04.824 INFO:teuthology.orchestra.run.smithi098.stdout: "poolname": ".mgr" 2024-05-09T10:51:04.824 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:04.824 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-05-09T10:51:04.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.832+0000 7fbb7de1f640 1 -- 172.21.15.98:0/1317932562 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fbb6406b4d0 msgr2=0x7fbb6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:04.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.832+0000 7fbb7de1f640 1 --2- 172.21.15.98:0/1317932562 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fbb6406b4d0 0x7fbb6406d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbb68034000 tx=0x7fbb6802e000 comp rx=0 tx=0).stop 2024-05-09T10:51:04.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.832+0000 7fbb7de1f640 1 -- 172.21.15.98:0/1317932562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb7806b230 msgr2=0x7fbb780b4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:04.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.832+0000 7fbb7de1f640 1 --2- 172.21.15.98:0/1317932562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb7806b230 0x7fbb780b4070 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009db0 tx=0x7fbb6c00c490 comp rx=0 tx=0).stop 2024-05-09T10:51:04.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.832+0000 7fbb7de1f640 1 -- 172.21.15.98:0/1317932562 shutdown_connections 2024-05-09T10:51:04.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.832+0000 7fbb7de1f640 1 --2- 172.21.15.98:0/1317932562 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fbb6406b4d0 0x7fbb6406d990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.832+0000 7fbb7de1f640 1 --2- 172.21.15.98:0/1317932562 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbb7806d780 0x7fbb78148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.832+0000 7fbb7de1f640 1 --2- 172.21.15.98:0/1317932562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbb7806b230 0x7fbb780b4070 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.832+0000 7fbb7de1f640 1 --2- 172.21.15.98:0/1317932562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb7806ae50 0x7fbb780b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.832+0000 7fbb7de1f640 1 -- 172.21.15.98:0/1317932562 >> 172.21.15.98:0/1317932562 conn(0x7fbb78066f20 msgr2=0x7fbb780cf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:04.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.833+0000 7fbb7de1f640 1 -- 172.21.15.98:0/1317932562 shutdown_connections 2024-05-09T10:51:04.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.833+0000 7fbb7de1f640 1 -- 172.21.15.98:0/1317932562 wait complete. 2024-05-09T10:51:04.837 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T10:51:04.837 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T10:51:04.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.946+0000 7fd9b4130640 1 Processor -- start 2024-05-09T10:51:04.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.947+0000 7fd9b4130640 1 -- start start 2024-05-09T10:51:04.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.947+0000 7fd9b4130640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9ac161450 0x7fd9ac0d4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.947+0000 7fd9b4130640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd9ac0b2a30 0x7fd9ac0d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.947+0000 7fd9b4130640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd9ac0d2650 0x7fd9ac0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.947+0000 7fd9b4130640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9ac0b0760 con 0x7fd9ac0b2a30 2024-05-09T10:51:04.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.947+0000 7fd9b4130640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9ac161830 con 0x7fd9ac0d2650 2024-05-09T10:51:04.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.947+0000 7fd9b4130640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac0cf2f0 con 0x7fd9ac161450 2024-05-09T10:51:04.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.947+0000 7fd9b16cb640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd9ac0b2a30 0x7fd9ac0d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.948+0000 7fd9b16cb640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd9ac0b2a30 0x7fd9ac0d2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48900/0 (socket says 172.21.15.98:48900) 2024-05-09T10:51:04.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.948+0000 7fd9b16cb640 1 -- 172.21.15.98:0/560991256 learned_addr learned my addr 172.21.15.98:0/560991256 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:04.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.948+0000 7fd9b1ecc640 1 --2- 172.21.15.98:0/560991256 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9ac161450 0x7fd9ac0d4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.948+0000 7fd9b26cd640 1 --2- 172.21.15.98:0/560991256 >> v2:172.21.15.105:3300/0 conn(0x7fd9ac0d2650 0x7fd9ac0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.948+0000 7fd9b16cb640 1 -- 172.21.15.98:0/560991256 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9ac161450 msgr2=0x7fd9ac0d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:04.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.948+0000 7fd9b16cb640 1 --2- 172.21.15.98:0/560991256 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9ac161450 0x7fd9ac0d4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.948+0000 7fd9b16cb640 1 -- 172.21.15.98:0/560991256 >> v2:172.21.15.105:3300/0 conn(0x7fd9ac0d2650 msgr2=0x7fd9ac0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:04.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.948+0000 7fd9b16cb640 1 --2- 172.21.15.98:0/560991256 >> v2:172.21.15.105:3300/0 conn(0x7fd9ac0d2650 0x7fd9ac0d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.948+0000 7fd9b16cb640 1 -- 172.21.15.98:0/560991256 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ac0cf5d0 con 0x7fd9ac0b2a30 2024-05-09T10:51:04.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.949+0000 7fd9b16cb640 1 --2- 172.21.15.98:0/560991256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9ac0b2a30 0x7fd9ac0d2110 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0a1ab0 tx=0x7fd9a002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c188326aa333cbf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:04.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.949+0000 7fd9b0eca640 1 -- 172.21.15.98:0/560991256 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0037070 con 0x7fd9ac0b2a30 2024-05-09T10:51:04.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.949+0000 7fd9b0eca640 1 -- 172.21.15.98:0/560991256 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a0032440 con 0x7fd9ac0b2a30 2024-05-09T10:51:04.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.949+0000 7fd9b0eca640 1 -- 172.21.15.98:0/560991256 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0032810 con 0x7fd9ac0b2a30 2024-05-09T10:51:04.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.949+0000 7fd9b4130640 1 -- 172.21.15.98:0/560991256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9ac0b2a30 msgr2=0x7fd9ac0d2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:04.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.949+0000 7fd9b4130640 1 --2- 172.21.15.98:0/560991256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9ac0b2a30 0x7fd9ac0d2110 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0a1ab0 tx=0x7fd9a002a140 comp rx=0 tx=0).stop 2024-05-09T10:51:04.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.949+0000 7fd9b4130640 1 -- 172.21.15.98:0/560991256 shutdown_connections 2024-05-09T10:51:04.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.949+0000 7fd9b4130640 1 --2- 172.21.15.98:0/560991256 >> v2:172.21.15.105:3300/0 conn(0x7fd9ac0d2650 0x7fd9ac0d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.949+0000 7fd9b4130640 1 --2- 172.21.15.98:0/560991256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9ac0b2a30 0x7fd9ac0d2110 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.949+0000 7fd9b4130640 1 --2- 172.21.15.98:0/560991256 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9ac161450 0x7fd9ac0d4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.949+0000 7fd9b4130640 1 -- 172.21.15.98:0/560991256 >> 172.21.15.98:0/560991256 conn(0x7fd9ac0d3b30 msgr2=0x7fd9ac0d3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:04.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.949+0000 7fd9b4130640 1 -- 172.21.15.98:0/560991256 shutdown_connections 2024-05-09T10:51:04.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.949+0000 7fd9b4130640 1 -- 172.21.15.98:0/560991256 wait complete. 2024-05-09T10:51:04.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.951+0000 7fd9b4130640 1 Processor -- start 2024-05-09T10:51:04.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.951+0000 7fd9b4130640 1 -- start start 2024-05-09T10:51:04.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.951+0000 7fd9b4130640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9ac0b2a30 0x7fd9ac15f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.951+0000 7fd9b4130640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9ac0d2650 0x7fd9ac0b6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.951+0000 7fd9b4130640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9ac161450 0x7fd9ac0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.951+0000 7fd9b4130640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac0d06a0 con 0x7fd9ac0d2650 2024-05-09T10:51:04.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.951+0000 7fd9b4130640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac0b09b0 con 0x7fd9ac161450 2024-05-09T10:51:04.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.951+0000 7fd9b4130640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac0d0820 con 0x7fd9ac0b2a30 2024-05-09T10:51:04.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.951+0000 7fd9b16cb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9ac0d2650 0x7fd9ac0b6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.951+0000 7fd9b16cb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9ac0d2650 0x7fd9ac0b6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48904/0 (socket says 172.21.15.98:48904) 2024-05-09T10:51:04.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.951+0000 7fd9b16cb640 1 -- 172.21.15.98:0/1882479581 learned_addr learned my addr 172.21.15.98:0/1882479581 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:04.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.951+0000 7fd9b1ecc640 1 --2- 172.21.15.98:0/1882479581 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9ac0b2a30 0x7fd9ac15f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.951+0000 7fd9b26cd640 1 --2- 172.21.15.98:0/1882479581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9ac161450 0x7fd9ac0b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.952+0000 7fd9b16cb640 1 -- 172.21.15.98:0/1882479581 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9ac0b2a30 msgr2=0x7fd9ac15f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:04.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.952+0000 7fd9b16cb640 1 --2- 172.21.15.98:0/1882479581 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9ac0b2a30 0x7fd9ac15f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.952+0000 7fd9b16cb640 1 -- 172.21.15.98:0/1882479581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9ac161450 msgr2=0x7fd9ac0b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:04.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.952+0000 7fd9b16cb640 1 --2- 172.21.15.98:0/1882479581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9ac161450 0x7fd9ac0b7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:04.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.952+0000 7fd9b16cb640 1 -- 172.21.15.98:0/1882479581 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ac0b67b0 con 0x7fd9ac0d2650 2024-05-09T10:51:04.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.952+0000 7fd9b26cd640 1 --2- 172.21.15.98:0/1882479581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9ac161450 0x7fd9ac0b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:04.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.952+0000 7fd9b16cb640 1 --2- 172.21.15.98:0/1882479581 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9ac0d2650 0x7fd9ac0b6040 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd9a002a180 tx=0x7fd9a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:04.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.952+0000 7fd9aa7fc640 1 -- 172.21.15.98:0/1882479581 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0041020 con 0x7fd9ac0d2650 2024-05-09T10:51:04.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.952+0000 7fd9aa7fc640 1 -- 172.21.15.98:0/1882479581 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a003d620 con 0x7fd9ac0d2650 2024-05-09T10:51:04.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.952+0000 7fd9aa7fc640 1 -- 172.21.15.98:0/1882479581 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0037040 con 0x7fd9ac0d2650 2024-05-09T10:51:04.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.952+0000 7fd9b4130640 1 -- 172.21.15.98:0/1882479581 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9ac0b6a90 con 0x7fd9ac0d2650 2024-05-09T10:51:04.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.952+0000 7fd9b4130640 1 -- 172.21.15.98:0/1882479581 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9ac0b7050 con 0x7fd9ac0d2650 2024-05-09T10:51:04.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.953+0000 7fd9b4130640 1 -- 172.21.15.98:0/1882479581 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9ac0a1ae0 con 0x7fd9ac0d2650 2024-05-09T10:51:04.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.954+0000 7fd9aa7fc640 1 -- 172.21.15.98:0/1882479581 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd9a0044020 con 0x7fd9ac0d2650 2024-05-09T10:51:04.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.954+0000 7fd9aa7fc640 1 --2- 172.21.15.98:0/1882479581 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd99806b4d0 0x7fd99806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:04.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.954+0000 7fd9aa7fc640 1 -- 172.21.15.98:0/1882479581 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd9a00aa580 con 0x7fd9ac0d2650 2024-05-09T10:51:04.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.954+0000 7fd9b1ecc640 1 --2- 172.21.15.98:0/1882479581 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd99806b4d0 0x7fd99806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:04.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.954+0000 7fd9b1ecc640 1 --2- 172.21.15.98:0/1882479581 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd99806b4d0 0x7fd99806d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd99c011340 tx=0x7fd99c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:04.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:04.958+0000 7fd9aa7fc640 1 -- 172.21.15.98:0/1882479581 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd9a004b030 con 0x7fd9ac0d2650 2024-05-09T10:51:05.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.131+0000 7fd9b4130640 1 -- 172.21.15.98:0/1882479581 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd9ac0b27f0 con 0x7fd9ac0d2650 2024-05-09T10:51:05.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.381+0000 7fd9aa7fc640 1 -- 172.21.15.98:0/1882479581 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v28) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd9a00796d0 con 0x7fd9ac0d2650 2024-05-09T10:51:05.374 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' created 2024-05-09T10:51:05.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.383+0000 7fd9b4130640 1 -- 172.21.15.98:0/1882479581 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd99806b4d0 msgr2=0x7fd99806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:05.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.383+0000 7fd9b4130640 1 --2- 172.21.15.98:0/1882479581 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd99806b4d0 0x7fd99806d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd99c011340 tx=0x7fd99c00b040 comp rx=0 tx=0).stop 2024-05-09T10:51:05.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.383+0000 7fd9b4130640 1 -- 172.21.15.98:0/1882479581 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9ac0d2650 msgr2=0x7fd9ac0b6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:05.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.383+0000 7fd9b4130640 1 --2- 172.21.15.98:0/1882479581 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9ac0d2650 0x7fd9ac0b6040 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd9a002a180 tx=0x7fd9a00046c0 comp rx=0 tx=0).stop 2024-05-09T10:51:05.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.383+0000 7fd9b4130640 1 -- 172.21.15.98:0/1882479581 shutdown_connections 2024-05-09T10:51:05.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.383+0000 7fd9b4130640 1 --2- 172.21.15.98:0/1882479581 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd99806b4d0 0x7fd99806d990 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:05.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.383+0000 7fd9b4130640 1 --2- 172.21.15.98:0/1882479581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9ac161450 0x7fd9ac0b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:05.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.383+0000 7fd9b4130640 1 --2- 172.21.15.98:0/1882479581 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9ac0d2650 0x7fd9ac0b6040 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:05.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.383+0000 7fd9b4130640 1 --2- 172.21.15.98:0/1882479581 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd9ac0b2a30 0x7fd9ac15f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:05.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.383+0000 7fd9b4130640 1 -- 172.21.15.98:0/1882479581 >> 172.21.15.98:0/1882479581 conn(0x7fd9ac0d3b30 msgr2=0x7fd9ac0b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:05.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.383+0000 7fd9b4130640 1 -- 172.21.15.98:0/1882479581 shutdown_connections 2024-05-09T10:51:05.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.384+0000 7fd9b4130640 1 -- 172.21.15.98:0/1882479581 wait complete. 2024-05-09T10:51:05.391 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T10:51:05.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.504+0000 7f510613b640 1 Processor -- start 2024-05-09T10:51:05.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.504+0000 7f510613b640 1 -- start start 2024-05-09T10:51:05.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.505+0000 7f510613b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f510015db90 0x7f51000af720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:05.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.505+0000 7f510613b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51000720d0 0x7f510006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:05.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.505+0000 7f510613b640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5100072690 con 0x7f510006abe0 2024-05-09T10:51:05.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.505+0000 7f510613b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51000afc60 con 0x7f510015db90 2024-05-09T10:51:05.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.505+0000 7f510613b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51000b5d70 con 0x7f51000720d0 2024-05-09T10:51:05.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.505+0000 7f50fffff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f510006abe0 0x7f510006aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52762/0 (socket says 172.21.15.98:52762) 2024-05-09T10:51:05.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.505+0000 7f50fffff640 1 -- 172.21.15.98:0/4001447187 learned_addr learned my addr 172.21.15.98:0/4001447187 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:05.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.505+0000 7f50fe7fc640 1 -- 172.21.15.98:0/4001447187 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 830200977 0 0) 0x7f5100072690 con 0x7f510006abe0 2024-05-09T10:51:05.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.505+0000 7f50ff7fe640 1 --2- 172.21.15.98:0/4001447187 >> v2:172.21.15.105:3300/0 conn(0x7f510015db90 0x7f51000af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:05.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.505+0000 7f50feffd640 1 --2- 172.21.15.98:0/4001447187 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51000720d0 0x7f510006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:05.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.505+0000 7f50fe7fc640 1 -- 172.21.15.98:0/4001447187 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50e8003520 con 0x7f510006abe0 2024-05-09T10:51:05.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50fe7fc640 1 -- 172.21.15.98:0/4001447187 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3401794493 0 0) 0x7f50e8003520 con 0x7f510006abe0 2024-05-09T10:51:05.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50fe7fc640 1 -- 172.21.15.98:0/4001447187 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51000720d0 msgr2=0x7f510006d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:05.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50fe7fc640 1 --2- 172.21.15.98:0/4001447187 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51000720d0 0x7f510006d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:05.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50fe7fc640 1 -- 172.21.15.98:0/4001447187 >> v2:172.21.15.105:3300/0 conn(0x7f510015db90 msgr2=0x7f51000af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:05.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50fe7fc640 1 --2- 172.21.15.98:0/4001447187 >> v2:172.21.15.105:3300/0 conn(0x7f510015db90 0x7f51000af720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:05.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50fe7fc640 1 -- 172.21.15.98:0/4001447187 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51000b57b0 con 0x7f510006abe0 2024-05-09T10:51:05.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50fe7fc640 1 -- 172.21.15.98:0/4001447187 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f50f80019f0 con 0x7f510006abe0 2024-05-09T10:51:05.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50feffd640 1 --2- 172.21.15.98:0/4001447187 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51000720d0 0x7f510006d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:05.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50fe7fc640 1 -- 172.21.15.98:0/4001447187 >> v1:172.21.15.98:6789/0 conn(0x7f510006abe0 legacy=0x7f510006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:05.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50fe7fc640 1 --2- 172.21.15.98:0/4001447187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50e80039b0 0x7f50e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:05.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50fe7fc640 1 --2- 172.21.15.98:0/4001447187 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51000720d0 0x7f50e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:05.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50fe7fc640 1 --2- 172.21.15.98:0/4001447187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50e8004d40 0x7f50e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:05.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50fe7fc640 1 -- 172.21.15.98:0/4001447187 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51000afc60 con 0x7f50e80039b0 2024-05-09T10:51:05.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50fe7fc640 1 -- 172.21.15.98:0/4001447187 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51000b5d70 con 0x7f50e8004d40 2024-05-09T10:51:05.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50fe7fc640 1 -- 172.21.15.98:0/4001447187 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e800ba10 con 0x7f51000720d0 2024-05-09T10:51:05.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50ff7fe640 1 --2- 172.21.15.98:0/4001447187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50e80039b0 0x7f50e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:05.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50feffd640 1 --2- 172.21.15.98:0/4001447187 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51000720d0 0x7f50e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:05.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50fffff640 1 --2- 172.21.15.98:0/4001447187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50e8004d40 0x7f50e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:05.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50ff7fe640 1 -- 172.21.15.98:0/4001447187 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51000720d0 msgr2=0x7f50e8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:05.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50ff7fe640 1 --2- 172.21.15.98:0/4001447187 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51000720d0 0x7f50e8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:05.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50ff7fe640 1 -- 172.21.15.98:0/4001447187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50e8004d40 msgr2=0x7f50e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:05.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50ff7fe640 1 --2- 172.21.15.98:0/4001447187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50e8004d40 0x7f50e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:05.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50ff7fe640 1 -- 172.21.15.98:0/4001447187 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e8003520 con 0x7f50e80039b0 2024-05-09T10:51:05.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.506+0000 7f50fffff640 1 --2- 172.21.15.98:0/4001447187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50e8004d40 0x7f50e80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:05.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.507+0000 7f50ff7fe640 1 --2- 172.21.15.98:0/4001447187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50e80039b0 0x7f50e8003dd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009ac0 tx=0x7f50e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c242d27c5a67f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:05.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.507+0000 7f50fe7fc640 1 -- 172.21.15.98:0/4001447187 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f0036070 con 0x7f50e80039b0 2024-05-09T10:51:05.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.507+0000 7f50fe7fc640 1 -- 172.21.15.98:0/4001447187 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f0031400 con 0x7f50e80039b0 2024-05-09T10:51:05.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.507+0000 7f50fe7fc640 1 -- 172.21.15.98:0/4001447187 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f0009690 con 0x7f50e80039b0 2024-05-09T10:51:05.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.507+0000 7f510613b640 1 -- 172.21.15.98:0/4001447187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50e80039b0 msgr2=0x7f50e8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:05.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.507+0000 7f510613b640 1 --2- 172.21.15.98:0/4001447187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50e80039b0 0x7f50e8003dd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009ac0 tx=0x7f50e8005830 comp rx=0 tx=0).stop 2024-05-09T10:51:05.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.507+0000 7f50fffff640 1 -- 172.21.15.98:0/4001447187 reap_dead start 2024-05-09T10:51:05.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.507+0000 7f510613b640 1 -- 172.21.15.98:0/4001447187 shutdown_connections 2024-05-09T10:51:05.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.507+0000 7f510613b640 1 -- 172.21.15.98:0/4001447187 >> 172.21.15.98:0/4001447187 conn(0x7f5100066fd0 msgr2=0x7f510006bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:05.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.508+0000 7f510613b640 1 -- 172.21.15.98:0/4001447187 shutdown_connections 2024-05-09T10:51:05.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.508+0000 7f510613b640 1 -- 172.21.15.98:0/4001447187 wait complete. 2024-05-09T10:51:05.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.509+0000 7f510613b640 1 Processor -- start 2024-05-09T10:51:05.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.509+0000 7f510613b640 1 -- start start 2024-05-09T10:51:05.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.510+0000 7f510613b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50e80039b0 0x7f5100151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:05.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.510+0000 7f510613b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5100160060 0x7f5100155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:05.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.510+0000 7f510613b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5100157500 0x7f51000d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:05.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.510+0000 7f510613b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510015df70 con 0x7f50e80039b0 2024-05-09T10:51:05.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.510+0000 7f510613b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51000d0670 con 0x7f5100160060 2024-05-09T10:51:05.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.510+0000 7f510613b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510015e0f0 con 0x7f5100157500 2024-05-09T10:51:05.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.510+0000 7f50ff7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50e80039b0 0x7f5100151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:05.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.510+0000 7f50ff7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50e80039b0 0x7f5100151b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48926/0 (socket says 172.21.15.98:48926) 2024-05-09T10:51:05.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.510+0000 7f50ff7fe640 1 -- 172.21.15.98:0/1869135364 learned_addr learned my addr 172.21.15.98:0/1869135364 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:05.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.510+0000 7f50feffd640 1 --2- 172.21.15.98:0/1869135364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5100160060 0x7f5100155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:05.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.510+0000 7f50fffff640 1 --2- 172.21.15.98:0/1869135364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5100157500 0x7f51000d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:05.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.510+0000 7f50ff7fe640 1 -- 172.21.15.98:0/1869135364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5100157500 msgr2=0x7f51000d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:05.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.510+0000 7f50ff7fe640 1 --2- 172.21.15.98:0/1869135364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5100157500 0x7f51000d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:05.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.510+0000 7f50ff7fe640 1 -- 172.21.15.98:0/1869135364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5100160060 msgr2=0x7f5100155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:05.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.510+0000 7f50ff7fe640 1 --2- 172.21.15.98:0/1869135364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5100160060 0x7f5100155af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:05.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.510+0000 7f50ff7fe640 1 -- 172.21.15.98:0/1869135364 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51001520d0 con 0x7f50e80039b0 2024-05-09T10:51:05.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.510+0000 7f50ff7fe640 1 --2- 172.21.15.98:0/1869135364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50e80039b0 0x7f5100151b90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009ac0 tx=0x7f50f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:05.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.510+0000 7f50e7fff640 1 -- 172.21.15.98:0/1869135364 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f0040020 con 0x7f50e80039b0 2024-05-09T10:51:05.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.510+0000 7f50e7fff640 1 -- 172.21.15.98:0/1869135364 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f0003b40 con 0x7f50e80039b0 2024-05-09T10:51:05.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.510+0000 7f510613b640 1 -- 172.21.15.98:0/1869135364 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5100156460 con 0x7f50e80039b0 2024-05-09T10:51:05.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.510+0000 7f50e7fff640 1 -- 172.21.15.98:0/1869135364 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f0036040 con 0x7f50e80039b0 2024-05-09T10:51:05.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.510+0000 7f510613b640 1 -- 172.21.15.98:0/1869135364 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5100156d00 con 0x7f50e80039b0 2024-05-09T10:51:05.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.511+0000 7f510613b640 1 -- 172.21.15.98:0/1869135364 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5100156850 con 0x7f50e80039b0 2024-05-09T10:51:05.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.512+0000 7f50e7fff640 1 -- 172.21.15.98:0/1869135364 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50f0043020 con 0x7f50e80039b0 2024-05-09T10:51:05.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.512+0000 7f50e7fff640 1 --2- 172.21.15.98:0/1869135364 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f50ec06b500 0x7f50ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:05.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.512+0000 7f50e7fff640 1 -- 172.21.15.98:0/1869135364 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f50f00aa040 con 0x7f50e80039b0 2024-05-09T10:51:05.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.512+0000 7f50feffd640 1 --2- 172.21.15.98:0/1869135364 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f50ec06b500 0x7f50ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:05.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.513+0000 7f50feffd640 1 --2- 172.21.15.98:0/1869135364 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f50ec06b500 0x7f50ec06d9c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f510006e260 tx=0x7f50f4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:05.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.517+0000 7f50e7fff640 1 -- 172.21.15.98:0/1869135364 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f50f004a070 con 0x7f50e80039b0 2024-05-09T10:51:05.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:05.689+0000 7f510613b640 1 -- 172.21.15.98:0/1869135364 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f510004f000 con 0x7f50e80039b0 2024-05-09T10:51:06.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.383+0000 7f50e7fff640 1 -- 172.21.15.98:0/1869135364 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v29) v1 ==== 170+0+0 (secure 0 0 0) 0x7f50f00044e0 con 0x7f50e80039b0 2024-05-09T10:51:06.377 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' created 2024-05-09T10:51:06.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.387+0000 7f510613b640 1 -- 172.21.15.98:0/1869135364 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f50ec06b500 msgr2=0x7f50ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:06.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.387+0000 7f510613b640 1 --2- 172.21.15.98:0/1869135364 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f50ec06b500 0x7f50ec06d9c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f510006e260 tx=0x7f50f4008040 comp rx=0 tx=0).stop 2024-05-09T10:51:06.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.387+0000 7f510613b640 1 -- 172.21.15.98:0/1869135364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50e80039b0 msgr2=0x7f5100151b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:06.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.387+0000 7f510613b640 1 --2- 172.21.15.98:0/1869135364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50e80039b0 0x7f5100151b90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009ac0 tx=0x7f50f0004730 comp rx=0 tx=0).stop 2024-05-09T10:51:06.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.389+0000 7f510613b640 1 -- 172.21.15.98:0/1869135364 shutdown_connections 2024-05-09T10:51:06.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.389+0000 7f510613b640 1 --2- 172.21.15.98:0/1869135364 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f50ec06b500 0x7f50ec06d9c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:06.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.389+0000 7f510613b640 1 --2- 172.21.15.98:0/1869135364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5100157500 0x7f51000d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:06.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.389+0000 7f510613b640 1 --2- 172.21.15.98:0/1869135364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5100160060 0x7f5100155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:06.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.389+0000 7f510613b640 1 --2- 172.21.15.98:0/1869135364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50e80039b0 0x7f5100151b90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:06.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.389+0000 7f510613b640 1 -- 172.21.15.98:0/1869135364 >> 172.21.15.98:0/1869135364 conn(0x7f5100066fd0 msgr2=0x7f510015d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:06.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.390+0000 7f510613b640 1 -- 172.21.15.98:0/1869135364 shutdown_connections 2024-05-09T10:51:06.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.393+0000 7f510613b640 1 -- 172.21.15.98:0/1869135364 wait complete. 2024-05-09T10:51:06.406 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T10:51:06.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.525+0000 7f44dc79e640 1 Processor -- start 2024-05-09T10:51:06.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.525+0000 7f44dc79e640 1 -- start start 2024-05-09T10:51:06.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.526+0000 7f44dc79e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f44d40b20e0 0x7f44d40ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:06.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.526+0000 7f44dc79e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f44d40b2780 0x7f44d40ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:06.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.526+0000 7f44dc79e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d40d1710 con 0x7f44d40cec10 2024-05-09T10:51:06.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.526+0000 7f44dc79e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d40b2e70 con 0x7f44d40b2780 2024-05-09T10:51:06.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.526+0000 7f44dc79e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d40d05e0 con 0x7f44d40b20e0 2024-05-09T10:51:06.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.526+0000 7f44d9d39640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f44d40b2780 0x7f44d40ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:06.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.526+0000 7f44d9d39640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f44d40b2780 0x7f44d40ce6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48940/0 (socket says 172.21.15.98:48940) 2024-05-09T10:51:06.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.526+0000 7f44dad3b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f44d40cec10 0x7f44d406bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52764/0 (socket says 172.21.15.98:52764) 2024-05-09T10:51:06.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.526+0000 7f44d9d39640 1 -- 172.21.15.98:0/4282814025 learned_addr learned my addr 172.21.15.98:0/4282814025 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:06.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.526+0000 7f44da53a640 1 --2- 172.21.15.98:0/4282814025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f44d40b20e0 0x7f44d40ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:06.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.526+0000 7f44d9d39640 1 -- 172.21.15.98:0/4282814025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f44d40b20e0 msgr2=0x7f44d40ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:06.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.526+0000 7f44d9d39640 1 --2- 172.21.15.98:0/4282814025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f44d40b20e0 0x7f44d40ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:06.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.526+0000 7f44d9d39640 1 -- 172.21.15.98:0/4282814025 >> v1:172.21.15.98:6789/0 conn(0x7f44d40cec10 legacy=0x7f44d406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:06.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.526+0000 7f44d9d39640 1 -- 172.21.15.98:0/4282814025 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d40d08c0 con 0x7f44d40b2780 2024-05-09T10:51:06.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.527+0000 7f44d9d39640 1 --2- 172.21.15.98:0/4282814025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44d40b2780 0x7f44d40ce6d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f44c4009870 tx=0x7f44c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4d5bf3426db4d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:06.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.527+0000 7f44d9538640 1 -- 172.21.15.98:0/4282814025 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c4037070 con 0x7f44d40b2780 2024-05-09T10:51:06.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.527+0000 7f44d9538640 1 -- 172.21.15.98:0/4282814025 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c4032440 con 0x7f44d40b2780 2024-05-09T10:51:06.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.527+0000 7f44d9538640 1 -- 172.21.15.98:0/4282814025 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c4032810 con 0x7f44d40b2780 2024-05-09T10:51:06.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.528+0000 7f44dc79e640 1 -- 172.21.15.98:0/4282814025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44d40b2780 msgr2=0x7f44d40ce6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:06.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.528+0000 7f44dc79e640 1 --2- 172.21.15.98:0/4282814025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44d40b2780 0x7f44d40ce6d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f44c4009870 tx=0x7f44c402a270 comp rx=0 tx=0).stop 2024-05-09T10:51:06.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.528+0000 7f44dc79e640 1 -- 172.21.15.98:0/4282814025 shutdown_connections 2024-05-09T10:51:06.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.528+0000 7f44dc79e640 1 --2- 172.21.15.98:0/4282814025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44d40b2780 0x7f44d40ce6d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:06.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.528+0000 7f44dc79e640 1 --2- 172.21.15.98:0/4282814025 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f44d40b20e0 0x7f44d40ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:06.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.528+0000 7f44dc79e640 1 -- 172.21.15.98:0/4282814025 >> 172.21.15.98:0/4282814025 conn(0x7f44d40b34e0 msgr2=0x7f44d40b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:06.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.528+0000 7f44dc79e640 1 -- 172.21.15.98:0/4282814025 shutdown_connections 2024-05-09T10:51:06.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.528+0000 7f44dc79e640 1 -- 172.21.15.98:0/4282814025 wait complete. 2024-05-09T10:51:06.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.529+0000 7f44dc79e640 1 Processor -- start 2024-05-09T10:51:06.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.529+0000 7f44dc79e640 1 -- start start 2024-05-09T10:51:06.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.530+0000 7f44dc79e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44d40b20e0 0x7f44d415ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:06.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.530+0000 7f44dc79e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f44d40b2780 0x7f44d41f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:06.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.530+0000 7f44dc79e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44d40cec10 0x7f44d415e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:06.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.530+0000 7f44dc79e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d40cd070 con 0x7f44d40cec10 2024-05-09T10:51:06.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.530+0000 7f44dc79e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d40b3150 con 0x7f44d40b20e0 2024-05-09T10:51:06.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.530+0000 7f44dc79e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d40cd1f0 con 0x7f44d40b2780 2024-05-09T10:51:06.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.530+0000 7f44dad3b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44d40cec10 0x7f44d415e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:06.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.530+0000 7f44dad3b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44d40cec10 0x7f44d415e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48946/0 (socket says 172.21.15.98:48946) 2024-05-09T10:51:06.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.530+0000 7f44dad3b640 1 -- 172.21.15.98:0/2516522142 learned_addr learned my addr 172.21.15.98:0/2516522142 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:06.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.530+0000 7f44d9d39640 1 --2- 172.21.15.98:0/2516522142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f44d40b2780 0x7f44d41f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:06.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.530+0000 7f44da53a640 1 --2- 172.21.15.98:0/2516522142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44d40b20e0 0x7f44d415ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:06.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.530+0000 7f44dad3b640 1 -- 172.21.15.98:0/2516522142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f44d40b2780 msgr2=0x7f44d41f6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:06.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.530+0000 7f44dad3b640 1 --2- 172.21.15.98:0/2516522142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f44d40b2780 0x7f44d41f6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:06.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.530+0000 7f44dad3b640 1 -- 172.21.15.98:0/2516522142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44d40b20e0 msgr2=0x7f44d415ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:06.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.530+0000 7f44dad3b640 1 --2- 172.21.15.98:0/2516522142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44d40b20e0 0x7f44d415ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:06.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.530+0000 7f44dad3b640 1 -- 172.21.15.98:0/2516522142 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d41f7530 con 0x7f44d40cec10 2024-05-09T10:51:06.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.530+0000 7f44da53a640 1 --2- 172.21.15.98:0/2516522142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44d40b20e0 0x7f44d415ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:06.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.530+0000 7f44dad3b640 1 --2- 172.21.15.98:0/2516522142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44d40cec10 0x7f44d415e1b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00c400 tx=0x7f44cc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:06.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.530+0000 7f44caffd640 1 -- 172.21.15.98:0/2516522142 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc0074e0 con 0x7f44d40cec10 2024-05-09T10:51:06.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.530+0000 7f44caffd640 1 -- 172.21.15.98:0/2516522142 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44cc007680 con 0x7f44d40cec10 2024-05-09T10:51:06.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.530+0000 7f44dc79e640 1 -- 172.21.15.98:0/2516522142 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d40b60e0 con 0x7f44d40cec10 2024-05-09T10:51:06.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.530+0000 7f44caffd640 1 -- 172.21.15.98:0/2516522142 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc0049e0 con 0x7f44d40cec10 2024-05-09T10:51:06.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.530+0000 7f44dc79e640 1 -- 172.21.15.98:0/2516522142 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d41f7ab0 con 0x7f44d40cec10 2024-05-09T10:51:06.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.531+0000 7f44dc79e640 1 -- 172.21.15.98:0/2516522142 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d415b0a0 con 0x7f44d40cec10 2024-05-09T10:51:06.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.532+0000 7f44caffd640 1 -- 172.21.15.98:0/2516522142 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44cc01a020 con 0x7f44d40cec10 2024-05-09T10:51:06.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.532+0000 7f44caffd640 1 --2- 172.21.15.98:0/2516522142 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f44b806b500 0x7f44b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:06.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.533+0000 7f44da53a640 1 --2- 172.21.15.98:0/2516522142 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f44b806b500 0x7f44b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:06.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.533+0000 7f44caffd640 1 -- 172.21.15.98:0/2516522142 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f44cc089880 con 0x7f44d40cec10 2024-05-09T10:51:06.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.533+0000 7f44da53a640 1 --2- 172.21.15.98:0/2516522142 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f44b806b500 0x7f44b806d9c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f44d0009a80 tx=0x7f44d00039d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:06.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.537+0000 7f44caffd640 1 -- 172.21.15.98:0/2516522142 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44cc058760 con 0x7f44d40cec10 2024-05-09T10:51:06.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:06.743+0000 7f44dc79e640 1 -- 172.21.15.98:0/2516522142 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f44d415e6f0 con 0x7f44d40cec10 2024-05-09T10:51:07.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.391+0000 7f44caffd640 1 -- 172.21.15.98:0/2516522142 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7f44cc058060 con 0x7f44d40cec10 2024-05-09T10:51:07.385 INFO:teuthology.orchestra.run.smithi098.stderr:new fs with metadata pool 4 and data pool 5 2024-05-09T10:51:07.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.395+0000 7f44dc79e640 1 -- 172.21.15.98:0/2516522142 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f44b806b500 msgr2=0x7f44b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:07.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.395+0000 7f44dc79e640 1 --2- 172.21.15.98:0/2516522142 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f44b806b500 0x7f44b806d9c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f44d0009a80 tx=0x7f44d00039d0 comp rx=0 tx=0).stop 2024-05-09T10:51:07.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.395+0000 7f44dc79e640 1 -- 172.21.15.98:0/2516522142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44d40cec10 msgr2=0x7f44d415e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:07.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.395+0000 7f44dc79e640 1 --2- 172.21.15.98:0/2516522142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44d40cec10 0x7f44d415e1b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00c400 tx=0x7f44cc00c8c0 comp rx=0 tx=0).stop 2024-05-09T10:51:07.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.395+0000 7f44dc79e640 1 -- 172.21.15.98:0/2516522142 shutdown_connections 2024-05-09T10:51:07.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.395+0000 7f44dc79e640 1 --2- 172.21.15.98:0/2516522142 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f44b806b500 0x7f44b806d9c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:07.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.395+0000 7f44dc79e640 1 --2- 172.21.15.98:0/2516522142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44d40cec10 0x7f44d415e1b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:07.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.395+0000 7f44dc79e640 1 --2- 172.21.15.98:0/2516522142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f44d40b2780 0x7f44d41f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:07.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.395+0000 7f44dc79e640 1 --2- 172.21.15.98:0/2516522142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f44d40b20e0 0x7f44d415ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:07.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.395+0000 7f44dc79e640 1 -- 172.21.15.98:0/2516522142 >> 172.21.15.98:0/2516522142 conn(0x7f44d40b34e0 msgr2=0x7f44d406e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:07.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.396+0000 7f44dc79e640 1 -- 172.21.15.98:0/2516522142 shutdown_connections 2024-05-09T10:51:07.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.396+0000 7f44dc79e640 1 -- 172.21.15.98:0/2516522142 wait complete. 2024-05-09T10:51:07.413 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:51:07.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.522+0000 7f8c2eae2640 1 Processor -- start 2024-05-09T10:51:07.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.522+0000 7f8c2eae2640 1 -- start start 2024-05-09T10:51:07.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.522+0000 7f8c2eae2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8c2806cac0 0x7f8c2806bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:07.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.522+0000 7f8c2eae2640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8c2815e0c0 0x7f8c280b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:07.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.522+0000 7f8c2eae2640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c280d0c70 con 0x7f8c2815dce0 2024-05-09T10:51:07.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.522+0000 7f8c2eae2640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c280b4350 con 0x7f8c2815e0c0 2024-05-09T10:51:07.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.522+0000 7f8c2eae2640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c280d0f50 con 0x7f8c2806cac0 2024-05-09T10:51:07.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.523+0000 7f8c2d07f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8c2815e0c0 0x7f8c280b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:07.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.523+0000 7f8c2d07f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8c2815e0c0 0x7f8c280b1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48954/0 (socket says 172.21.15.98:48954) 2024-05-09T10:51:07.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.523+0000 7f8c2d07f640 1 -- 172.21.15.98:0/1595762590 learned_addr learned my addr 172.21.15.98:0/1595762590 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:07.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.523+0000 7f8c27fff640 1 --2- 172.21.15.98:0/1595762590 >> v2:172.21.15.105:3300/0 conn(0x7f8c2806cac0 0x7f8c2806bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:07.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.523+0000 7f8c2d07f640 1 -- 172.21.15.98:0/1595762590 >> v2:172.21.15.105:3300/0 conn(0x7f8c2806cac0 msgr2=0x7f8c2806bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:07.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.523+0000 7f8c2d07f640 1 --2- 172.21.15.98:0/1595762590 >> v2:172.21.15.105:3300/0 conn(0x7f8c2806cac0 0x7f8c2806bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:07.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.523+0000 7f8c2d07f640 1 -- 172.21.15.98:0/1595762590 >> v1:172.21.15.105:6789/0 conn(0x7f8c2815dce0 legacy=0x7f8c2806c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:07.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.523+0000 7f8c2d07f640 1 -- 172.21.15.98:0/1595762590 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c280b0fd0 con 0x7f8c2815e0c0 2024-05-09T10:51:07.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.523+0000 7f8c27fff640 1 --2- 172.21.15.98:0/1595762590 >> v2:172.21.15.105:3300/0 conn(0x7f8c2806cac0 0x7f8c2806bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:07.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.524+0000 7f8c2d07f640 1 --2- 172.21.15.98:0/1595762590 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2815e0c0 0x7f8c280b1ab0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8c280717d0 tx=0x7f8c2002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef6bbeeb57f0818 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:07.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.524+0000 7f8c277fe640 1 -- 172.21.15.98:0/1595762590 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c20037070 con 0x7f8c2815e0c0 2024-05-09T10:51:07.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.524+0000 7f8c277fe640 1 -- 172.21.15.98:0/1595762590 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2002ae50 con 0x7f8c2815e0c0 2024-05-09T10:51:07.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.524+0000 7f8c277fe640 1 -- 172.21.15.98:0/1595762590 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c20032860 con 0x7f8c2815e0c0 2024-05-09T10:51:07.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.524+0000 7f8c2eae2640 1 -- 172.21.15.98:0/1595762590 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2815e0c0 msgr2=0x7f8c280b1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:07.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.524+0000 7f8c2eae2640 1 --2- 172.21.15.98:0/1595762590 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2815e0c0 0x7f8c280b1ab0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8c280717d0 tx=0x7f8c2002a300 comp rx=0 tx=0).stop 2024-05-09T10:51:07.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.524+0000 7f8c2eae2640 1 -- 172.21.15.98:0/1595762590 shutdown_connections 2024-05-09T10:51:07.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.524+0000 7f8c2eae2640 1 --2- 172.21.15.98:0/1595762590 >> v2:172.21.15.105:3300/0 conn(0x7f8c2806cac0 0x7f8c2806bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:07.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.524+0000 7f8c2eae2640 1 --2- 172.21.15.98:0/1595762590 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2815e0c0 0x7f8c280b1ab0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:07.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.524+0000 7f8c2eae2640 1 -- 172.21.15.98:0/1595762590 >> 172.21.15.98:0/1595762590 conn(0x7f8c280b34e0 msgr2=0x7f8c280b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:07.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.524+0000 7f8c2eae2640 1 -- 172.21.15.98:0/1595762590 shutdown_connections 2024-05-09T10:51:07.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.524+0000 7f8c2eae2640 1 -- 172.21.15.98:0/1595762590 wait complete. 2024-05-09T10:51:07.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.525+0000 7f8c2eae2640 1 Processor -- start 2024-05-09T10:51:07.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.526+0000 7f8c2eae2640 1 -- start start 2024-05-09T10:51:07.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.526+0000 7f8c2eae2640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c2806cac0 0x7f8c2815be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:07.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.526+0000 7f8c2eae2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c2815dce0 0x7f8c2814c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:07.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.526+0000 7f8c2eae2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2815e0c0 0x7f8c280c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:07.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.526+0000 7f8c2eae2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c280d0700 con 0x7f8c2815e0c0 2024-05-09T10:51:07.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.526+0000 7f8c2eae2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c280b1590 con 0x7f8c2815dce0 2024-05-09T10:51:07.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.526+0000 7f8c2eae2640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c280d0880 con 0x7f8c2806cac0 2024-05-09T10:51:07.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.527+0000 7f8c2d07f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2815e0c0 0x7f8c280c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:07.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.527+0000 7f8c2d07f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2815e0c0 0x7f8c280c55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48968/0 (socket says 172.21.15.98:48968) 2024-05-09T10:51:07.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.527+0000 7f8c2d07f640 1 -- 172.21.15.98:0/923630100 learned_addr learned my addr 172.21.15.98:0/923630100 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:07.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.527+0000 7f8c27fff640 1 --2- 172.21.15.98:0/923630100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c2815dce0 0x7f8c2814c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:07.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.527+0000 7f8c2c87e640 1 --2- 172.21.15.98:0/923630100 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c2806cac0 0x7f8c2815be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:07.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.527+0000 7f8c2d07f640 1 -- 172.21.15.98:0/923630100 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c2806cac0 msgr2=0x7f8c2815be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:07.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.527+0000 7f8c2d07f640 1 --2- 172.21.15.98:0/923630100 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c2806cac0 0x7f8c2815be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:07.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.527+0000 7f8c2d07f640 1 -- 172.21.15.98:0/923630100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c2815dce0 msgr2=0x7f8c2814c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:07.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.527+0000 7f8c2d07f640 1 --2- 172.21.15.98:0/923630100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c2815dce0 0x7f8c2814c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:07.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.527+0000 7f8c2d07f640 1 -- 172.21.15.98:0/923630100 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c280c5b20 con 0x7f8c2815e0c0 2024-05-09T10:51:07.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.527+0000 7f8c2c87e640 1 --2- 172.21.15.98:0/923630100 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c2806cac0 0x7f8c2815be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:07.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.527+0000 7f8c2d07f640 1 --2- 172.21.15.98:0/923630100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2815e0c0 0x7f8c280c55e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8c2002a340 tx=0x7f8c200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:07.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.527+0000 7f8c257fa640 1 -- 172.21.15.98:0/923630100 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c20041020 con 0x7f8c2815e0c0 2024-05-09T10:51:07.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.527+0000 7f8c257fa640 1 -- 172.21.15.98:0/923630100 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2003c890 con 0x7f8c2815e0c0 2024-05-09T10:51:07.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.527+0000 7f8c257fa640 1 -- 172.21.15.98:0/923630100 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c20037040 con 0x7f8c2815e0c0 2024-05-09T10:51:07.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.527+0000 7f8c2eae2640 1 -- 172.21.15.98:0/923630100 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c280b4b50 con 0x7f8c2815e0c0 2024-05-09T10:51:07.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.527+0000 7f8c2eae2640 1 -- 172.21.15.98:0/923630100 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2815b630 con 0x7f8c2815e0c0 2024-05-09T10:51:07.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.528+0000 7f8c2eae2640 1 -- 172.21.15.98:0/923630100 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c28069cb0 con 0x7f8c2815e0c0 2024-05-09T10:51:07.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.529+0000 7f8c257fa640 1 -- 172.21.15.98:0/923630100 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c20044020 con 0x7f8c2815e0c0 2024-05-09T10:51:07.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.529+0000 7f8c257fa640 1 --2- 172.21.15.98:0/923630100 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8c0806b500 0x7f8c0806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:07.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.529+0000 7f8c257fa640 1 -- 172.21.15.98:0/923630100 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c200aaa80 con 0x7f8c2815e0c0 2024-05-09T10:51:07.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.529+0000 7f8c2c87e640 1 --2- 172.21.15.98:0/923630100 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8c0806b500 0x7f8c0806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:07.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.530+0000 7f8c2c87e640 1 --2- 172.21.15.98:0/923630100 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8c0806b500 0x7f8c0806d9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8c18001d70 tx=0x7f8c18009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:07.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.533+0000 7f8c257fa640 1 -- 172.21.15.98:0/923630100 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c2004b030 con 0x7f8c2815e0c0 2024-05-09T10:51:07.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.703+0000 7f8c2eae2640 1 -- 172.21.15.98:0/923630100 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c280c5d50 con 0x7f8c2815e0c0 2024-05-09T10:51:07.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.704+0000 7f8c257fa640 1 -- 172.21.15.98:0/923630100 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+16682 (secure 0 0 0) 0x7f8c20079900 con 0x7f8c2815e0c0 2024-05-09T10:51:07.698 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:07.698 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":30,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:51:06.744689+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:51:05.150410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-09T10:51:05.708391+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:51:07.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.706+0000 7f8c2eae2640 1 -- 172.21.15.98:0/923630100 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8c0806b500 msgr2=0x7f8c0806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:07.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.706+0000 7f8c2eae2640 1 --2- 172.21.15.98:0/923630100 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8c0806b500 0x7f8c0806d9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8c18001d70 tx=0x7f8c18009040 comp rx=0 tx=0).stop 2024-05-09T10:51:07.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.706+0000 7f8c2eae2640 1 -- 172.21.15.98:0/923630100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2815e0c0 msgr2=0x7f8c280c55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:07.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.706+0000 7f8c2eae2640 1 --2- 172.21.15.98:0/923630100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2815e0c0 0x7f8c280c55e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8c2002a340 tx=0x7f8c200046c0 comp rx=0 tx=0).stop 2024-05-09T10:51:07.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.706+0000 7f8c2eae2640 1 -- 172.21.15.98:0/923630100 shutdown_connections 2024-05-09T10:51:07.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.706+0000 7f8c2eae2640 1 --2- 172.21.15.98:0/923630100 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8c0806b500 0x7f8c0806d9c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:07.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.706+0000 7f8c2eae2640 1 --2- 172.21.15.98:0/923630100 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2815e0c0 0x7f8c280c55e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:07.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.706+0000 7f8c2eae2640 1 --2- 172.21.15.98:0/923630100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c2815dce0 0x7f8c2814c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:07.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.706+0000 7f8c2eae2640 1 --2- 172.21.15.98:0/923630100 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c2806cac0 0x7f8c2815be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:07.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.706+0000 7f8c2eae2640 1 -- 172.21.15.98:0/923630100 >> 172.21.15.98:0/923630100 conn(0x7f8c280b34e0 msgr2=0x7f8c2806f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:07.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.707+0000 7f8c2eae2640 1 -- 172.21.15.98:0/923630100 shutdown_connections 2024-05-09T10:51:07.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.707+0000 7f8c2eae2640 1 -- 172.21.15.98:0/923630100 wait complete. 2024-05-09T10:51:07.711 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:51:07.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.818+0000 7fc33a4f7640 1 Processor -- start 2024-05-09T10:51:07.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.818+0000 7fc33a4f7640 1 -- start start 2024-05-09T10:51:07.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.819+0000 7fc33a4f7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc33415ba70 0x7fc3340cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:07.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.819+0000 7fc33a4f7640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3340b3610 con 0x7fc3340b43d0 2024-05-09T10:51:07.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.819+0000 7fc33a4f7640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3340b38f0 con 0x7fc33415be50 2024-05-09T10:51:07.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.819+0000 7fc33a4f7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3340b3bd0 con 0x7fc33415ba70 2024-05-09T10:51:07.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.819+0000 7fc338a94640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc3340b43d0 0x7fc33406cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52774/0 (socket says 172.21.15.98:52774) 2024-05-09T10:51:07.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.819+0000 7fc338a94640 1 -- 172.21.15.98:0/12398846 learned_addr learned my addr 172.21.15.98:0/12398846 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:07.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.819+0000 7fc333fff640 1 --2- 172.21.15.98:0/12398846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc33415ba70 0x7fc3340cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:07.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.819+0000 7fc332ffd640 1 -- 172.21.15.98:0/12398846 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3421341093 0 0) 0x7fc3340b3610 con 0x7fc3340b43d0 2024-05-09T10:51:07.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.819+0000 7fc332ffd640 1 -- 172.21.15.98:0/12398846 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc31c003520 con 0x7fc3340b43d0 2024-05-09T10:51:07.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.819+0000 7fc332ffd640 1 -- 172.21.15.98:0/12398846 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1820385893 0 0) 0x7fc3340b38f0 con 0x7fc33415be50 2024-05-09T10:51:07.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.819+0000 7fc332ffd640 1 -- 172.21.15.98:0/12398846 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3340b3610 con 0x7fc33415be50 2024-05-09T10:51:07.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.820+0000 7fc333fff640 1 -- 172.21.15.98:0/12398846 >> v1:172.21.15.105:6789/0 conn(0x7fc33415be50 legacy=0x7fc3340cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:07.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.820+0000 7fc333fff640 1 -- 172.21.15.98:0/12398846 >> v1:172.21.15.98:6789/0 conn(0x7fc3340b43d0 legacy=0x7fc33406cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:07.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.820+0000 7fc332ffd640 1 -- 172.21.15.98:0/12398846 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2688467597 0 0) 0x7fc31c003520 con 0x7fc3340b43d0 2024-05-09T10:51:07.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.820+0000 7fc333fff640 1 -- 172.21.15.98:0/12398846 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3340cce60 con 0x7fc33415ba70 2024-05-09T10:51:07.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.820+0000 7fc333fff640 1 --2- 172.21.15.98:0/12398846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc33415ba70 0x7fc3340cc920 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc3240097d0 tx=0x7fc32402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=208f065999984711 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:07.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.821+0000 7fc332ffd640 1 -- 172.21.15.98:0/12398846 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc324037070 con 0x7fc33415ba70 2024-05-09T10:51:07.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.821+0000 7fc332ffd640 1 -- 172.21.15.98:0/12398846 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32402ac30 con 0x7fc33415ba70 2024-05-09T10:51:07.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.821+0000 7fc332ffd640 1 -- 172.21.15.98:0/12398846 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0019f0 con 0x7fc33415ba70 2024-05-09T10:51:07.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.821+0000 7fc33a4f7640 1 -- 172.21.15.98:0/12398846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc33415ba70 msgr2=0x7fc3340cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:07.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.821+0000 7fc33a4f7640 1 --2- 172.21.15.98:0/12398846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc33415ba70 0x7fc3340cc920 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc3240097d0 tx=0x7fc32402a1a0 comp rx=0 tx=0).stop 2024-05-09T10:51:07.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.821+0000 7fc33a4f7640 1 -- 172.21.15.98:0/12398846 shutdown_connections 2024-05-09T10:51:07.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.821+0000 7fc33a4f7640 1 --2- 172.21.15.98:0/12398846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc33415ba70 0x7fc3340cc920 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:07.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.821+0000 7fc33a4f7640 1 -- 172.21.15.98:0/12398846 >> 172.21.15.98:0/12398846 conn(0x7fc33406abe0 msgr2=0x7fc33406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:07.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.821+0000 7fc33a4f7640 1 -- 172.21.15.98:0/12398846 shutdown_connections 2024-05-09T10:51:07.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.821+0000 7fc33a4f7640 1 -- 172.21.15.98:0/12398846 wait complete. 2024-05-09T10:51:07.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.822+0000 7fc33a4f7640 1 Processor -- start 2024-05-09T10:51:07.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.822+0000 7fc33a4f7640 1 -- start start 2024-05-09T10:51:07.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.823+0000 7fc33a4f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3340b43d0 0x7fc334158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:07.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.823+0000 7fc33a4f7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc33415ba70 0x7fc334147dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:07.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.823+0000 7fc33a4f7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc33415be50 0x7fc334148300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:07.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.823+0000 7fc33a4f7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3340cfcd0 con 0x7fc33415ba70 2024-05-09T10:51:07.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.823+0000 7fc33a4f7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3340cfb50 con 0x7fc3340b43d0 2024-05-09T10:51:07.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.823+0000 7fc33a4f7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3340cfe50 con 0x7fc33415be50 2024-05-09T10:51:07.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.823+0000 7fc3337fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc33415ba70 0x7fc334147dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:07.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.823+0000 7fc3337fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc33415ba70 0x7fc334147dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48972/0 (socket says 172.21.15.98:48972) 2024-05-09T10:51:07.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.823+0000 7fc3337fe640 1 -- 172.21.15.98:0/1010656016 learned_addr learned my addr 172.21.15.98:0/1010656016 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:07.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.824+0000 7fc3337fe640 1 -- 172.21.15.98:0/1010656016 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc33415be50 msgr2=0x7fc334148300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:07.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.824+0000 7fc338a94640 1 --2- 172.21.15.98:0/1010656016 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc33415be50 0x7fc334148300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:07.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.824+0000 7fc333fff640 1 --2- 172.21.15.98:0/1010656016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3340b43d0 0x7fc334158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:07.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.824+0000 7fc3337fe640 1 --2- 172.21.15.98:0/1010656016 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc33415be50 0x7fc334148300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:07.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.824+0000 7fc3337fe640 1 -- 172.21.15.98:0/1010656016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3340b43d0 msgr2=0x7fc334158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:07.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.824+0000 7fc3337fe640 1 --2- 172.21.15.98:0/1010656016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3340b43d0 0x7fc334158b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:07.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.824+0000 7fc3337fe640 1 -- 172.21.15.98:0/1010656016 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3340a9470 con 0x7fc33415ba70 2024-05-09T10:51:07.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.824+0000 7fc3337fe640 1 --2- 172.21.15.98:0/1010656016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc33415ba70 0x7fc334147dc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc32800c3c0 tx=0x7fc32800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:07.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.824+0000 7fc330ff9640 1 -- 172.21.15.98:0/1010656016 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc328017070 con 0x7fc33415ba70 2024-05-09T10:51:07.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.824+0000 7fc330ff9640 1 -- 172.21.15.98:0/1010656016 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3280052f0 con 0x7fc33415ba70 2024-05-09T10:51:07.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.824+0000 7fc330ff9640 1 -- 172.21.15.98:0/1010656016 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc328003830 con 0x7fc33415ba70 2024-05-09T10:51:07.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.824+0000 7fc33a4f7640 1 -- 172.21.15.98:0/1010656016 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3341f60e0 con 0x7fc33415ba70 2024-05-09T10:51:07.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.824+0000 7fc33a4f7640 1 -- 172.21.15.98:0/1010656016 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3340a99f0 con 0x7fc33415ba70 2024-05-09T10:51:07.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.825+0000 7fc33a4f7640 1 -- 172.21.15.98:0/1010656016 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc334069cb0 con 0x7fc33415ba70 2024-05-09T10:51:07.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.825+0000 7fc330ff9640 1 -- 172.21.15.98:0/1010656016 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc32801a070 con 0x7fc33415ba70 2024-05-09T10:51:07.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.826+0000 7fc330ff9640 1 --2- 172.21.15.98:0/1010656016 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc32006b4d0 0x7fc32006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:07.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.826+0000 7fc330ff9640 1 -- 172.21.15.98:0/1010656016 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3280895d0 con 0x7fc33415ba70 2024-05-09T10:51:07.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.826+0000 7fc333fff640 1 --2- 172.21.15.98:0/1010656016 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc32006b4d0 0x7fc32006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:07.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.826+0000 7fc333fff640 1 --2- 172.21.15.98:0/1010656016 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc32006b4d0 0x7fc32006d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc324034000 tx=0x7fc32402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:07.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:07.830+0000 7fc330ff9640 1 -- 172.21.15.98:0/1010656016 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc328058510 con 0x7fc33415ba70 2024-05-09T10:51:07.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.002+0000 7fc33a4f7640 1 -- 172.21.15.98:0/1010656016 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc33406c5a0 con 0x7fc33415ba70 2024-05-09T10:51:07.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.003+0000 7fc330ff9640 1 -- 172.21.15.98:0/1010656016 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+16682 (secure 0 0 0) 0x7fc328057e10 con 0x7fc33415ba70 2024-05-09T10:51:07.997 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:07.997 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":30,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:51:06.744689+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:51:05.150410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-09T10:51:05.708391+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:51:07.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.005+0000 7fc33a4f7640 1 -- 172.21.15.98:0/1010656016 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc32006b4d0 msgr2=0x7fc32006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:07.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.005+0000 7fc33a4f7640 1 --2- 172.21.15.98:0/1010656016 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc32006b4d0 0x7fc32006d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc324034000 tx=0x7fc32402e000 comp rx=0 tx=0).stop 2024-05-09T10:51:07.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.005+0000 7fc33a4f7640 1 -- 172.21.15.98:0/1010656016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc33415ba70 msgr2=0x7fc334147dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:07.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.005+0000 7fc33a4f7640 1 --2- 172.21.15.98:0/1010656016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc33415ba70 0x7fc334147dc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc32800c3c0 tx=0x7fc32800c880 comp rx=0 tx=0).stop 2024-05-09T10:51:07.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.005+0000 7fc33a4f7640 1 -- 172.21.15.98:0/1010656016 shutdown_connections 2024-05-09T10:51:08.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.005+0000 7fc33a4f7640 1 --2- 172.21.15.98:0/1010656016 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc32006b4d0 0x7fc32006d990 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:08.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.005+0000 7fc33a4f7640 1 --2- 172.21.15.98:0/1010656016 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc33415be50 0x7fc334148300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:08.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.005+0000 7fc33a4f7640 1 --2- 172.21.15.98:0/1010656016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc33415ba70 0x7fc334147dc0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:08.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.005+0000 7fc33a4f7640 1 --2- 172.21.15.98:0/1010656016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3340b43d0 0x7fc334158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:08.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.005+0000 7fc33a4f7640 1 -- 172.21.15.98:0/1010656016 >> 172.21.15.98:0/1010656016 conn(0x7fc33406abe0 msgr2=0x7fc3340b09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:08.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.006+0000 7fc33a4f7640 1 -- 172.21.15.98:0/1010656016 shutdown_connections 2024-05-09T10:51:08.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.006+0000 7fc33a4f7640 1 -- 172.21.15.98:0/1010656016 wait complete. 2024-05-09T10:51:08.010 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-09T10:51:08.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.120+0000 7ff4098b3640 1 Processor -- start 2024-05-09T10:51:08.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.120+0000 7ff4098b3640 1 -- start start 2024-05-09T10:51:08.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff4098b3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4040d2820 0x7ff4040d4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:08.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff4098b3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff4040b2a30 0x7ff4040b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:08.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff4098b3640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff404161830 con 0x7ff404161450 2024-05-09T10:51:08.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff4098b3640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4040b0760 con 0x7ff4040b2a30 2024-05-09T10:51:08.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff4098b3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040cf2f0 con 0x7ff4040d2820 2024-05-09T10:51:08.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff402ffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff404161450 0x7ff4040d2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52784/0 (socket says 172.21.15.98:52784) 2024-05-09T10:51:08.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff402ffd640 1 -- 172.21.15.98:0/2902818950 learned_addr learned my addr 172.21.15.98:0/2902818950 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:08.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff3fbfff640 1 --2- 172.21.15.98:0/2902818950 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4040d2820 0x7ff4040d4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:08.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff4037fe640 1 --2- 172.21.15.98:0/2902818950 >> v2:172.21.15.105:3300/0 conn(0x7ff4040b2a30 0x7ff4040b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:08.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff4027fc640 1 -- 172.21.15.98:0/2902818950 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3450934974 0 0) 0x7ff3f4001110 con 0x7ff404161450 2024-05-09T10:51:08.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff4027fc640 1 -- 172.21.15.98:0/2902818950 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3f0003520 con 0x7ff404161450 2024-05-09T10:51:08.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff4027fc640 1 -- 172.21.15.98:0/2902818950 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3049370876 0 0) 0x7ff3f0003520 con 0x7ff404161450 2024-05-09T10:51:08.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff4027fc640 1 -- 172.21.15.98:0/2902818950 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4040d2820 msgr2=0x7ff4040d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:08.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff4027fc640 1 --2- 172.21.15.98:0/2902818950 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4040d2820 0x7ff4040d4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:08.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff4027fc640 1 -- 172.21.15.98:0/2902818950 >> v2:172.21.15.105:3300/0 conn(0x7ff4040b2a30 msgr2=0x7ff4040b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:08.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff4027fc640 1 --2- 172.21.15.98:0/2902818950 >> v2:172.21.15.105:3300/0 conn(0x7ff4040b2a30 0x7ff4040b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:08.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff4027fc640 1 -- 172.21.15.98:0/2902818950 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4040cf5d0 con 0x7ff404161450 2024-05-09T10:51:08.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff4027fc640 1 -- 172.21.15.98:0/2902818950 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7ff3f4001840 con 0x7ff404161450 2024-05-09T10:51:08.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff4027fc640 1 -- 172.21.15.98:0/2902818950 >> v1:172.21.15.98:6789/0 conn(0x7ff404161450 legacy=0x7ff4040d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:08.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff4027fc640 1 --2- 172.21.15.98:0/2902818950 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4040d2820 0x7ff3f00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:08.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff4027fc640 1 --2- 172.21.15.98:0/2902818950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3f0003f30 0x7ff3f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:08.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff4027fc640 1 --2- 172.21.15.98:0/2902818950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3f0004d40 0x7ff3f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:08.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff4027fc640 1 -- 172.21.15.98:0/2902818950 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040b0760 con 0x7ff3f0003f30 2024-05-09T10:51:08.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff4027fc640 1 -- 172.21.15.98:0/2902818950 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040cf2f0 con 0x7ff3f0004d40 2024-05-09T10:51:08.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff4027fc640 1 -- 172.21.15.98:0/2902818950 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f000ba10 con 0x7ff4040d2820 2024-05-09T10:51:08.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.121+0000 7ff3fbfff640 1 --2- 172.21.15.98:0/2902818950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3f0003f30 0x7ff3f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:08.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.122+0000 7ff402ffd640 1 --2- 172.21.15.98:0/2902818950 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4040d2820 0x7ff3f00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:08.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.122+0000 7ff4037fe640 1 --2- 172.21.15.98:0/2902818950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3f0004d40 0x7ff3f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:08.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.122+0000 7ff3fbfff640 1 -- 172.21.15.98:0/2902818950 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4040d2820 msgr2=0x7ff3f00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:08.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.122+0000 7ff3fbfff640 1 --2- 172.21.15.98:0/2902818950 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4040d2820 0x7ff3f00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:08.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.122+0000 7ff3fbfff640 1 -- 172.21.15.98:0/2902818950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3f0004d40 msgr2=0x7ff3f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:08.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.122+0000 7ff3fbfff640 1 --2- 172.21.15.98:0/2902818950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3f0004d40 0x7ff3f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:08.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.122+0000 7ff3fbfff640 1 -- 172.21.15.98:0/2902818950 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f0003520 con 0x7ff3f0003f30 2024-05-09T10:51:08.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.122+0000 7ff4037fe640 1 --2- 172.21.15.98:0/2902818950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3f0004d40 0x7ff3f00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:08.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.122+0000 7ff402ffd640 1 --2- 172.21.15.98:0/2902818950 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff4040d2820 0x7ff3f00039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:08.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.122+0000 7ff3fbfff640 1 --2- 172.21.15.98:0/2902818950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3f0003f30 0x7ff3f0004800 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009af0 tx=0x7ff3f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb66374799d8f24b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:08.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.123+0000 7ff4027fc640 1 -- 172.21.15.98:0/2902818950 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec036070 con 0x7ff3f0003f30 2024-05-09T10:51:08.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.123+0000 7ff4027fc640 1 -- 172.21.15.98:0/2902818950 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ec031400 con 0x7ff3f0003f30 2024-05-09T10:51:08.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.123+0000 7ff4027fc640 1 -- 172.21.15.98:0/2902818950 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec009690 con 0x7ff3f0003f30 2024-05-09T10:51:08.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.123+0000 7ff4098b3640 1 -- 172.21.15.98:0/2902818950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3f0003f30 msgr2=0x7ff3f0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:08.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.123+0000 7ff4098b3640 1 --2- 172.21.15.98:0/2902818950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3f0003f30 0x7ff3f0004800 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009af0 tx=0x7ff3f0005830 comp rx=0 tx=0).stop 2024-05-09T10:51:08.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.123+0000 7ff4037fe640 1 -- 172.21.15.98:0/2902818950 reap_dead start 2024-05-09T10:51:08.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.123+0000 7ff4098b3640 1 -- 172.21.15.98:0/2902818950 shutdown_connections 2024-05-09T10:51:08.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.123+0000 7ff4098b3640 1 -- 172.21.15.98:0/2902818950 >> 172.21.15.98:0/2902818950 conn(0x7ff4040d3b30 msgr2=0x7ff4040d3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:08.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.123+0000 7ff4098b3640 1 -- 172.21.15.98:0/2902818950 shutdown_connections 2024-05-09T10:51:08.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.123+0000 7ff4098b3640 1 -- 172.21.15.98:0/2902818950 wait complete. 2024-05-09T10:51:08.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.124+0000 7ff4098b3640 1 Processor -- start 2024-05-09T10:51:08.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.124+0000 7ff4098b3640 1 -- start start 2024-05-09T10:51:08.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.125+0000 7ff4098b3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3f0003f30 0x7ff4040cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:08.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.125+0000 7ff4098b3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4041f7e50 0x7ff404158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:08.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.125+0000 7ff4098b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4040b1c10 0x7ff4040b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:08.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.125+0000 7ff4098b3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040d4650 con 0x7ff4041f7e50 2024-05-09T10:51:08.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.125+0000 7ff4098b3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040b09b0 con 0x7ff4040b1c10 2024-05-09T10:51:08.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.125+0000 7ff4098b3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040d47d0 con 0x7ff3f0003f30 2024-05-09T10:51:08.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.125+0000 7ff3fbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4041f7e50 0x7ff404158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:08.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.125+0000 7ff3fbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4041f7e50 0x7ff404158d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48988/0 (socket says 172.21.15.98:48988) 2024-05-09T10:51:08.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.125+0000 7ff3fbfff640 1 -- 172.21.15.98:0/1405567177 learned_addr learned my addr 172.21.15.98:0/1405567177 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:08.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.125+0000 7ff402ffd640 1 --2- 172.21.15.98:0/1405567177 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3f0003f30 0x7ff4040cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:08.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.125+0000 7ff3fbfff640 1 -- 172.21.15.98:0/1405567177 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3f0003f30 msgr2=0x7ff4040cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:08.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.125+0000 7ff3fbfff640 1 --2- 172.21.15.98:0/1405567177 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3f0003f30 0x7ff4040cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:08.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.125+0000 7ff3fbfff640 1 -- 172.21.15.98:0/1405567177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4040b1c10 msgr2=0x7ff4040b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:08.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.125+0000 7ff4037fe640 1 --2- 172.21.15.98:0/1405567177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4040b1c10 0x7ff4040b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:08.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.125+0000 7ff3fbfff640 1 --2- 172.21.15.98:0/1405567177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4040b1c10 0x7ff4040b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:08.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.125+0000 7ff3fbfff640 1 -- 172.21.15.98:0/1405567177 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4041f82b0 con 0x7ff4041f7e50 2024-05-09T10:51:08.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.125+0000 7ff4037fe640 1 --2- 172.21.15.98:0/1405567177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4040b1c10 0x7ff4040b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:08.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.126+0000 7ff3fbfff640 1 --2- 172.21.15.98:0/1405567177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4041f7e50 0x7ff404158d80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec034040 tx=0x7ff3ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:08.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.126+0000 7ff400ff9640 1 -- 172.21.15.98:0/1405567177 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec040020 con 0x7ff4041f7e50 2024-05-09T10:51:08.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.126+0000 7ff400ff9640 1 -- 172.21.15.98:0/1405567177 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ec0038d0 con 0x7ff4041f7e50 2024-05-09T10:51:08.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.126+0000 7ff4098b3640 1 -- 172.21.15.98:0/1405567177 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4041592c0 con 0x7ff4041f7e50 2024-05-09T10:51:08.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.126+0000 7ff400ff9640 1 -- 172.21.15.98:0/1405567177 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec036040 con 0x7ff4041f7e50 2024-05-09T10:51:08.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.126+0000 7ff4098b3640 1 -- 172.21.15.98:0/1405567177 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4040b95b0 con 0x7ff4041f7e50 2024-05-09T10:51:08.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.127+0000 7ff4098b3640 1 -- 172.21.15.98:0/1405567177 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4040a1ae0 con 0x7ff4041f7e50 2024-05-09T10:51:08.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.127+0000 7ff400ff9640 1 -- 172.21.15.98:0/1405567177 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff3ec043020 con 0x7ff4041f7e50 2024-05-09T10:51:08.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.128+0000 7ff400ff9640 1 --2- 172.21.15.98:0/1405567177 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff3e406b4d0 0x7ff3e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:08.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.128+0000 7ff400ff9640 1 -- 172.21.15.98:0/1405567177 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3ec0aa1b0 con 0x7ff4041f7e50 2024-05-09T10:51:08.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.128+0000 7ff402ffd640 1 --2- 172.21.15.98:0/1405567177 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff3e406b4d0 0x7ff3e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:08.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.128+0000 7ff402ffd640 1 --2- 172.21.15.98:0/1405567177 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff3e406b4d0 0x7ff3e406d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4007c70 tx=0x7ff3f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:08.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.132+0000 7ff400ff9640 1 -- 172.21.15.98:0/1405567177 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff3ec04a070 con 0x7ff4041f7e50 2024-05-09T10:51:08.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.338+0000 7ff4098b3640 1 -- 172.21.15.98:0/1405567177 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff4040b27f0 con 0x7ff4041f7e50 2024-05-09T10:51:08.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.400+0000 7ff400ff9640 1 -- 172.21.15.98:0/1405567177 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v12) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff3ec0790c0 con 0x7ff4041f7e50 2024-05-09T10:51:08.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.403+0000 7ff4098b3640 1 -- 172.21.15.98:0/1405567177 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff3e406b4d0 msgr2=0x7ff3e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:08.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.403+0000 7ff4098b3640 1 --2- 172.21.15.98:0/1405567177 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff3e406b4d0 0x7ff3e406d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4007c70 tx=0x7ff3f4009040 comp rx=0 tx=0).stop 2024-05-09T10:51:08.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.403+0000 7ff4098b3640 1 -- 172.21.15.98:0/1405567177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4041f7e50 msgr2=0x7ff404158d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:08.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.403+0000 7ff4098b3640 1 --2- 172.21.15.98:0/1405567177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4041f7e50 0x7ff404158d80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec034040 tx=0x7ff3ec004730 comp rx=0 tx=0).stop 2024-05-09T10:51:08.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.403+0000 7ff4098b3640 1 -- 172.21.15.98:0/1405567177 shutdown_connections 2024-05-09T10:51:08.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.403+0000 7ff4098b3640 1 --2- 172.21.15.98:0/1405567177 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff3e406b4d0 0x7ff3e406d990 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:08.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.403+0000 7ff4098b3640 1 --2- 172.21.15.98:0/1405567177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4040b1c10 0x7ff4040b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:08.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.403+0000 7ff4098b3640 1 --2- 172.21.15.98:0/1405567177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4041f7e50 0x7ff404158d80 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:08.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.403+0000 7ff4098b3640 1 --2- 172.21.15.98:0/1405567177 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3f0003f30 0x7ff4040cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:08.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.403+0000 7ff4098b3640 1 -- 172.21.15.98:0/1405567177 >> 172.21.15.98:0/1405567177 conn(0x7ff4040d3b30 msgr2=0x7ff4040b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:08.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.403+0000 7ff4098b3640 1 -- 172.21.15.98:0/1405567177 shutdown_connections 2024-05-09T10:51:08.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.403+0000 7ff4098b3640 1 -- 172.21.15.98:0/1405567177 wait complete. 2024-05-09T10:51:08.409 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-09T10:51:08.409 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-09T10:51:08.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.518+0000 7f97520f5640 1 Processor -- start 2024-05-09T10:51:08.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.518+0000 7f97520f5640 1 -- start start 2024-05-09T10:51:08.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.519+0000 7f97520f5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f974c0ca530 0x7f974c15f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:08.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.519+0000 7f97520f5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f974c0cee80 0x7f974c0c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:08.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.519+0000 7f97520f5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f974c0ae960 con 0x7f974c15a8f0 2024-05-09T10:51:08.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.519+0000 7f97520f5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f974c15f5d0 con 0x7f974c0ca530 2024-05-09T10:51:08.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.519+0000 7f97520f5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974c0cb540 con 0x7f974c0cee80 2024-05-09T10:51:08.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.519+0000 7f974b7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f974c15a8f0 0x7f974c0ce770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52800/0 (socket says 172.21.15.98:52800) 2024-05-09T10:51:08.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.519+0000 7f974b7fe640 1 -- 172.21.15.98:0/1602232036 learned_addr learned my addr 172.21.15.98:0/1602232036 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:08.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.519+0000 7f974affd640 1 --2- 172.21.15.98:0/1602232036 >> v2:172.21.15.105:3300/0 conn(0x7f974c0ca530 0x7f974c15f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:08.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.520+0000 7f974bfff640 1 --2- 172.21.15.98:0/1602232036 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f974c0cee80 0x7f974c0c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:08.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.520+0000 7f974a7fc640 1 -- 172.21.15.98:0/1602232036 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1358464589 0 0) 0x7f974c0ae960 con 0x7f974c15a8f0 2024-05-09T10:51:08.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.520+0000 7f974a7fc640 1 -- 172.21.15.98:0/1602232036 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9734003520 con 0x7f974c15a8f0 2024-05-09T10:51:08.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.520+0000 7f974affd640 1 -- 172.21.15.98:0/1602232036 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f974c0cee80 msgr2=0x7f974c0c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:08.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.520+0000 7f974affd640 1 --2- 172.21.15.98:0/1602232036 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f974c0cee80 0x7f974c0c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:08.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.520+0000 7f974affd640 1 -- 172.21.15.98:0/1602232036 >> v1:172.21.15.98:6789/0 conn(0x7f974c15a8f0 legacy=0x7f974c0ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:08.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.520+0000 7f974a7fc640 1 -- 172.21.15.98:0/1602232036 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3263368881 0 0) 0x7f9734003520 con 0x7f974c15a8f0 2024-05-09T10:51:08.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.520+0000 7f974affd640 1 -- 172.21.15.98:0/1602232036 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974c0ca9c0 con 0x7f974c0ca530 2024-05-09T10:51:08.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.520+0000 7f974bfff640 1 --2- 172.21.15.98:0/1602232036 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f974c0cee80 0x7f974c0c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:51:08.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.521+0000 7f974affd640 1 --2- 172.21.15.98:0/1602232036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f974c0ca530 0x7f974c15f090 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f97400097a0 tx=0x7f974002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d1f1df397544d1f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:08.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.521+0000 7f974a7fc640 1 -- 172.21.15.98:0/1602232036 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9740037070 con 0x7f974c0ca530 2024-05-09T10:51:08.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.521+0000 7f974a7fc640 1 -- 172.21.15.98:0/1602232036 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974002aba0 con 0x7f974c0ca530 2024-05-09T10:51:08.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.521+0000 7f974a7fc640 1 -- 172.21.15.98:0/1602232036 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973c0019f0 con 0x7f974c0ca530 2024-05-09T10:51:08.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.521+0000 7f97520f5640 1 -- 172.21.15.98:0/1602232036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f974c0ca530 msgr2=0x7f974c15f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:08.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.521+0000 7f97520f5640 1 --2- 172.21.15.98:0/1602232036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f974c0ca530 0x7f974c15f090 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f97400097a0 tx=0x7f974002a110 comp rx=0 tx=0).stop 2024-05-09T10:51:08.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.521+0000 7f97520f5640 1 -- 172.21.15.98:0/1602232036 shutdown_connections 2024-05-09T10:51:08.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.521+0000 7f97520f5640 1 --2- 172.21.15.98:0/1602232036 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f974c0cee80 0x7f974c0c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:08.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.521+0000 7f97520f5640 1 --2- 172.21.15.98:0/1602232036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f974c0ca530 0x7f974c15f090 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:08.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.521+0000 7f97520f5640 1 -- 172.21.15.98:0/1602232036 >> 172.21.15.98:0/1602232036 conn(0x7f974c0cf760 msgr2=0x7f974c0cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:08.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.521+0000 7f97520f5640 1 -- 172.21.15.98:0/1602232036 shutdown_connections 2024-05-09T10:51:08.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.521+0000 7f97520f5640 1 -- 172.21.15.98:0/1602232036 wait complete. 2024-05-09T10:51:08.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.522+0000 7f97520f5640 1 Processor -- start 2024-05-09T10:51:08.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.523+0000 7f97520f5640 1 -- start start 2024-05-09T10:51:08.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.523+0000 7f97520f5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f974c0cee80 0x7f974c15c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:08.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.523+0000 7f97520f5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f974c15a8f0 0x7f974c0b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:08.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.523+0000 7f97520f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f974c155730 0x7f974c15cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:08.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.523+0000 7f97520f5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c0cb260 con 0x7f974c0cee80 2024-05-09T10:51:08.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.523+0000 7f97520f5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c0caf80 con 0x7f974c155730 2024-05-09T10:51:08.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.523+0000 7f97520f5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974c15acd0 con 0x7f974c15a8f0 2024-05-09T10:51:08.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.524+0000 7f974b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f974c0cee80 0x7f974c15c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:08.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.524+0000 7f974b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f974c0cee80 0x7f974c15c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48996/0 (socket says 172.21.15.98:48996) 2024-05-09T10:51:08.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.524+0000 7f974b7fe640 1 -- 172.21.15.98:0/2656288794 learned_addr learned my addr 172.21.15.98:0/2656288794 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:08.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.524+0000 7f974affd640 1 --2- 172.21.15.98:0/2656288794 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f974c15a8f0 0x7f974c0b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:08.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.524+0000 7f974bfff640 1 --2- 172.21.15.98:0/2656288794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f974c155730 0x7f974c15cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:08.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.524+0000 7f974b7fe640 1 -- 172.21.15.98:0/2656288794 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f974c15a8f0 msgr2=0x7f974c0b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:08.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.524+0000 7f974b7fe640 1 --2- 172.21.15.98:0/2656288794 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f974c15a8f0 0x7f974c0b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:08.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.524+0000 7f974b7fe640 1 -- 172.21.15.98:0/2656288794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f974c155730 msgr2=0x7f974c15cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:08.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.524+0000 7f974b7fe640 1 --2- 172.21.15.98:0/2656288794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f974c155730 0x7f974c15cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:08.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.524+0000 7f974b7fe640 1 -- 172.21.15.98:0/2656288794 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974c154060 con 0x7f974c0cee80 2024-05-09T10:51:08.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.524+0000 7f974bfff640 1 --2- 172.21.15.98:0/2656288794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f974c155730 0x7f974c15cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:08.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.524+0000 7f974b7fe640 1 --2- 172.21.15.98:0/2656288794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f974c0cee80 0x7f974c15c590 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f973c002230 tx=0x7f973c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:08.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.524+0000 7f9733fff640 1 -- 172.21.15.98:0/2656288794 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973c017070 con 0x7f974c0cee80 2024-05-09T10:51:08.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.524+0000 7f9733fff640 1 -- 172.21.15.98:0/2656288794 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973c00ad40 con 0x7f974c0cee80 2024-05-09T10:51:08.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.524+0000 7f97520f5640 1 -- 172.21.15.98:0/2656288794 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974c154340 con 0x7f974c0cee80 2024-05-09T10:51:08.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.524+0000 7f9733fff640 1 -- 172.21.15.98:0/2656288794 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973c012620 con 0x7f974c0cee80 2024-05-09T10:51:08.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.524+0000 7f97520f5640 1 -- 172.21.15.98:0/2656288794 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f974c154900 con 0x7f974c0cee80 2024-05-09T10:51:08.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.525+0000 7f97520f5640 1 -- 172.21.15.98:0/2656288794 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f974c0c5cd0 con 0x7f974c0cee80 2024-05-09T10:51:08.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.526+0000 7f9733fff640 1 -- 172.21.15.98:0/2656288794 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f973c01a070 con 0x7f974c0cee80 2024-05-09T10:51:08.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.526+0000 7f9733fff640 1 --2- 172.21.15.98:0/2656288794 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f973806b4d0 0x7f973806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:08.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.526+0000 7f9733fff640 1 -- 172.21.15.98:0/2656288794 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f973c08a030 con 0x7f974c0cee80 2024-05-09T10:51:08.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.526+0000 7f974affd640 1 --2- 172.21.15.98:0/2656288794 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f973806b4d0 0x7f973806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:08.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.527+0000 7f974affd640 1 --2- 172.21.15.98:0/2656288794 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f973806b4d0 0x7f973806d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9740034000 tx=0x7f974002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:08.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.530+0000 7f9733fff640 1 -- 172.21.15.98:0/2656288794 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f974c0c5cd0 con 0x7f974c0cee80 2024-05-09T10:51:08.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:08.736+0000 7f97520f5640 1 -- 172.21.15.98:0/2656288794 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f974c072c80 con 0x7f974c0cee80 2024-05-09T10:51:09.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.413+0000 7f9733fff640 1 -- 172.21.15.98:0/2656288794 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v13) v1 ==== 122+0+0 (secure 0 0 0) 0x7f973c0588f0 con 0x7f974c0cee80 2024-05-09T10:51:09.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.416+0000 7f97520f5640 1 -- 172.21.15.98:0/2656288794 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f973806b4d0 msgr2=0x7f973806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:09.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.416+0000 7f97520f5640 1 --2- 172.21.15.98:0/2656288794 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f973806b4d0 0x7f973806d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9740034000 tx=0x7f974002e000 comp rx=0 tx=0).stop 2024-05-09T10:51:09.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.416+0000 7f97520f5640 1 -- 172.21.15.98:0/2656288794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f974c0cee80 msgr2=0x7f974c15c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:09.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.416+0000 7f97520f5640 1 --2- 172.21.15.98:0/2656288794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f974c0cee80 0x7f974c15c590 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f973c002230 tx=0x7f973c00a430 comp rx=0 tx=0).stop 2024-05-09T10:51:09.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.416+0000 7f97520f5640 1 -- 172.21.15.98:0/2656288794 shutdown_connections 2024-05-09T10:51:09.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.416+0000 7f97520f5640 1 --2- 172.21.15.98:0/2656288794 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f973806b4d0 0x7f973806d990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:09.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.416+0000 7f97520f5640 1 --2- 172.21.15.98:0/2656288794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f974c155730 0x7f974c15cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:09.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.416+0000 7f97520f5640 1 --2- 172.21.15.98:0/2656288794 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f974c15a8f0 0x7f974c0b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:09.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.416+0000 7f97520f5640 1 --2- 172.21.15.98:0/2656288794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f974c0cee80 0x7f974c15c590 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:09.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.416+0000 7f97520f5640 1 -- 172.21.15.98:0/2656288794 >> 172.21.15.98:0/2656288794 conn(0x7f974c0cf760 msgr2=0x7f974c0cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:09.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.416+0000 7f97520f5640 1 -- 172.21.15.98:0/2656288794 shutdown_connections 2024-05-09T10:51:09.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.416+0000 7f97520f5640 1 -- 172.21.15.98:0/2656288794 wait complete. 2024-05-09T10:51:09.422 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-09T10:51:09.422 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T10:51:09.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.534+0000 7fc1216bc640 1 Processor -- start 2024-05-09T10:51:09.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.534+0000 7fc1216bc640 1 -- start start 2024-05-09T10:51:09.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.535+0000 7fc1216bc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc11c15de40 0x7fc11c0d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:09.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.535+0000 7fc1216bc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc11c0d0b80 0x7fc11c0b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:09.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.535+0000 7fc1216bc640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11c15e220 con 0x7fc11c0cf250 2024-05-09T10:51:09.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.535+0000 7fc1216bc640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc11c0d1790 con 0x7fc11c15de40 2024-05-09T10:51:09.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.535+0000 7fc1216bc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc11c0d19e0 con 0x7fc11c0d0b80 2024-05-09T10:51:09.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.535+0000 7fc11affd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc11c15de40 0x7fc11c0d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:09.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.535+0000 7fc1127fc640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc11c0cf250 0x7fc11c0d0470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52802/0 (socket says 172.21.15.98:52802) 2024-05-09T10:51:09.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.535+0000 7fc1127fc640 1 -- 172.21.15.98:0/1476896991 learned_addr learned my addr 172.21.15.98:0/1476896991 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:09.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.535+0000 7fc11b7fe640 1 --2- 172.21.15.98:0/1476896991 >> v2:172.21.15.105:3300/0 conn(0x7fc11c0d0b80 0x7fc11c0b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:09.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.535+0000 7fc11affd640 1 -- 172.21.15.98:0/1476896991 >> v2:172.21.15.105:3300/0 conn(0x7fc11c0d0b80 msgr2=0x7fc11c0b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:09.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.535+0000 7fc11affd640 1 --2- 172.21.15.98:0/1476896991 >> v2:172.21.15.105:3300/0 conn(0x7fc11c0d0b80 0x7fc11c0b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:09.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.535+0000 7fc11affd640 1 -- 172.21.15.98:0/1476896991 >> v1:172.21.15.98:6789/0 conn(0x7fc11c0cf250 legacy=0x7fc11c0d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:09.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.535+0000 7fc11affd640 1 -- 172.21.15.98:0/1476896991 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c0d1cc0 con 0x7fc11c15de40 2024-05-09T10:51:09.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.535+0000 7fc11a7fc640 1 -- 172.21.15.98:0/1476896991 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4065507058 0 0) 0x7fc11c15e220 con 0x7fc11c0cf250 2024-05-09T10:51:09.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.536+0000 7fc11affd640 1 --2- 172.21.15.98:0/1476896991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc11c15de40 0x7fc11c0d27a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0ce490 tx=0x7fc10c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9535d489af757 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:09.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.536+0000 7fc11a7fc640 1 -- 172.21.15.98:0/1476896991 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c037070 con 0x7fc11c15de40 2024-05-09T10:51:09.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.536+0000 7fc11a7fc640 1 -- 172.21.15.98:0/1476896991 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10c032440 con 0x7fc11c15de40 2024-05-09T10:51:09.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.536+0000 7fc11a7fc640 1 -- 172.21.15.98:0/1476896991 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c032810 con 0x7fc11c15de40 2024-05-09T10:51:09.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.537+0000 7fc1216bc640 1 -- 172.21.15.98:0/1476896991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc11c15de40 msgr2=0x7fc11c0d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:09.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.537+0000 7fc1216bc640 1 --2- 172.21.15.98:0/1476896991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc11c15de40 0x7fc11c0d27a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0ce490 tx=0x7fc10c02a1a0 comp rx=0 tx=0).stop 2024-05-09T10:51:09.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.537+0000 7fc1216bc640 1 -- 172.21.15.98:0/1476896991 shutdown_connections 2024-05-09T10:51:09.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.537+0000 7fc1216bc640 1 --2- 172.21.15.98:0/1476896991 >> v2:172.21.15.105:3300/0 conn(0x7fc11c0d0b80 0x7fc11c0b6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:09.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.537+0000 7fc1216bc640 1 --2- 172.21.15.98:0/1476896991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc11c15de40 0x7fc11c0d27a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:09.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.537+0000 7fc1216bc640 1 -- 172.21.15.98:0/1476896991 >> 172.21.15.98:0/1476896991 conn(0x7fc11c09ce90 msgr2=0x7fc11c0d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:09.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.537+0000 7fc1216bc640 1 -- 172.21.15.98:0/1476896991 shutdown_connections 2024-05-09T10:51:09.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.537+0000 7fc1216bc640 1 -- 172.21.15.98:0/1476896991 wait complete. 2024-05-09T10:51:09.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.538+0000 7fc1216bc640 1 Processor -- start 2024-05-09T10:51:09.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.538+0000 7fc1216bc640 1 -- start start 2024-05-09T10:51:09.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.539+0000 7fc1216bc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc11c0cf250 0x7fc11c0b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:09.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.539+0000 7fc1216bc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc11c0d0b80 0x7fc11c1e8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:09.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.539+0000 7fc1216bc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc11c15de40 0x7fc11c1e8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:09.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.539+0000 7fc1216bc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c0d4680 con 0x7fc11c0cf250 2024-05-09T10:51:09.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.539+0000 7fc1216bc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c0d2280 con 0x7fc11c0d0b80 2024-05-09T10:51:09.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.539+0000 7fc1216bc640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c0d4800 con 0x7fc11c15de40 2024-05-09T10:51:09.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.539+0000 7fc11affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc11c0cf250 0x7fc11c0b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:09.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.539+0000 7fc11affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc11c0cf250 0x7fc11c0b6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49008/0 (socket says 172.21.15.98:49008) 2024-05-09T10:51:09.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.539+0000 7fc11affd640 1 -- 172.21.15.98:0/953896847 learned_addr learned my addr 172.21.15.98:0/953896847 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:09.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.539+0000 7fc1127fc640 1 --2- 172.21.15.98:0/953896847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc11c0d0b80 0x7fc11c1e8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:09.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.539+0000 7fc11b7fe640 1 --2- 172.21.15.98:0/953896847 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc11c15de40 0x7fc11c1e8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:09.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.539+0000 7fc11affd640 1 -- 172.21.15.98:0/953896847 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc11c15de40 msgr2=0x7fc11c1e8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:09.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.539+0000 7fc11affd640 1 --2- 172.21.15.98:0/953896847 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc11c15de40 0x7fc11c1e8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:09.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.539+0000 7fc11affd640 1 -- 172.21.15.98:0/953896847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc11c0d0b80 msgr2=0x7fc11c1e8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:09.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.539+0000 7fc11affd640 1 --2- 172.21.15.98:0/953896847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc11c0d0b80 0x7fc11c1e8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:09.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.539+0000 7fc11affd640 1 -- 172.21.15.98:0/953896847 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c0cb1c0 con 0x7fc11c0cf250 2024-05-09T10:51:09.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.540+0000 7fc11affd640 1 --2- 172.21.15.98:0/953896847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc11c0cf250 0x7fc11c0b6ac0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc10c000c00 tx=0x7fc10c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:09.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.540+0000 7fc113fff640 1 -- 172.21.15.98:0/953896847 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c037040 con 0x7fc11c0cf250 2024-05-09T10:51:09.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.540+0000 7fc113fff640 1 -- 172.21.15.98:0/953896847 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10c042070 con 0x7fc11c0cf250 2024-05-09T10:51:09.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.540+0000 7fc1216bc640 1 -- 172.21.15.98:0/953896847 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c0cb4a0 con 0x7fc11c0cf250 2024-05-09T10:51:09.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.540+0000 7fc113fff640 1 -- 172.21.15.98:0/953896847 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c030bd0 con 0x7fc11c0cf250 2024-05-09T10:51:09.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.540+0000 7fc1216bc640 1 -- 172.21.15.98:0/953896847 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c0cba60 con 0x7fc11c0cf250 2024-05-09T10:51:09.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.541+0000 7fc1216bc640 1 -- 172.21.15.98:0/953896847 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c0d30d0 con 0x7fc11c0cf250 2024-05-09T10:51:09.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.541+0000 7fc113fff640 1 -- 172.21.15.98:0/953896847 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc10c047460 con 0x7fc11c0cf250 2024-05-09T10:51:09.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.542+0000 7fc113fff640 1 --2- 172.21.15.98:0/953896847 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc0fc06b4d0 0x7fc0fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:09.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.542+0000 7fc1127fc640 1 --2- 172.21.15.98:0/953896847 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc0fc06b4d0 0x7fc0fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:09.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.542+0000 7fc113fff640 1 -- 172.21.15.98:0/953896847 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc10c0aae00 con 0x7fc11c0cf250 2024-05-09T10:51:09.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.542+0000 7fc1127fc640 1 --2- 172.21.15.98:0/953896847 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc0fc06b4d0 0x7fc0fc06d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc11c1ea350 tx=0x7fc104007b90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:09.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.546+0000 7fc113fff640 1 -- 172.21.15.98:0/953896847 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc11c0d30d0 con 0x7fc11c0cf250 2024-05-09T10:51:09.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.754+0000 7fc1216bc640 1 -- 172.21.15.98:0/953896847 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc11c06d0b0 con 0x7fc11c0cf250 2024-05-09T10:51:09.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.755+0000 7fc113fff640 1 -- 172.21.15.98:0/953896847 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fc10c032e20 con 0x7fc11c0cf250 2024-05-09T10:51:09.748 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:09.748 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2332373437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2332373437},{"type":"v1","addr":"172.21.15.105:6834","nonce":2332373437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4849,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/4092257628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":4092257628},{"type":"v1","addr":"172.21.15.98:6836","nonce":4092257628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4855,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3957569958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3957569958},{"type":"v1","addr":"172.21.15.105:6835","nonce":3957569958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T10:51:06.745057+0000","modified":"2024-05-09T10:51:08.738161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5127":{"gid":5127,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/164638690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":164638690},{"type":"v1","addr":"172.21.15.98:6837","nonce":164638690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-09T10:51:09.748 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 13 2024-05-09T10:51:09.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.757+0000 7fc1216bc640 1 -- 172.21.15.98:0/953896847 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc0fc06b4d0 msgr2=0x7fc0fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:09.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.757+0000 7fc1216bc640 1 --2- 172.21.15.98:0/953896847 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc0fc06b4d0 0x7fc0fc06d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc11c1ea350 tx=0x7fc104007b90 comp rx=0 tx=0).stop 2024-05-09T10:51:09.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.757+0000 7fc1216bc640 1 -- 172.21.15.98:0/953896847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc11c0cf250 msgr2=0x7fc11c0b6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:09.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.757+0000 7fc1216bc640 1 --2- 172.21.15.98:0/953896847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc11c0cf250 0x7fc11c0b6ac0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc10c000c00 tx=0x7fc10c0046c0 comp rx=0 tx=0).stop 2024-05-09T10:51:09.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.757+0000 7fc1216bc640 1 -- 172.21.15.98:0/953896847 shutdown_connections 2024-05-09T10:51:09.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.757+0000 7fc1216bc640 1 --2- 172.21.15.98:0/953896847 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc0fc06b4d0 0x7fc0fc06d990 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:09.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.757+0000 7fc1216bc640 1 --2- 172.21.15.98:0/953896847 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc11c15de40 0x7fc11c1e8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:09.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.757+0000 7fc1216bc640 1 --2- 172.21.15.98:0/953896847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc11c0d0b80 0x7fc11c1e8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:09.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.757+0000 7fc1216bc640 1 --2- 172.21.15.98:0/953896847 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc11c0cf250 0x7fc11c0b6ac0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:09.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.757+0000 7fc1216bc640 1 -- 172.21.15.98:0/953896847 >> 172.21.15.98:0/953896847 conn(0x7fc11c09ce90 msgr2=0x7fc11c0b77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:09.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.757+0000 7fc1216bc640 1 -- 172.21.15.98:0/953896847 shutdown_connections 2024-05-09T10:51:09.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.757+0000 7fc1216bc640 1 -- 172.21.15.98:0/953896847 wait complete. 2024-05-09T10:51:09.761 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:51:09.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.867+0000 7fd2cecac640 1 Processor -- start 2024-05-09T10:51:09.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.867+0000 7fd2cecac640 1 -- start start 2024-05-09T10:51:09.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.868+0000 7fd2cecac640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd2c815ba70 0x7fd2c80cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:09.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.868+0000 7fd2cecac640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2c815be50 0x7fd2c80cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:09.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.868+0000 7fd2cecac640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd2c80b43d0 0x7fd2c80cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:09.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.868+0000 7fd2cecac640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c80b3610 con 0x7fd2c80b43d0 2024-05-09T10:51:09.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.868+0000 7fd2cecac640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c80b38f0 con 0x7fd2c815ba70 2024-05-09T10:51:09.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.868+0000 7fd2cecac640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80b3bd0 con 0x7fd2c815be50 2024-05-09T10:51:09.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.868+0000 7fd2cd249640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd2c80b43d0 0x7fd2c80cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:09.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.868+0000 7fd2cd249640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd2c80b43d0 0x7fd2c80cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49016/0 (socket says 172.21.15.98:49016) 2024-05-09T10:51:09.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.868+0000 7fd2cd249640 1 -- 172.21.15.98:0/1422295898 learned_addr learned my addr 172.21.15.98:0/1422295898 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:09.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.868+0000 7fd2cd249640 1 -- 172.21.15.98:0/1422295898 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2c815be50 msgr2=0x7fd2c80cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:09.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.868+0000 7fd2bffff640 1 --2- 172.21.15.98:0/1422295898 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2c815be50 0x7fd2c80cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:09.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.868+0000 7fd2cca48640 1 --2- 172.21.15.98:0/1422295898 >> v2:172.21.15.105:3300/0 conn(0x7fd2c815ba70 0x7fd2c80cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:09.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.868+0000 7fd2cd249640 1 --2- 172.21.15.98:0/1422295898 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2c815be50 0x7fd2c80cf440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:09.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.868+0000 7fd2cd249640 1 -- 172.21.15.98:0/1422295898 >> v2:172.21.15.105:3300/0 conn(0x7fd2c815ba70 msgr2=0x7fd2c80cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:09.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.868+0000 7fd2cd249640 1 --2- 172.21.15.98:0/1422295898 >> v2:172.21.15.105:3300/0 conn(0x7fd2c815ba70 0x7fd2c80cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:09.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.868+0000 7fd2cd249640 1 -- 172.21.15.98:0/1422295898 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c80cce60 con 0x7fd2c80b43d0 2024-05-09T10:51:09.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.868+0000 7fd2cca48640 1 --2- 172.21.15.98:0/1422295898 >> v2:172.21.15.105:3300/0 conn(0x7fd2c815ba70 0x7fd2c80cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:09.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.869+0000 7fd2cd249640 1 --2- 172.21.15.98:0/1422295898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2c80b43d0 0x7fd2c80cf980 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80cd220 tx=0x7fd2c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac5617e1df2fbb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:09.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.869+0000 7fd2bf7fe640 1 -- 172.21.15.98:0/1422295898 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0037070 con 0x7fd2c80b43d0 2024-05-09T10:51:09.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.869+0000 7fd2bf7fe640 1 -- 172.21.15.98:0/1422295898 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c0032440 con 0x7fd2c80b43d0 2024-05-09T10:51:09.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.869+0000 7fd2bf7fe640 1 -- 172.21.15.98:0/1422295898 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0032810 con 0x7fd2c80b43d0 2024-05-09T10:51:09.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.870+0000 7fd2cecac640 1 -- 172.21.15.98:0/1422295898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2c80b43d0 msgr2=0x7fd2c80cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:09.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.870+0000 7fd2cecac640 1 --2- 172.21.15.98:0/1422295898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2c80b43d0 0x7fd2c80cf980 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80cd220 tx=0x7fd2c002a1e0 comp rx=0 tx=0).stop 2024-05-09T10:51:09.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.870+0000 7fd2cecac640 1 -- 172.21.15.98:0/1422295898 shutdown_connections 2024-05-09T10:51:09.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.870+0000 7fd2cecac640 1 --2- 172.21.15.98:0/1422295898 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2c80b43d0 0x7fd2c80cf980 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:09.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.870+0000 7fd2cecac640 1 --2- 172.21.15.98:0/1422295898 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2c815be50 0x7fd2c80cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:09.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.870+0000 7fd2cecac640 1 --2- 172.21.15.98:0/1422295898 >> v2:172.21.15.105:3300/0 conn(0x7fd2c815ba70 0x7fd2c80cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:09.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.870+0000 7fd2cecac640 1 -- 172.21.15.98:0/1422295898 >> 172.21.15.98:0/1422295898 conn(0x7fd2c806abe0 msgr2=0x7fd2c806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:09.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.870+0000 7fd2cecac640 1 -- 172.21.15.98:0/1422295898 shutdown_connections 2024-05-09T10:51:09.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.870+0000 7fd2cecac640 1 -- 172.21.15.98:0/1422295898 wait complete. 2024-05-09T10:51:09.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.871+0000 7fd2cecac640 1 Processor -- start 2024-05-09T10:51:09.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.871+0000 7fd2cecac640 1 -- start start 2024-05-09T10:51:09.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.872+0000 7fd2cecac640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2c80b43d0 0x7fd2c8155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:09.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.872+0000 7fd2cecac640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2c815ba70 0x7fd2c80ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:09.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.872+0000 7fd2cecac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2c815be50 0x7fd2c81511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:09.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.872+0000 7fd2cecac640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80aefa0 con 0x7fd2c80b43d0 2024-05-09T10:51:09.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.872+0000 7fd2cecac640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80aee20 con 0x7fd2c815be50 2024-05-09T10:51:09.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.872+0000 7fd2cecac640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80af120 con 0x7fd2c815ba70 2024-05-09T10:51:09.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.872+0000 7fd2cca48640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2c80b43d0 0x7fd2c8155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:09.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.872+0000 7fd2cca48640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2c80b43d0 0x7fd2c8155770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49022/0 (socket says 172.21.15.98:49022) 2024-05-09T10:51:09.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.872+0000 7fd2cca48640 1 -- 172.21.15.98:0/3621708850 learned_addr learned my addr 172.21.15.98:0/3621708850 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:09.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.872+0000 7fd2bffff640 1 --2- 172.21.15.98:0/3621708850 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2c815ba70 0x7fd2c80ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:09.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.872+0000 7fd2cd249640 1 --2- 172.21.15.98:0/3621708850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2c815be50 0x7fd2c81511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:09.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.872+0000 7fd2cca48640 1 -- 172.21.15.98:0/3621708850 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2c815ba70 msgr2=0x7fd2c80ade60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:09.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.872+0000 7fd2cca48640 1 --2- 172.21.15.98:0/3621708850 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2c815ba70 0x7fd2c80ade60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:09.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.872+0000 7fd2cca48640 1 -- 172.21.15.98:0/3621708850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2c815be50 msgr2=0x7fd2c81511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:09.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.872+0000 7fd2cca48640 1 --2- 172.21.15.98:0/3621708850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2c815be50 0x7fd2c81511d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:09.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.872+0000 7fd2cca48640 1 -- 172.21.15.98:0/3621708850 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c8155e50 con 0x7fd2c80b43d0 2024-05-09T10:51:09.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.872+0000 7fd2cca48640 1 --2- 172.21.15.98:0/3621708850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2c80b43d0 0x7fd2c8155770 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400cf10 tx=0x7fd2c400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:09.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.872+0000 7fd2bd7fa640 1 -- 172.21.15.98:0/3621708850 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c401a070 con 0x7fd2c80b43d0 2024-05-09T10:51:09.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.873+0000 7fd2bd7fa640 1 -- 172.21.15.98:0/3621708850 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c400d840 con 0x7fd2c80b43d0 2024-05-09T10:51:09.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.873+0000 7fd2bd7fa640 1 -- 172.21.15.98:0/3621708850 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c400dc10 con 0x7fd2c80b43d0 2024-05-09T10:51:09.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.873+0000 7fd2cecac640 1 -- 172.21.15.98:0/3621708850 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c8156130 con 0x7fd2c80b43d0 2024-05-09T10:51:09.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.873+0000 7fd2cecac640 1 -- 172.21.15.98:0/3621708850 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c81566f0 con 0x7fd2c80b43d0 2024-05-09T10:51:09.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.874+0000 7fd2cecac640 1 -- 172.21.15.98:0/3621708850 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c8069cb0 con 0x7fd2c80b43d0 2024-05-09T10:51:09.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.874+0000 7fd2bd7fa640 1 -- 172.21.15.98:0/3621708850 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd2c401d070 con 0x7fd2c80b43d0 2024-05-09T10:51:09.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.874+0000 7fd2bd7fa640 1 --2- 172.21.15.98:0/3621708850 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd2b406b4d0 0x7fd2b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:09.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.875+0000 7fd2bd7fa640 1 -- 172.21.15.98:0/3621708850 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd2c408a9a0 con 0x7fd2c80b43d0 2024-05-09T10:51:09.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.875+0000 7fd2bffff640 1 --2- 172.21.15.98:0/3621708850 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd2b406b4d0 0x7fd2b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:09.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.875+0000 7fd2bffff640 1 --2- 172.21.15.98:0/3621708850 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd2b406b4d0 0x7fd2b406d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80d0480 tx=0x7fd2b8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:09.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:09.879+0000 7fd2bd7fa640 1 -- 172.21.15.98:0/3621708850 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd2c4059290 con 0x7fd2c80b43d0 2024-05-09T10:51:10.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.047+0000 7fd2cecac640 1 -- 172.21.15.98:0/3621708850 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2c806c5a0 con 0x7fd2c80b43d0 2024-05-09T10:51:10.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.048+0000 7fd2bd7fa640 1 -- 172.21.15.98:0/3621708850 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+16669 (secure 0 0 0) 0x7fd2c405d330 con 0x7fd2c80b43d0 2024-05-09T10:51:10.042 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:10.042 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":31,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:51:08.400895+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:51:05.150410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-09T10:51:05.708391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:51:10.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.050+0000 7fd2cecac640 1 -- 172.21.15.98:0/3621708850 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd2b406b4d0 msgr2=0x7fd2b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:10.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.050+0000 7fd2cecac640 1 --2- 172.21.15.98:0/3621708850 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd2b406b4d0 0x7fd2b406d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80d0480 tx=0x7fd2b8008040 comp rx=0 tx=0).stop 2024-05-09T10:51:10.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.050+0000 7fd2cecac640 1 -- 172.21.15.98:0/3621708850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2c80b43d0 msgr2=0x7fd2c8155770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:10.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.050+0000 7fd2cecac640 1 --2- 172.21.15.98:0/3621708850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2c80b43d0 0x7fd2c8155770 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400cf10 tx=0x7fd2c400a710 comp rx=0 tx=0).stop 2024-05-09T10:51:10.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.050+0000 7fd2cecac640 1 -- 172.21.15.98:0/3621708850 shutdown_connections 2024-05-09T10:51:10.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.050+0000 7fd2cecac640 1 --2- 172.21.15.98:0/3621708850 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd2b406b4d0 0x7fd2b406d990 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.050+0000 7fd2cecac640 1 --2- 172.21.15.98:0/3621708850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2c815be50 0x7fd2c81511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.050+0000 7fd2cecac640 1 --2- 172.21.15.98:0/3621708850 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2c815ba70 0x7fd2c80ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.050+0000 7fd2cecac640 1 --2- 172.21.15.98:0/3621708850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2c80b43d0 0x7fd2c8155770 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.050+0000 7fd2cecac640 1 -- 172.21.15.98:0/3621708850 >> 172.21.15.98:0/3621708850 conn(0x7fd2c806abe0 msgr2=0x7fd2c80af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:10.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.050+0000 7fd2cecac640 1 -- 172.21.15.98:0/3621708850 shutdown_connections 2024-05-09T10:51:10.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.050+0000 7fd2cecac640 1 -- 172.21.15.98:0/3621708850 wait complete. 2024-05-09T10:51:10.055 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-09T10:51:10.055 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T10:51:10.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.162+0000 7f0023722640 1 Processor -- start 2024-05-09T10:51:10.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.162+0000 7f0023722640 1 -- start start 2024-05-09T10:51:10.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.163+0000 7f0023722640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f001c15ba70 0x7f001c0cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:10.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.163+0000 7f0023722640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f001c0b43d0 0x7f001c06cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:10.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.163+0000 7f0023722640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f001c0b3610 con 0x7f001c15be50 2024-05-09T10:51:10.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.163+0000 7f0023722640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f001c0b38f0 con 0x7f001c15ba70 2024-05-09T10:51:10.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.163+0000 7f0023722640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f001c0b3bd0 con 0x7f001c0b43d0 2024-05-09T10:51:10.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.163+0000 7f00214be640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f001c15ba70 0x7f001c0cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:10.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.163+0000 7f00214be640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f001c15ba70 0x7f001c0cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49032/0 (socket says 172.21.15.98:49032) 2024-05-09T10:51:10.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.163+0000 7f00214be640 1 -- 172.21.15.98:0/2169621783 learned_addr learned my addr 172.21.15.98:0/2169621783 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:10.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.163+0000 7f00214be640 1 -- 172.21.15.98:0/2169621783 >> v2:172.21.15.105:3300/0 conn(0x7f001c0b43d0 msgr2=0x7f001c06cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:10.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.163+0000 7f0021cbf640 1 --2- 172.21.15.98:0/2169621783 >> v2:172.21.15.105:3300/0 conn(0x7f001c0b43d0 0x7f001c06cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:10.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.163+0000 7f00214be640 1 --2- 172.21.15.98:0/2169621783 >> v2:172.21.15.105:3300/0 conn(0x7f001c0b43d0 0x7f001c06cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.163+0000 7f00214be640 1 -- 172.21.15.98:0/2169621783 >> v1:172.21.15.105:6789/0 conn(0x7f001c15be50 legacy=0x7f001c0cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:10.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.163+0000 7f00214be640 1 -- 172.21.15.98:0/2169621783 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001c0cce60 con 0x7f001c15ba70 2024-05-09T10:51:10.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.163+0000 7f0021cbf640 1 --2- 172.21.15.98:0/2169621783 >> v2:172.21.15.105:3300/0 conn(0x7f001c0b43d0 0x7f001c06cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:10.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.164+0000 7f00214be640 1 --2- 172.21.15.98:0/2169621783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f001c15ba70 0x7f001c0cc920 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f001c06e710 tx=0x7f001402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950d9309082e4ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:10.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.164+0000 7f001bfff640 1 -- 172.21.15.98:0/2169621783 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0014037070 con 0x7f001c15ba70 2024-05-09T10:51:10.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.164+0000 7f001bfff640 1 -- 172.21.15.98:0/2169621783 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0014032440 con 0x7f001c15ba70 2024-05-09T10:51:10.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.164+0000 7f001bfff640 1 -- 172.21.15.98:0/2169621783 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0014032810 con 0x7f001c15ba70 2024-05-09T10:51:10.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.165+0000 7f0023722640 1 -- 172.21.15.98:0/2169621783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f001c15ba70 msgr2=0x7f001c0cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:10.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.165+0000 7f0023722640 1 --2- 172.21.15.98:0/2169621783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f001c15ba70 0x7f001c0cc920 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f001c06e710 tx=0x7f001402a230 comp rx=0 tx=0).stop 2024-05-09T10:51:10.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.165+0000 7f0023722640 1 -- 172.21.15.98:0/2169621783 shutdown_connections 2024-05-09T10:51:10.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.165+0000 7f0023722640 1 --2- 172.21.15.98:0/2169621783 >> v2:172.21.15.105:3300/0 conn(0x7f001c0b43d0 0x7f001c06cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.165+0000 7f0023722640 1 --2- 172.21.15.98:0/2169621783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f001c15ba70 0x7f001c0cc920 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.165+0000 7f0023722640 1 -- 172.21.15.98:0/2169621783 >> 172.21.15.98:0/2169621783 conn(0x7f001c06abe0 msgr2=0x7f001c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:10.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.165+0000 7f0023722640 1 -- 172.21.15.98:0/2169621783 shutdown_connections 2024-05-09T10:51:10.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.165+0000 7f0023722640 1 -- 172.21.15.98:0/2169621783 wait complete. 2024-05-09T10:51:10.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.166+0000 7f0023722640 1 Processor -- start 2024-05-09T10:51:10.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.166+0000 7f0023722640 1 -- start start 2024-05-09T10:51:10.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.167+0000 7f0023722640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f001c0b43d0 0x7f001c156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:10.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.167+0000 7f0023722640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f001c15ba70 0x7f001c1e6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:10.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.167+0000 7f0023722640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f001c15be50 0x7f001c143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:10.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.167+0000 7f0023722640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c0cfcd0 con 0x7f001c15be50 2024-05-09T10:51:10.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.167+0000 7f0023722640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c0cfb50 con 0x7f001c0b43d0 2024-05-09T10:51:10.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.167+0000 7f0023722640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f001c0cfe50 con 0x7f001c15ba70 2024-05-09T10:51:10.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.167+0000 7f0021cbf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f001c15be50 0x7f001c143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:10.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.167+0000 7f0021cbf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f001c15be50 0x7f001c143fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49042/0 (socket says 172.21.15.98:49042) 2024-05-09T10:51:10.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.167+0000 7f0021cbf640 1 -- 172.21.15.98:0/2401448294 learned_addr learned my addr 172.21.15.98:0/2401448294 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:10.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.167+0000 7f0020cbd640 1 --2- 172.21.15.98:0/2401448294 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f001c15ba70 0x7f001c1e6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:10.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.167+0000 7f00214be640 1 --2- 172.21.15.98:0/2401448294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f001c0b43d0 0x7f001c156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:10.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.167+0000 7f0021cbf640 1 -- 172.21.15.98:0/2401448294 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f001c15ba70 msgr2=0x7f001c1e6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:10.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.167+0000 7f0021cbf640 1 --2- 172.21.15.98:0/2401448294 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f001c15ba70 0x7f001c1e6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.167+0000 7f0021cbf640 1 -- 172.21.15.98:0/2401448294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f001c0b43d0 msgr2=0x7f001c156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:10.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.167+0000 7f0021cbf640 1 --2- 172.21.15.98:0/2401448294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f001c0b43d0 0x7f001c156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.167+0000 7f0021cbf640 1 -- 172.21.15.98:0/2401448294 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001c144510 con 0x7f001c15be50 2024-05-09T10:51:10.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.167+0000 7f00214be640 1 --2- 172.21.15.98:0/2401448294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f001c0b43d0 0x7f001c156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:10.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.167+0000 7f0021cbf640 1 --2- 172.21.15.98:0/2401448294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f001c15be50 0x7f001c143fd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f000c00cef0 tx=0x7f000c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:10.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.167+0000 7f0019ffb640 1 -- 172.21.15.98:0/2401448294 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c01a070 con 0x7f001c15be50 2024-05-09T10:51:10.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.167+0000 7f0019ffb640 1 -- 172.21.15.98:0/2401448294 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000c00d860 con 0x7f001c15be50 2024-05-09T10:51:10.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.167+0000 7f0019ffb640 1 -- 172.21.15.98:0/2401448294 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c00dc50 con 0x7f001c15be50 2024-05-09T10:51:10.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.167+0000 7f0023722640 1 -- 172.21.15.98:0/2401448294 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001c1555b0 con 0x7f001c15be50 2024-05-09T10:51:10.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.168+0000 7f0023722640 1 -- 172.21.15.98:0/2401448294 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001c155b70 con 0x7f001c15be50 2024-05-09T10:51:10.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.169+0000 7f0023722640 1 -- 172.21.15.98:0/2401448294 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001c069cb0 con 0x7f001c15be50 2024-05-09T10:51:10.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.169+0000 7f0019ffb640 1 -- 172.21.15.98:0/2401448294 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f000c01d070 con 0x7f001c15be50 2024-05-09T10:51:10.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.169+0000 7f0019ffb640 1 --2- 172.21.15.98:0/2401448294 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f000806b4d0 0x7f000806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:10.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.170+0000 7f0019ffb640 1 -- 172.21.15.98:0/2401448294 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f000c08a990 con 0x7f001c15be50 2024-05-09T10:51:10.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.170+0000 7f00214be640 1 --2- 172.21.15.98:0/2401448294 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f000806b4d0 0x7f000806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:10.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.170+0000 7f00214be640 1 --2- 172.21.15.98:0/2401448294 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f000806b4d0 0x7f000806d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0014034000 tx=0x7f001402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:10.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.174+0000 7f0019ffb640 1 -- 172.21.15.98:0/2401448294 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f000c059310 con 0x7f001c15be50 2024-05-09T10:51:10.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.340+0000 7f0023722640 1 -- 172.21.15.98:0/2401448294 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f001c06c5a0 con 0x7f000806b4d0 2024-05-09T10:51:10.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.346+0000 7f0019ffb640 1 -- 172.21.15.98:0/2401448294 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246634 (secure 0 0 0) 0x7f001407e380 con 0x7f000806b4d0 2024-05-09T10:51:10.342 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:10.343 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-05-09T10:51:10.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.348+0000 7f0023722640 1 -- 172.21.15.98:0/2401448294 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f000806b4d0 msgr2=0x7f000806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:10.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.348+0000 7f0023722640 1 --2- 172.21.15.98:0/2401448294 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f000806b4d0 0x7f000806d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0014034000 tx=0x7f001402e000 comp rx=0 tx=0).stop 2024-05-09T10:51:10.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.348+0000 7f0023722640 1 -- 172.21.15.98:0/2401448294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f001c15be50 msgr2=0x7f001c143fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:10.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.348+0000 7f0023722640 1 --2- 172.21.15.98:0/2401448294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f001c15be50 0x7f001c143fd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f000c00cef0 tx=0x7f000c00a710 comp rx=0 tx=0).stop 2024-05-09T10:51:10.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.349+0000 7f0023722640 1 -- 172.21.15.98:0/2401448294 shutdown_connections 2024-05-09T10:51:10.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.349+0000 7f0023722640 1 --2- 172.21.15.98:0/2401448294 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f000806b4d0 0x7f000806d990 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.349+0000 7f0023722640 1 --2- 172.21.15.98:0/2401448294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f001c15be50 0x7f001c143fd0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.349+0000 7f0023722640 1 --2- 172.21.15.98:0/2401448294 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f001c15ba70 0x7f001c1e6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.349+0000 7f0023722640 1 --2- 172.21.15.98:0/2401448294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f001c0b43d0 0x7f001c156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.349+0000 7f0023722640 1 -- 172.21.15.98:0/2401448294 >> 172.21.15.98:0/2401448294 conn(0x7f001c06abe0 msgr2=0x7f001c15c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:10.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.349+0000 7f0023722640 1 -- 172.21.15.98:0/2401448294 shutdown_connections 2024-05-09T10:51:10.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.349+0000 7f0023722640 1 -- 172.21.15.98:0/2401448294 wait complete. 2024-05-09T10:51:10.353 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-05-09T10:51:09.374919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219880,"kb_used_data":3688,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754840,"statfs":{"total":773094113280,"available":772868956160,"internally_reserved":0,"allocated":3776512,"data_stored":2149144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000826"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393755+0000","last_change":"2024-05-09T10:51:06.393755+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393755+0000","last_undegraded":"2024-05-09T10:51:06.393755+0000","last_fullsized":"2024-05-09T10:51:06.393755+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392812+0000","last_change":"2024-05-09T10:51:06.392812+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392812+0000","last_undegraded":"2024-05-09T10:51:06.392812+0000","last_fullsized":"2024-05-09T10:51:06.392812+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388265+0000","last_change":"2024-05-09T10:51:06.388265+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388265+0000","last_undegraded":"2024-05-09T10:51:06.388265+0000","last_fullsized":"2024-05-09T10:51:06.388265+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393261+0000","last_change":"2024-05-09T10:51:06.393261+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393261+0000","last_undegraded":"2024-05-09T10:51:06.393261+0000","last_fullsized":"2024-05-09T10:51:06.393261+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.389283+0000","last_change":"2024-05-09T10:51:06.389283+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.389283+0000","last_undegraded":"2024-05-09T10:51:06.389283+0000","last_fullsized":"2024-05-09T10:51:06.389283+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.394709+0000","last_change":"2024-05-09T10:51:06.394709+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.394709+0000","last_undegraded":"2024-05-09T10:51:06.394709+0000","last_fullsized":"2024-05-09T10:51:06.394709+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395480+0000","last_change":"2024-05-09T10:51:06.395480+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395480+0000","last_undegraded":"2024-05-09T10:51:06.395480+0000","last_fullsized":"2024-05-09T10:51:06.395480+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.398792+0000","last_change":"2024-05-09T10:51:06.398792+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.398792+0000","last_undegraded":"2024-05-09T10:51:06.398792+0000","last_fullsized":"2024-05-09T10:51:06.398792+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.394178+0000","last_change":"2024-05-09T10:51:06.394178+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.394178+0000","last_undegraded":"2024-05-09T10:51:06.394178+0000","last_fullsized":"2024-05-09T10:51:06.394178+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392823+0000","last_change":"2024-05-09T10:51:06.392823+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392823+0000","last_undegraded":"2024-05-09T10:51:06.392823+0000","last_fullsized":"2024-05-09T10:51:06.392823+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.396067+0000","last_change":"2024-05-09T10:51:06.396067+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.396067+0000","last_undegraded":"2024-05-09T10:51:06.396067+0000","last_fullsized":"2024-05-09T10:51:06.396067+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388701+0000","last_change":"2024-05-09T10:51:06.388701+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388701+0000","last_undegraded":"2024-05-09T10:51:06.388701+0000","last_fullsized":"2024-05-09T10:51:06.388701+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388834+0000","last_change":"2024-05-09T10:51:06.388834+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388834+0000","last_undegraded":"2024-05-09T10:51:06.388834+0000","last_fullsized":"2024-05-09T10:51:06.388834+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393806+0000","last_change":"2024-05-09T10:51:06.393806+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393806+0000","last_undegraded":"2024-05-09T10:51:06.393806+0000","last_fullsized":"2024-05-09T10:51:06.393806+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388334+0000","last_change":"2024-05-09T10:51:06.388334+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388334+0000","last_undegraded":"2024-05-09T10:51:06.388334+0000","last_fullsized":"2024-05-09T10:51:06.388334+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.396634+0000","last_change":"2024-05-09T10:51:06.396634+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.396634+0000","last_undegraded":"2024-05-09T10:51:06.396634+0000","last_fullsized":"2024-05-09T10:51:06.396634+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.397070+0000","last_change":"2024-05-09T10:51:06.397070+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.397070+0000","last_undegraded":"2024-05-09T10:51:06.397070+0000","last_fullsized":"2024-05-09T10:51:06.397070+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392398+0000","last_change":"2024-05-09T10:51:06.392398+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392398+0000","last_undegraded":"2024-05-09T10:51:06.392398+0000","last_fullsized":"2024-05-09T10:51:06.392398+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395656+0000","last_change":"2024-05-09T10:51:06.395656+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395656+0000","last_undegraded":"2024-05-09T10:51:06.395656+0000","last_fullsized":"2024-05-09T10:51:06.395656+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388193+0000","last_change":"2024-05-09T10:51:06.388193+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388193+0000","last_undegraded":"2024-05-09T10:51:06.388193+0000","last_fullsized":"2024-05-09T10:51:06.388193+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.398904+0000","last_change":"2024-05-09T10:51:06.398904+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.398904+0000","last_undegraded":"2024-05-09T10:51:06.398904+0000","last_fullsized":"2024-05-09T10:51:06.398904+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.396580+0000","last_change":"2024-05-09T10:51:06.396580+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.396580+0000","last_undegraded":"2024-05-09T10:51:06.396580+0000","last_fullsized":"2024-05-09T10:51:06.396580+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.391128+0000","last_change":"2024-05-09T10:51:06.391128+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.391128+0000","last_undegraded":"2024-05-09T10:51:06.391128+0000","last_fullsized":"2024-05-09T10:51:06.391128+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.387807+0000","last_change":"2024-05-09T10:51:06.387807+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.387807+0000","last_undegraded":"2024-05-09T10:51:06.387807+0000","last_fullsized":"2024-05-09T10:51:06.387807+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.391876+0000","last_change":"2024-05-09T10:51:06.391876+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.391876+0000","last_undegraded":"2024-05-09T10:51:06.391876+0000","last_fullsized":"2024-05-09T10:51:06.391876+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.391884+0000","last_change":"2024-05-09T10:51:06.391884+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.391884+0000","last_undegraded":"2024-05-09T10:51:06.391884+0000","last_fullsized":"2024-05-09T10:51:06.391884+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392175+0000","last_change":"2024-05-09T10:51:06.392175+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392175+0000","last_undegraded":"2024-05-09T10:51:06.392175+0000","last_fullsized":"2024-05-09T10:51:06.392175+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388098+0000","last_change":"2024-05-09T10:51:06.388098+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388098+0000","last_undegraded":"2024-05-09T10:51:06.388098+0000","last_fullsized":"2024-05-09T10:51:06.388098+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.398517+0000","last_change":"2024-05-09T10:51:06.398517+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.398517+0000","last_undegraded":"2024-05-09T10:51:06.398517+0000","last_fullsized":"2024-05-09T10:51:06.398517+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.396453+0000","last_change":"2024-05-09T10:51:06.396453+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.396453+0000","last_undegraded":"2024-05-09T10:51:06.396453+0000","last_fullsized":"2024-05-09T10:51:06.396453+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392904+0000","last_change":"2024-05-09T10:51:06.392904+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392904+0000","last_undegraded":"2024-05-09T10:51:06.392904+0000","last_fullsized":"2024-05-09T10:51:06.392904+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.398378+0000","last_change":"2024-05-09T10:51:06.398378+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.398378+0000","last_undegraded":"2024-05-09T10:51:06.398378+0000","last_fullsized":"2024-05-09T10:51:06.398378+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.390548+0000","last_change":"2024-05-09T10:51:06.390548+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.390548+0000","last_undegraded":"2024-05-09T10:51:06.390548+0000","last_fullsized":"2024-05-09T10:51:06.390548+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.403221+0000","last_change":"2024-05-09T10:51:06.399065+0000","last_active":"2024-05-09T10:51:06.403221+0000","last_peered":"2024-05-09T10:51:06.403221+0000","last_clean":"2024-05-09T10:51:06.403221+0000","last_became_active":"2024-05-09T10:51:06.398631+0000","last_became_peered":"2024-05-09T10:51:06.398631+0000","last_unstale":"2024-05-09T10:51:06.403221+0000","last_undegraded":"2024-05-09T10:51:06.403221+0000","last_fullsized":"2024-05-09T10:51:06.403221+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:21.607813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393181+0000","last_change":"2024-05-09T10:51:06.393181+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393181+0000","last_undegraded":"2024-05-09T10:51:06.393181+0000","last_fullsized":"2024-05-09T10:51:06.393181+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.405979+0000","last_change":"2024-05-09T10:51:06.405043+0000","last_active":"2024-05-09T10:51:06.405979+0000","last_peered":"2024-05-09T10:51:06.405979+0000","last_clean":"2024-05-09T10:51:06.405979+0000","last_became_active":"2024-05-09T10:51:06.404375+0000","last_became_peered":"2024-05-09T10:51:06.404375+0000","last_unstale":"2024-05-09T10:51:06.405979+0000","last_undegraded":"2024-05-09T10:51:06.405979+0000","last_fullsized":"2024-05-09T10:51:06.405979+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:03.704175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.399481+0000","last_change":"2024-05-09T10:51:06.399481+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.399481+0000","last_undegraded":"2024-05-09T10:51:06.399481+0000","last_fullsized":"2024-05-09T10:51:06.399481+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406256+0000","last_change":"2024-05-09T10:51:06.403754+0000","last_active":"2024-05-09T10:51:06.406256+0000","last_peered":"2024-05-09T10:51:06.406256+0000","last_clean":"2024-05-09T10:51:06.406256+0000","last_became_active":"2024-05-09T10:51:06.403476+0000","last_became_peered":"2024-05-09T10:51:06.403476+0000","last_unstale":"2024-05-09T10:51:06.406256+0000","last_undegraded":"2024-05-09T10:51:06.406256+0000","last_fullsized":"2024-05-09T10:51:06.406256+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:48:18.069829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.391713+0000","last_change":"2024-05-09T10:51:06.391713+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.391713+0000","last_undegraded":"2024-05-09T10:51:06.391713+0000","last_fullsized":"2024-05-09T10:51:06.391713+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.410669+0000","last_change":"2024-05-09T10:51:06.401479+0000","last_active":"2024-05-09T10:51:06.410669+0000","last_peered":"2024-05-09T10:51:06.410669+0000","last_clean":"2024-05-09T10:51:06.410669+0000","last_became_active":"2024-05-09T10:51:06.401079+0000","last_became_peered":"2024-05-09T10:51:06.401079+0000","last_unstale":"2024-05-09T10:51:06.410669+0000","last_undegraded":"2024-05-09T10:51:06.410669+0000","last_fullsized":"2024-05-09T10:51:06.410669+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:56.016304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395508+0000","last_change":"2024-05-09T10:51:06.395508+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395508+0000","last_undegraded":"2024-05-09T10:51:06.395508+0000","last_fullsized":"2024-05-09T10:51:06.395508+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408534+0000","last_change":"2024-05-09T10:51:06.403964+0000","last_active":"2024-05-09T10:51:06.408534+0000","last_peered":"2024-05-09T10:51:06.408534+0000","last_clean":"2024-05-09T10:51:06.408534+0000","last_became_active":"2024-05-09T10:51:06.403691+0000","last_became_peered":"2024-05-09T10:51:06.403691+0000","last_unstale":"2024-05-09T10:51:06.408534+0000","last_undegraded":"2024-05-09T10:51:06.408534+0000","last_fullsized":"2024-05-09T10:51:06.408534+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:26.083086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.387999+0000","last_change":"2024-05-09T10:51:06.387999+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.387999+0000","last_undegraded":"2024-05-09T10:51:06.387999+0000","last_fullsized":"2024-05-09T10:51:06.387999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408477+0000","last_change":"2024-05-09T10:51:06.403128+0000","last_active":"2024-05-09T10:51:06.408477+0000","last_peered":"2024-05-09T10:51:06.408477+0000","last_clean":"2024-05-09T10:51:06.408477+0000","last_became_active":"2024-05-09T10:51:06.402841+0000","last_became_peered":"2024-05-09T10:51:06.402841+0000","last_unstale":"2024-05-09T10:51:06.408477+0000","last_undegraded":"2024-05-09T10:51:06.408477+0000","last_fullsized":"2024-05-09T10:51:06.408477+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:33:59.577821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.397348+0000","last_change":"2024-05-09T10:51:06.397348+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.397348+0000","last_undegraded":"2024-05-09T10:51:06.397348+0000","last_fullsized":"2024-05-09T10:51:06.397348+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.414048+0000","last_change":"2024-05-09T10:51:06.410970+0000","last_active":"2024-05-09T10:51:06.414048+0000","last_peered":"2024-05-09T10:51:06.414048+0000","last_clean":"2024-05-09T10:51:06.414048+0000","last_became_active":"2024-05-09T10:51:06.410608+0000","last_became_peered":"2024-05-09T10:51:06.410608+0000","last_unstale":"2024-05-09T10:51:06.414048+0000","last_undegraded":"2024-05-09T10:51:06.414048+0000","last_fullsized":"2024-05-09T10:51:06.414048+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:30:44.917770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.391151+0000","last_change":"2024-05-09T10:51:06.391151+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.391151+0000","last_undegraded":"2024-05-09T10:51:06.391151+0000","last_fullsized":"2024-05-09T10:51:06.391151+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408469+0000","last_change":"2024-05-09T10:51:06.403392+0000","last_active":"2024-05-09T10:51:06.408469+0000","last_peered":"2024-05-09T10:51:06.408469+0000","last_clean":"2024-05-09T10:51:06.408469+0000","last_became_active":"2024-05-09T10:51:06.403111+0000","last_became_peered":"2024-05-09T10:51:06.403111+0000","last_unstale":"2024-05-09T10:51:06.408469+0000","last_undegraded":"2024-05-09T10:51:06.408469+0000","last_fullsized":"2024-05-09T10:51:06.408469+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:56.016304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388231+0000","last_change":"2024-05-09T10:51:06.388231+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388231+0000","last_undegraded":"2024-05-09T10:51:06.388231+0000","last_fullsized":"2024-05-09T10:51:06.388231+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408273+0000","last_change":"2024-05-09T10:51:06.406795+0000","last_active":"2024-05-09T10:51:06.408273+0000","last_peered":"2024-05-09T10:51:06.408273+0000","last_clean":"2024-05-09T10:51:06.408273+0000","last_became_active":"2024-05-09T10:51:06.406516+0000","last_became_peered":"2024-05-09T10:51:06.406516+0000","last_unstale":"2024-05-09T10:51:06.408273+0000","last_undegraded":"2024-05-09T10:51:06.408273+0000","last_fullsized":"2024-05-09T10:51:06.408273+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:29.722036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393382+0000","last_change":"2024-05-09T10:51:06.393382+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393382+0000","last_undegraded":"2024-05-09T10:51:06.393382+0000","last_fullsized":"2024-05-09T10:51:06.393382+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.401982+0000","last_change":"2024-05-09T10:51:06.399077+0000","last_active":"2024-05-09T10:51:06.401982+0000","last_peered":"2024-05-09T10:51:06.401982+0000","last_clean":"2024-05-09T10:51:06.401982+0000","last_became_active":"2024-05-09T10:51:06.398760+0000","last_became_peered":"2024-05-09T10:51:06.398760+0000","last_unstale":"2024-05-09T10:51:06.401982+0000","last_undegraded":"2024-05-09T10:51:06.401982+0000","last_fullsized":"2024-05-09T10:51:06.401982+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:51.058627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.394715+0000","last_change":"2024-05-09T10:51:06.394715+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.394715+0000","last_undegraded":"2024-05-09T10:51:06.394715+0000","last_fullsized":"2024-05-09T10:51:06.394715+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408196+0000","last_change":"2024-05-09T10:51:06.402648+0000","last_active":"2024-05-09T10:51:06.408196+0000","last_peered":"2024-05-09T10:51:06.408196+0000","last_clean":"2024-05-09T10:51:06.408196+0000","last_became_active":"2024-05-09T10:51:06.402315+0000","last_became_peered":"2024-05-09T10:51:06.402315+0000","last_unstale":"2024-05-09T10:51:06.408196+0000","last_undegraded":"2024-05-09T10:51:06.408196+0000","last_fullsized":"2024-05-09T10:51:06.408196+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:29.722036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.389652+0000","last_change":"2024-05-09T10:51:06.389652+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.389652+0000","last_undegraded":"2024-05-09T10:51:06.389652+0000","last_fullsized":"2024-05-09T10:51:06.389652+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406190+0000","last_change":"2024-05-09T10:51:06.403735+0000","last_active":"2024-05-09T10:51:06.406190+0000","last_peered":"2024-05-09T10:51:06.406190+0000","last_clean":"2024-05-09T10:51:06.406190+0000","last_became_active":"2024-05-09T10:51:06.403469+0000","last_became_peered":"2024-05-09T10:51:06.403469+0000","last_unstale":"2024-05-09T10:51:06.406190+0000","last_undegraded":"2024-05-09T10:51:06.406190+0000","last_fullsized":"2024-05-09T10:51:06.406190+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:03.704175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393445+0000","last_change":"2024-05-09T10:51:06.393445+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393445+0000","last_undegraded":"2024-05-09T10:51:06.393445+0000","last_fullsized":"2024-05-09T10:51:06.393445+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.409330+0000","last_change":"2024-05-09T10:51:06.406522+0000","last_active":"2024-05-09T10:51:06.409330+0000","last_peered":"2024-05-09T10:51:06.409330+0000","last_clean":"2024-05-09T10:51:06.409330+0000","last_became_active":"2024-05-09T10:51:06.406242+0000","last_became_peered":"2024-05-09T10:51:06.406242+0000","last_unstale":"2024-05-09T10:51:06.409330+0000","last_undegraded":"2024-05-09T10:51:06.409330+0000","last_fullsized":"2024-05-09T10:51:06.409330+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:21.592545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395214+0000","last_change":"2024-05-09T10:51:06.395214+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395214+0000","last_undegraded":"2024-05-09T10:51:06.395214+0000","last_fullsized":"2024-05-09T10:51:06.395214+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406193+0000","last_change":"2024-05-09T10:51:06.402262+0000","last_active":"2024-05-09T10:51:06.406193+0000","last_peered":"2024-05-09T10:51:06.406193+0000","last_clean":"2024-05-09T10:51:06.406193+0000","last_became_active":"2024-05-09T10:51:06.401936+0000","last_became_peered":"2024-05-09T10:51:06.401936+0000","last_unstale":"2024-05-09T10:51:06.406193+0000","last_undegraded":"2024-05-09T10:51:06.406193+0000","last_fullsized":"2024-05-09T10:51:06.406193+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:05:03.035522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.397979+0000","last_change":"2024-05-09T10:51:06.397979+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.397979+0000","last_undegraded":"2024-05-09T10:51:06.397979+0000","last_fullsized":"2024-05-09T10:51:06.397979+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406601+0000","last_change":"2024-05-09T10:51:06.405127+0000","last_active":"2024-05-09T10:51:06.406601+0000","last_peered":"2024-05-09T10:51:06.406601+0000","last_clean":"2024-05-09T10:51:06.406601+0000","last_became_active":"2024-05-09T10:51:06.404682+0000","last_became_peered":"2024-05-09T10:51:06.404682+0000","last_unstale":"2024-05-09T10:51:06.406601+0000","last_undegraded":"2024-05-09T10:51:06.406601+0000","last_fullsized":"2024-05-09T10:51:06.406601+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:21.393250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392434+0000","last_change":"2024-05-09T10:51:06.392434+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392434+0000","last_undegraded":"2024-05-09T10:51:06.392434+0000","last_fullsized":"2024-05-09T10:51:06.392434+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.402616+0000","last_change":"2024-05-09T10:51:06.401431+0000","last_active":"2024-05-09T10:51:06.402616+0000","last_peered":"2024-05-09T10:51:06.402616+0000","last_clean":"2024-05-09T10:51:06.402616+0000","last_became_active":"2024-05-09T10:51:06.401097+0000","last_became_peered":"2024-05-09T10:51:06.401097+0000","last_unstale":"2024-05-09T10:51:06.402616+0000","last_undegraded":"2024-05-09T10:51:06.402616+0000","last_fullsized":"2024-05-09T10:51:06.402616+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:05:03.035522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.398088+0000","last_change":"2024-05-09T10:51:06.398088+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.398088+0000","last_undegraded":"2024-05-09T10:51:06.398088+0000","last_fullsized":"2024-05-09T10:51:06.398088+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406321+0000","last_change":"2024-05-09T10:51:06.403074+0000","last_active":"2024-05-09T10:51:06.406321+0000","last_peered":"2024-05-09T10:51:06.406321+0000","last_clean":"2024-05-09T10:51:06.406321+0000","last_became_active":"2024-05-09T10:51:06.402721+0000","last_became_peered":"2024-05-09T10:51:06.402721+0000","last_unstale":"2024-05-09T10:51:06.406321+0000","last_undegraded":"2024-05-09T10:51:06.406321+0000","last_fullsized":"2024-05-09T10:51:06.406321+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:03.704175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388885+0000","last_change":"2024-05-09T10:51:06.388885+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388885+0000","last_undegraded":"2024-05-09T10:51:06.388885+0000","last_fullsized":"2024-05-09T10:51:06.388885+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.409346+0000","last_change":"2024-05-09T10:51:06.407069+0000","last_active":"2024-05-09T10:51:06.409346+0000","last_peered":"2024-05-09T10:51:06.409346+0000","last_clean":"2024-05-09T10:51:06.409346+0000","last_became_active":"2024-05-09T10:51:06.406780+0000","last_became_peered":"2024-05-09T10:51:06.406780+0000","last_unstale":"2024-05-09T10:51:06.409346+0000","last_undegraded":"2024-05-09T10:51:06.409346+0000","last_fullsized":"2024-05-09T10:51:06.409346+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:01.273826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392827+0000","last_change":"2024-05-09T10:51:06.392827+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392827+0000","last_undegraded":"2024-05-09T10:51:06.392827+0000","last_fullsized":"2024-05-09T10:51:06.392827+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.411703+0000","last_change":"2024-05-09T10:51:06.411247+0000","last_active":"2024-05-09T10:51:06.411703+0000","last_peered":"2024-05-09T10:51:06.411703+0000","last_clean":"2024-05-09T10:51:06.411703+0000","last_became_active":"2024-05-09T10:51:06.411001+0000","last_became_peered":"2024-05-09T10:51:06.411001+0000","last_unstale":"2024-05-09T10:51:06.411703+0000","last_undegraded":"2024-05-09T10:51:06.411703+0000","last_fullsized":"2024-05-09T10:51:06.411703+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:35.165729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388204+0000","last_change":"2024-05-09T10:51:06.388204+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388204+0000","last_undegraded":"2024-05-09T10:51:06.388204+0000","last_fullsized":"2024-05-09T10:51:06.388204+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408650+0000","last_change":"2024-05-09T10:51:06.405035+0000","last_active":"2024-05-09T10:51:06.408650+0000","last_peered":"2024-05-09T10:51:06.408650+0000","last_clean":"2024-05-09T10:51:06.408650+0000","last_became_active":"2024-05-09T10:51:06.404712+0000","last_became_peered":"2024-05-09T10:51:06.404712+0000","last_unstale":"2024-05-09T10:51:06.408650+0000","last_undegraded":"2024-05-09T10:51:06.408650+0000","last_fullsized":"2024-05-09T10:51:06.408650+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:47:47.684710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388110+0000","last_change":"2024-05-09T10:51:06.388110+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388110+0000","last_undegraded":"2024-05-09T10:51:06.388110+0000","last_fullsized":"2024-05-09T10:51:06.388110+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.401280+0000","last_change":"2024-05-09T10:51:06.396633+0000","last_active":"2024-05-09T10:51:06.401280+0000","last_peered":"2024-05-09T10:51:06.401280+0000","last_clean":"2024-05-09T10:51:06.401280+0000","last_became_active":"2024-05-09T10:51:06.396277+0000","last_became_peered":"2024-05-09T10:51:06.396277+0000","last_unstale":"2024-05-09T10:51:06.401280+0000","last_undegraded":"2024-05-09T10:51:06.401280+0000","last_fullsized":"2024-05-09T10:51:06.401280+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:29.685531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388416+0000","last_change":"2024-05-09T10:51:06.388416+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388416+0000","last_undegraded":"2024-05-09T10:51:06.388416+0000","last_fullsized":"2024-05-09T10:51:06.388416+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408562+0000","last_change":"2024-05-09T10:51:06.405148+0000","last_active":"2024-05-09T10:51:06.408562+0000","last_peered":"2024-05-09T10:51:06.408562+0000","last_clean":"2024-05-09T10:51:06.408562+0000","last_became_active":"2024-05-09T10:51:06.403901+0000","last_became_peered":"2024-05-09T10:51:06.403901+0000","last_unstale":"2024-05-09T10:51:06.408562+0000","last_undegraded":"2024-05-09T10:51:06.408562+0000","last_fullsized":"2024-05-09T10:51:06.408562+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:29.685531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388109+0000","last_change":"2024-05-09T10:51:06.388109+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388109+0000","last_undegraded":"2024-05-09T10:51:06.388109+0000","last_fullsized":"2024-05-09T10:51:06.388109+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408642+0000","last_change":"2024-05-09T10:51:06.404951+0000","last_active":"2024-05-09T10:51:06.408642+0000","last_peered":"2024-05-09T10:51:06.408642+0000","last_clean":"2024-05-09T10:51:06.408642+0000","last_became_active":"2024-05-09T10:51:06.404574+0000","last_became_peered":"2024-05-09T10:51:06.404574+0000","last_unstale":"2024-05-09T10:51:06.408642+0000","last_undegraded":"2024-05-09T10:51:06.408642+0000","last_fullsized":"2024-05-09T10:51:06.408642+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:31:55.748009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.396433+0000","last_change":"2024-05-09T10:51:06.396433+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.396433+0000","last_undegraded":"2024-05-09T10:51:06.396433+0000","last_fullsized":"2024-05-09T10:51:06.396433+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408223+0000","last_change":"2024-05-09T10:51:06.404809+0000","last_active":"2024-05-09T10:51:06.408223+0000","last_peered":"2024-05-09T10:51:06.408223+0000","last_clean":"2024-05-09T10:51:06.408223+0000","last_became_active":"2024-05-09T10:51:06.404529+0000","last_became_peered":"2024-05-09T10:51:06.404529+0000","last_unstale":"2024-05-09T10:51:06.408223+0000","last_undegraded":"2024-05-09T10:51:06.408223+0000","last_fullsized":"2024-05-09T10:51:06.408223+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:30:44.917770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388071+0000","last_change":"2024-05-09T10:51:06.388071+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388071+0000","last_undegraded":"2024-05-09T10:51:06.388071+0000","last_fullsized":"2024-05-09T10:51:06.388071+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.402311+0000","last_change":"2024-05-09T10:51:06.398765+0000","last_active":"2024-05-09T10:51:06.402311+0000","last_peered":"2024-05-09T10:51:06.402311+0000","last_clean":"2024-05-09T10:51:06.402311+0000","last_became_active":"2024-05-09T10:51:06.397480+0000","last_became_peered":"2024-05-09T10:51:06.397480+0000","last_unstale":"2024-05-09T10:51:06.402311+0000","last_undegraded":"2024-05-09T10:51:06.402311+0000","last_fullsized":"2024-05-09T10:51:06.402311+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:29.685531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.387775+0000","last_change":"2024-05-09T10:51:06.387775+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.387775+0000","last_undegraded":"2024-05-09T10:51:06.387775+0000","last_fullsized":"2024-05-09T10:51:06.387775+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.402004+0000","last_change":"2024-05-09T10:51:06.399613+0000","last_active":"2024-05-09T10:51:06.402004+0000","last_peered":"2024-05-09T10:51:06.402004+0000","last_clean":"2024-05-09T10:51:06.402004+0000","last_became_active":"2024-05-09T10:51:06.399333+0000","last_became_peered":"2024-05-09T10:51:06.399333+0000","last_unstale":"2024-05-09T10:51:06.402004+0000","last_undegraded":"2024-05-09T10:51:06.402004+0000","last_fullsized":"2024-05-09T10:51:06.402004+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:05:03.035522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.389289+0000","last_change":"2024-05-09T10:51:06.389289+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.389289+0000","last_undegraded":"2024-05-09T10:51:06.389289+0000","last_fullsized":"2024-05-09T10:51:06.389289+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.400013+0000","last_change":"2024-05-09T10:51:06.396469+0000","last_active":"2024-05-09T10:51:06.400013+0000","last_peered":"2024-05-09T10:51:06.400013+0000","last_clean":"2024-05-09T10:51:06.400013+0000","last_became_active":"2024-05-09T10:51:06.396180+0000","last_became_peered":"2024-05-09T10:51:06.396180+0000","last_unstale":"2024-05-09T10:51:06.400013+0000","last_undegraded":"2024-05-09T10:51:06.400013+0000","last_fullsized":"2024-05-09T10:51:06.400013+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:21.607813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392198+0000","last_change":"2024-05-09T10:51:06.392198+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392198+0000","last_undegraded":"2024-05-09T10:51:06.392198+0000","last_fullsized":"2024-05-09T10:51:06.392198+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.395406+0000","last_change":"2024-05-09T10:51:06.394559+0000","last_active":"2024-05-09T10:51:06.395406+0000","last_peered":"2024-05-09T10:51:06.395406+0000","last_clean":"2024-05-09T10:51:06.395406+0000","last_became_active":"2024-05-09T10:51:06.394288+0000","last_became_peered":"2024-05-09T10:51:06.394288+0000","last_unstale":"2024-05-09T10:51:06.395406+0000","last_undegraded":"2024-05-09T10:51:06.395406+0000","last_fullsized":"2024-05-09T10:51:06.395406+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:21.607813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.389645+0000","last_change":"2024-05-09T10:51:06.389645+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.389645+0000","last_undegraded":"2024-05-09T10:51:06.389645+0000","last_fullsized":"2024-05-09T10:51:06.389645+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.404581+0000","last_change":"2024-05-09T10:51:06.400034+0000","last_active":"2024-05-09T10:51:06.404581+0000","last_peered":"2024-05-09T10:51:06.404581+0000","last_clean":"2024-05-09T10:51:06.404581+0000","last_became_active":"2024-05-09T10:51:06.397993+0000","last_became_peered":"2024-05-09T10:51:06.397993+0000","last_unstale":"2024-05-09T10:51:06.404581+0000","last_undegraded":"2024-05-09T10:51:06.404581+0000","last_fullsized":"2024-05-09T10:51:06.404581+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:51.058627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393340+0000","last_change":"2024-05-09T10:51:06.393340+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393340+0000","last_undegraded":"2024-05-09T10:51:06.393340+0000","last_fullsized":"2024-05-09T10:51:06.393340+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406276+0000","last_change":"2024-05-09T10:51:06.400625+0000","last_active":"2024-05-09T10:51:06.406276+0000","last_peered":"2024-05-09T10:51:06.406276+0000","last_clean":"2024-05-09T10:51:06.406276+0000","last_became_active":"2024-05-09T10:51:06.400119+0000","last_became_peered":"2024-05-09T10:51:06.400119+0000","last_unstale":"2024-05-09T10:51:06.406276+0000","last_undegraded":"2024-05-09T10:51:06.406276+0000","last_fullsized":"2024-05-09T10:51:06.406276+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:34:06.133956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395771+0000","last_change":"2024-05-09T10:51:06.395771+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395771+0000","last_undegraded":"2024-05-09T10:51:06.395771+0000","last_fullsized":"2024-05-09T10:51:06.395771+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408282+0000","last_change":"2024-05-09T10:51:06.406673+0000","last_active":"2024-05-09T10:51:06.408282+0000","last_peered":"2024-05-09T10:51:06.408282+0000","last_clean":"2024-05-09T10:51:06.408282+0000","last_became_active":"2024-05-09T10:51:06.406391+0000","last_became_peered":"2024-05-09T10:51:06.406391+0000","last_unstale":"2024-05-09T10:51:06.408282+0000","last_undegraded":"2024-05-09T10:51:06.408282+0000","last_fullsized":"2024-05-09T10:51:06.408282+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:45:51.854599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395869+0000","last_change":"2024-05-09T10:51:06.395869+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395869+0000","last_undegraded":"2024-05-09T10:51:06.395869+0000","last_fullsized":"2024-05-09T10:51:06.395869+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408907+0000","last_change":"2024-05-09T10:51:06.407600+0000","last_active":"2024-05-09T10:51:06.408907+0000","last_peered":"2024-05-09T10:51:06.408907+0000","last_clean":"2024-05-09T10:51:06.408907+0000","last_became_active":"2024-05-09T10:51:06.407354+0000","last_became_peered":"2024-05-09T10:51:06.407354+0000","last_unstale":"2024-05-09T10:51:06.408907+0000","last_undegraded":"2024-05-09T10:51:06.408907+0000","last_fullsized":"2024-05-09T10:51:06.408907+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:51.058627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":186,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.401241+0000","last_change":"2024-05-09T10:50:08.410364+0000","last_active":"2024-05-09T10:51:06.401241+0000","last_peered":"2024-05-09T10:51:06.401241+0000","last_clean":"2024-05-09T10:51:06.401241+0000","last_became_active":"2024-05-09T10:50:08.409457+0000","last_became_peered":"2024-05-09T10:50:08.409457+0000","last_unstale":"2024-05-09T10:51:06.401241+0000","last_undegraded":"2024-05-09T10:51:06.401241+0000","last_fullsized":"2024-05-09T10:51:06.401241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_clean_scrub_stamp":"2024-05-09T10:50:07.396510+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:03.566698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":917504,"data_stored":711419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":917504,"data_stored":711419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T10:51:10.357 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T10:51:10.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.500+0000 7ff134ebd640 1 Processor -- start 2024-05-09T10:51:10.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.500+0000 7ff134ebd640 1 -- start start 2024-05-09T10:51:10.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.501+0000 7ff134ebd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1301443e0 0x7ff1300720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:10.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.501+0000 7ff134ebd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff130143d70 0x7ff13015edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:10.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.501+0000 7ff134ebd640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff130143b20 con 0x7ff130143740 2024-05-09T10:51:10.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.501+0000 7ff134ebd640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff13015f460 con 0x7ff130143d70 2024-05-09T10:51:10.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.501+0000 7ff134ebd640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1300b28c0 con 0x7ff1301443e0 2024-05-09T10:51:10.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.501+0000 7ff125d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1301443e0 0x7ff1300720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:10.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.501+0000 7ff12ed9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff130143d70 0x7ff13015edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:10.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.501+0000 7ff125d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1301443e0 0x7ff1300720a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:42122/0 (socket says 172.21.15.98:42122) 2024-05-09T10:51:10.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.501+0000 7ff125d9b640 1 -- 172.21.15.98:0/3070058366 learned_addr learned my addr 172.21.15.98:0/3070058366 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:10.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.501+0000 7ff12e59c640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff130143740 0x7ff13015e6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.98:50270/0 (socket says 172.21.15.98:50270) 2024-05-09T10:51:10.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.502+0000 7ff12dd9b640 1 -- 172.21.15.98:0/3070058366 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1270645864 0 0) 0x7ff130143b20 con 0x7ff130143740 2024-05-09T10:51:10.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.502+0000 7ff12dd9b640 1 -- 172.21.15.98:0/3070058366 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff11c003520 con 0x7ff130143740 2024-05-09T10:51:10.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.502+0000 7ff125d9b640 1 -- 172.21.15.98:0/3070058366 >> v2:172.21.15.105:3300/0 conn(0x7ff130143d70 msgr2=0x7ff13015edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:10.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.502+0000 7ff125d9b640 1 --2- 172.21.15.98:0/3070058366 >> v2:172.21.15.105:3300/0 conn(0x7ff130143d70 0x7ff13015edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.502+0000 7ff125d9b640 1 -- 172.21.15.98:0/3070058366 >> v1:172.21.15.105:6789/0 conn(0x7ff130143740 legacy=0x7ff13015e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:10.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.502+0000 7ff125d9b640 1 -- 172.21.15.98:0/3070058366 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1300b2ba0 con 0x7ff1301443e0 2024-05-09T10:51:10.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.502+0000 7ff12ed9d640 1 --2- 172.21.15.98:0/3070058366 >> v2:172.21.15.105:3300/0 conn(0x7ff130143d70 0x7ff13015edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:51:10.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.502+0000 7ff125d9b640 1 --2- 172.21.15.98:0/3070058366 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1301443e0 0x7ff1300720a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff118009870 tx=0x7ff11802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eef50d62f8b34edd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:10.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.503+0000 7ff12dd9b640 1 -- 172.21.15.98:0/3070058366 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118037070 con 0x7ff1301443e0 2024-05-09T10:51:10.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.503+0000 7ff12dd9b640 1 -- 172.21.15.98:0/3070058366 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff118032440 con 0x7ff1301443e0 2024-05-09T10:51:10.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.503+0000 7ff12dd9b640 1 -- 172.21.15.98:0/3070058366 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1180327f0 con 0x7ff1301443e0 2024-05-09T10:51:10.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.503+0000 7ff134ebd640 1 -- 172.21.15.98:0/3070058366 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1301443e0 msgr2=0x7ff1300720a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:10.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.503+0000 7ff134ebd640 1 --2- 172.21.15.98:0/3070058366 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1301443e0 0x7ff1300720a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff118009870 tx=0x7ff11802a1c0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.503+0000 7ff134ebd640 1 -- 172.21.15.98:0/3070058366 shutdown_connections 2024-05-09T10:51:10.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.503+0000 7ff134ebd640 1 --2- 172.21.15.98:0/3070058366 >> v2:172.21.15.105:3300/0 conn(0x7ff130143d70 0x7ff13015edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.503+0000 7ff134ebd640 1 --2- 172.21.15.98:0/3070058366 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1301443e0 0x7ff1300720a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.503+0000 7ff134ebd640 1 -- 172.21.15.98:0/3070058366 >> 172.21.15.98:0/3070058366 conn(0x7ff13006abe0 msgr2=0x7ff13006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:10.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.503+0000 7ff134ebd640 1 -- 172.21.15.98:0/3070058366 shutdown_connections 2024-05-09T10:51:10.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.503+0000 7ff134ebd640 1 -- 172.21.15.98:0/3070058366 wait complete. 2024-05-09T10:51:10.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.505+0000 7ff134ebd640 1 Processor -- start 2024-05-09T10:51:10.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.505+0000 7ff134ebd640 1 -- start start 2024-05-09T10:51:10.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.506+0000 7ff134ebd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff130143740 0x7ff13013c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:10.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.506+0000 7ff134ebd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff130143d70 0x7ff130140570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:10.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.506+0000 7ff134ebd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1301443e0 0x7ff13013e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:10.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.506+0000 7ff134ebd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1300b3490 con 0x7ff130143d70 2024-05-09T10:51:10.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.506+0000 7ff134ebd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1300b3310 con 0x7ff130143740 2024-05-09T10:51:10.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.506+0000 7ff134ebd640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1300b3610 con 0x7ff1301443e0 2024-05-09T10:51:10.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.506+0000 7ff125d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff130143d70 0x7ff130140570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:10.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.506+0000 7ff125d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff130143d70 0x7ff130140570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49052/0 (socket says 172.21.15.98:49052) 2024-05-09T10:51:10.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.506+0000 7ff125d9b640 1 -- 172.21.15.98:0/2291475874 learned_addr learned my addr 172.21.15.98:0/2291475874 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:10.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.506+0000 7ff12e59c640 1 --2- 172.21.15.98:0/2291475874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff130143740 0x7ff13013c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:10.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.506+0000 7ff12ed9d640 1 --2- 172.21.15.98:0/2291475874 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1301443e0 0x7ff13013e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:10.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.506+0000 7ff125d9b640 1 -- 172.21.15.98:0/2291475874 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1301443e0 msgr2=0x7ff13013e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:10.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.506+0000 7ff125d9b640 1 --2- 172.21.15.98:0/2291475874 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1301443e0 0x7ff13013e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.506+0000 7ff125d9b640 1 -- 172.21.15.98:0/2291475874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff130143740 msgr2=0x7ff13013c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:10.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.506+0000 7ff125d9b640 1 --2- 172.21.15.98:0/2291475874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff130143740 0x7ff13013c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.506+0000 7ff125d9b640 1 -- 172.21.15.98:0/2291475874 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13015a9a0 con 0x7ff130143d70 2024-05-09T10:51:10.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.506+0000 7ff125d9b640 1 --2- 172.21.15.98:0/2291475874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff130143d70 0x7ff130140570 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff118000c00 tx=0x7ff1180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:10.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.506+0000 7ff1277fe640 1 -- 172.21.15.98:0/2291475874 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118041020 con 0x7ff130143d70 2024-05-09T10:51:10.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.506+0000 7ff1277fe640 1 -- 172.21.15.98:0/2291475874 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11803d620 con 0x7ff130143d70 2024-05-09T10:51:10.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.506+0000 7ff134ebd640 1 -- 172.21.15.98:0/2291475874 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff130141dd0 con 0x7ff130143d70 2024-05-09T10:51:10.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.506+0000 7ff1277fe640 1 -- 172.21.15.98:0/2291475874 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118037040 con 0x7ff130143d70 2024-05-09T10:51:10.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.506+0000 7ff134ebd640 1 -- 172.21.15.98:0/2291475874 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff130142390 con 0x7ff130143d70 2024-05-09T10:51:10.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.508+0000 7ff134ebd640 1 -- 172.21.15.98:0/2291475874 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff130069cb0 con 0x7ff130143d70 2024-05-09T10:51:10.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.508+0000 7ff1277fe640 1 -- 172.21.15.98:0/2291475874 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff118044020 con 0x7ff130143d70 2024-05-09T10:51:10.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.508+0000 7ff1277fe640 1 --2- 172.21.15.98:0/2291475874 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff11006b4d0 0x7ff11006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:10.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.508+0000 7ff1277fe640 1 -- 172.21.15.98:0/2291475874 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1180aa6f0 con 0x7ff130143d70 2024-05-09T10:51:10.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.508+0000 7ff12e59c640 1 --2- 172.21.15.98:0/2291475874 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff11006b4d0 0x7ff11006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:10.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.509+0000 7ff12e59c640 1 --2- 172.21.15.98:0/2291475874 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff11006b4d0 0x7ff11006d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff120000f60 tx=0x7ff120007cf0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:10.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.512+0000 7ff1277fe640 1 -- 172.21.15.98:0/2291475874 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff11804b030 con 0x7ff130143d70 2024-05-09T10:51:10.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.680+0000 7ff134ebd640 1 -- 172.21.15.98:0/2291475874 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff1300673c0 con 0x7ff11006b4d0 2024-05-09T10:51:10.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.686+0000 7ff1277fe640 1 -- 172.21.15.98:0/2291475874 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246634 (secure 0 0 0) 0x7ff1300673c0 con 0x7ff11006b4d0 2024-05-09T10:51:10.682 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:10.683 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-05-09T10:51:10.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.689+0000 7ff134ebd640 1 -- 172.21.15.98:0/2291475874 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff11006b4d0 msgr2=0x7ff11006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:10.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.689+0000 7ff134ebd640 1 --2- 172.21.15.98:0/2291475874 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff11006b4d0 0x7ff11006d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff120000f60 tx=0x7ff120007cf0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.689+0000 7ff134ebd640 1 -- 172.21.15.98:0/2291475874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff130143d70 msgr2=0x7ff130140570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:10.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.689+0000 7ff134ebd640 1 --2- 172.21.15.98:0/2291475874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff130143d70 0x7ff130140570 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff118000c00 tx=0x7ff1180046c0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.689+0000 7ff134ebd640 1 -- 172.21.15.98:0/2291475874 shutdown_connections 2024-05-09T10:51:10.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.689+0000 7ff134ebd640 1 --2- 172.21.15.98:0/2291475874 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff11006b4d0 0x7ff11006d990 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.689+0000 7ff134ebd640 1 --2- 172.21.15.98:0/2291475874 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff1301443e0 0x7ff13013e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.689+0000 7ff134ebd640 1 --2- 172.21.15.98:0/2291475874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff130143d70 0x7ff130140570 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.689+0000 7ff134ebd640 1 --2- 172.21.15.98:0/2291475874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff130143740 0x7ff13013c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.689+0000 7ff134ebd640 1 -- 172.21.15.98:0/2291475874 >> 172.21.15.98:0/2291475874 conn(0x7ff13006abe0 msgr2=0x7ff1300b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:10.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.689+0000 7ff134ebd640 1 -- 172.21.15.98:0/2291475874 shutdown_connections 2024-05-09T10:51:10.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.689+0000 7ff134ebd640 1 -- 172.21.15.98:0/2291475874 wait complete. 2024-05-09T10:51:10.693 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-05-09T10:51:09.374919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219880,"kb_used_data":3688,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754840,"statfs":{"total":773094113280,"available":772868956160,"internally_reserved":0,"allocated":3776512,"data_stored":2149144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000826"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393755+0000","last_change":"2024-05-09T10:51:06.393755+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393755+0000","last_undegraded":"2024-05-09T10:51:06.393755+0000","last_fullsized":"2024-05-09T10:51:06.393755+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392812+0000","last_change":"2024-05-09T10:51:06.392812+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392812+0000","last_undegraded":"2024-05-09T10:51:06.392812+0000","last_fullsized":"2024-05-09T10:51:06.392812+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388265+0000","last_change":"2024-05-09T10:51:06.388265+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388265+0000","last_undegraded":"2024-05-09T10:51:06.388265+0000","last_fullsized":"2024-05-09T10:51:06.388265+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393261+0000","last_change":"2024-05-09T10:51:06.393261+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393261+0000","last_undegraded":"2024-05-09T10:51:06.393261+0000","last_fullsized":"2024-05-09T10:51:06.393261+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.389283+0000","last_change":"2024-05-09T10:51:06.389283+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.389283+0000","last_undegraded":"2024-05-09T10:51:06.389283+0000","last_fullsized":"2024-05-09T10:51:06.389283+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.394709+0000","last_change":"2024-05-09T10:51:06.394709+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.394709+0000","last_undegraded":"2024-05-09T10:51:06.394709+0000","last_fullsized":"2024-05-09T10:51:06.394709+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395480+0000","last_change":"2024-05-09T10:51:06.395480+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395480+0000","last_undegraded":"2024-05-09T10:51:06.395480+0000","last_fullsized":"2024-05-09T10:51:06.395480+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.398792+0000","last_change":"2024-05-09T10:51:06.398792+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.398792+0000","last_undegraded":"2024-05-09T10:51:06.398792+0000","last_fullsized":"2024-05-09T10:51:06.398792+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.394178+0000","last_change":"2024-05-09T10:51:06.394178+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.394178+0000","last_undegraded":"2024-05-09T10:51:06.394178+0000","last_fullsized":"2024-05-09T10:51:06.394178+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392823+0000","last_change":"2024-05-09T10:51:06.392823+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392823+0000","last_undegraded":"2024-05-09T10:51:06.392823+0000","last_fullsized":"2024-05-09T10:51:06.392823+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.396067+0000","last_change":"2024-05-09T10:51:06.396067+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.396067+0000","last_undegraded":"2024-05-09T10:51:06.396067+0000","last_fullsized":"2024-05-09T10:51:06.396067+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388701+0000","last_change":"2024-05-09T10:51:06.388701+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388701+0000","last_undegraded":"2024-05-09T10:51:06.388701+0000","last_fullsized":"2024-05-09T10:51:06.388701+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388834+0000","last_change":"2024-05-09T10:51:06.388834+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388834+0000","last_undegraded":"2024-05-09T10:51:06.388834+0000","last_fullsized":"2024-05-09T10:51:06.388834+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393806+0000","last_change":"2024-05-09T10:51:06.393806+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393806+0000","last_undegraded":"2024-05-09T10:51:06.393806+0000","last_fullsized":"2024-05-09T10:51:06.393806+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388334+0000","last_change":"2024-05-09T10:51:06.388334+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388334+0000","last_undegraded":"2024-05-09T10:51:06.388334+0000","last_fullsized":"2024-05-09T10:51:06.388334+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.396634+0000","last_change":"2024-05-09T10:51:06.396634+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.396634+0000","last_undegraded":"2024-05-09T10:51:06.396634+0000","last_fullsized":"2024-05-09T10:51:06.396634+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.397070+0000","last_change":"2024-05-09T10:51:06.397070+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.397070+0000","last_undegraded":"2024-05-09T10:51:06.397070+0000","last_fullsized":"2024-05-09T10:51:06.397070+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392398+0000","last_change":"2024-05-09T10:51:06.392398+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392398+0000","last_undegraded":"2024-05-09T10:51:06.392398+0000","last_fullsized":"2024-05-09T10:51:06.392398+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395656+0000","last_change":"2024-05-09T10:51:06.395656+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395656+0000","last_undegraded":"2024-05-09T10:51:06.395656+0000","last_fullsized":"2024-05-09T10:51:06.395656+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388193+0000","last_change":"2024-05-09T10:51:06.388193+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388193+0000","last_undegraded":"2024-05-09T10:51:06.388193+0000","last_fullsized":"2024-05-09T10:51:06.388193+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.398904+0000","last_change":"2024-05-09T10:51:06.398904+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.398904+0000","last_undegraded":"2024-05-09T10:51:06.398904+0000","last_fullsized":"2024-05-09T10:51:06.398904+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.396580+0000","last_change":"2024-05-09T10:51:06.396580+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.396580+0000","last_undegraded":"2024-05-09T10:51:06.396580+0000","last_fullsized":"2024-05-09T10:51:06.396580+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.391128+0000","last_change":"2024-05-09T10:51:06.391128+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.391128+0000","last_undegraded":"2024-05-09T10:51:06.391128+0000","last_fullsized":"2024-05-09T10:51:06.391128+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.387807+0000","last_change":"2024-05-09T10:51:06.387807+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.387807+0000","last_undegraded":"2024-05-09T10:51:06.387807+0000","last_fullsized":"2024-05-09T10:51:06.387807+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.391876+0000","last_change":"2024-05-09T10:51:06.391876+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.391876+0000","last_undegraded":"2024-05-09T10:51:06.391876+0000","last_fullsized":"2024-05-09T10:51:06.391876+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.391884+0000","last_change":"2024-05-09T10:51:06.391884+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.391884+0000","last_undegraded":"2024-05-09T10:51:06.391884+0000","last_fullsized":"2024-05-09T10:51:06.391884+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392175+0000","last_change":"2024-05-09T10:51:06.392175+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392175+0000","last_undegraded":"2024-05-09T10:51:06.392175+0000","last_fullsized":"2024-05-09T10:51:06.392175+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388098+0000","last_change":"2024-05-09T10:51:06.388098+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388098+0000","last_undegraded":"2024-05-09T10:51:06.388098+0000","last_fullsized":"2024-05-09T10:51:06.388098+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.398517+0000","last_change":"2024-05-09T10:51:06.398517+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.398517+0000","last_undegraded":"2024-05-09T10:51:06.398517+0000","last_fullsized":"2024-05-09T10:51:06.398517+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.396453+0000","last_change":"2024-05-09T10:51:06.396453+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.396453+0000","last_undegraded":"2024-05-09T10:51:06.396453+0000","last_fullsized":"2024-05-09T10:51:06.396453+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392904+0000","last_change":"2024-05-09T10:51:06.392904+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392904+0000","last_undegraded":"2024-05-09T10:51:06.392904+0000","last_fullsized":"2024-05-09T10:51:06.392904+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.398378+0000","last_change":"2024-05-09T10:51:06.398378+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.398378+0000","last_undegraded":"2024-05-09T10:51:06.398378+0000","last_fullsized":"2024-05-09T10:51:06.398378+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.390548+0000","last_change":"2024-05-09T10:51:06.390548+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.390548+0000","last_undegraded":"2024-05-09T10:51:06.390548+0000","last_fullsized":"2024-05-09T10:51:06.390548+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.403221+0000","last_change":"2024-05-09T10:51:06.399065+0000","last_active":"2024-05-09T10:51:06.403221+0000","last_peered":"2024-05-09T10:51:06.403221+0000","last_clean":"2024-05-09T10:51:06.403221+0000","last_became_active":"2024-05-09T10:51:06.398631+0000","last_became_peered":"2024-05-09T10:51:06.398631+0000","last_unstale":"2024-05-09T10:51:06.403221+0000","last_undegraded":"2024-05-09T10:51:06.403221+0000","last_fullsized":"2024-05-09T10:51:06.403221+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:21.607813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393181+0000","last_change":"2024-05-09T10:51:06.393181+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393181+0000","last_undegraded":"2024-05-09T10:51:06.393181+0000","last_fullsized":"2024-05-09T10:51:06.393181+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.405979+0000","last_change":"2024-05-09T10:51:06.405043+0000","last_active":"2024-05-09T10:51:06.405979+0000","last_peered":"2024-05-09T10:51:06.405979+0000","last_clean":"2024-05-09T10:51:06.405979+0000","last_became_active":"2024-05-09T10:51:06.404375+0000","last_became_peered":"2024-05-09T10:51:06.404375+0000","last_unstale":"2024-05-09T10:51:06.405979+0000","last_undegraded":"2024-05-09T10:51:06.405979+0000","last_fullsized":"2024-05-09T10:51:06.405979+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:03.704175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.399481+0000","last_change":"2024-05-09T10:51:06.399481+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.399481+0000","last_undegraded":"2024-05-09T10:51:06.399481+0000","last_fullsized":"2024-05-09T10:51:06.399481+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406256+0000","last_change":"2024-05-09T10:51:06.403754+0000","last_active":"2024-05-09T10:51:06.406256+0000","last_peered":"2024-05-09T10:51:06.406256+0000","last_clean":"2024-05-09T10:51:06.406256+0000","last_became_active":"2024-05-09T10:51:06.403476+0000","last_became_peered":"2024-05-09T10:51:06.403476+0000","last_unstale":"2024-05-09T10:51:06.406256+0000","last_undegraded":"2024-05-09T10:51:06.406256+0000","last_fullsized":"2024-05-09T10:51:06.406256+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:48:18.069829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.391713+0000","last_change":"2024-05-09T10:51:06.391713+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.391713+0000","last_undegraded":"2024-05-09T10:51:06.391713+0000","last_fullsized":"2024-05-09T10:51:06.391713+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.410669+0000","last_change":"2024-05-09T10:51:06.401479+0000","last_active":"2024-05-09T10:51:06.410669+0000","last_peered":"2024-05-09T10:51:06.410669+0000","last_clean":"2024-05-09T10:51:06.410669+0000","last_became_active":"2024-05-09T10:51:06.401079+0000","last_became_peered":"2024-05-09T10:51:06.401079+0000","last_unstale":"2024-05-09T10:51:06.410669+0000","last_undegraded":"2024-05-09T10:51:06.410669+0000","last_fullsized":"2024-05-09T10:51:06.410669+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:56.016304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395508+0000","last_change":"2024-05-09T10:51:06.395508+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395508+0000","last_undegraded":"2024-05-09T10:51:06.395508+0000","last_fullsized":"2024-05-09T10:51:06.395508+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408534+0000","last_change":"2024-05-09T10:51:06.403964+0000","last_active":"2024-05-09T10:51:06.408534+0000","last_peered":"2024-05-09T10:51:06.408534+0000","last_clean":"2024-05-09T10:51:06.408534+0000","last_became_active":"2024-05-09T10:51:06.403691+0000","last_became_peered":"2024-05-09T10:51:06.403691+0000","last_unstale":"2024-05-09T10:51:06.408534+0000","last_undegraded":"2024-05-09T10:51:06.408534+0000","last_fullsized":"2024-05-09T10:51:06.408534+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:26.083086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.387999+0000","last_change":"2024-05-09T10:51:06.387999+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.387999+0000","last_undegraded":"2024-05-09T10:51:06.387999+0000","last_fullsized":"2024-05-09T10:51:06.387999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408477+0000","last_change":"2024-05-09T10:51:06.403128+0000","last_active":"2024-05-09T10:51:06.408477+0000","last_peered":"2024-05-09T10:51:06.408477+0000","last_clean":"2024-05-09T10:51:06.408477+0000","last_became_active":"2024-05-09T10:51:06.402841+0000","last_became_peered":"2024-05-09T10:51:06.402841+0000","last_unstale":"2024-05-09T10:51:06.408477+0000","last_undegraded":"2024-05-09T10:51:06.408477+0000","last_fullsized":"2024-05-09T10:51:06.408477+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:33:59.577821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.397348+0000","last_change":"2024-05-09T10:51:06.397348+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.397348+0000","last_undegraded":"2024-05-09T10:51:06.397348+0000","last_fullsized":"2024-05-09T10:51:06.397348+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.414048+0000","last_change":"2024-05-09T10:51:06.410970+0000","last_active":"2024-05-09T10:51:06.414048+0000","last_peered":"2024-05-09T10:51:06.414048+0000","last_clean":"2024-05-09T10:51:06.414048+0000","last_became_active":"2024-05-09T10:51:06.410608+0000","last_became_peered":"2024-05-09T10:51:06.410608+0000","last_unstale":"2024-05-09T10:51:06.414048+0000","last_undegraded":"2024-05-09T10:51:06.414048+0000","last_fullsized":"2024-05-09T10:51:06.414048+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:30:44.917770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.391151+0000","last_change":"2024-05-09T10:51:06.391151+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.391151+0000","last_undegraded":"2024-05-09T10:51:06.391151+0000","last_fullsized":"2024-05-09T10:51:06.391151+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408469+0000","last_change":"2024-05-09T10:51:06.403392+0000","last_active":"2024-05-09T10:51:06.408469+0000","last_peered":"2024-05-09T10:51:06.408469+0000","last_clean":"2024-05-09T10:51:06.408469+0000","last_became_active":"2024-05-09T10:51:06.403111+0000","last_became_peered":"2024-05-09T10:51:06.403111+0000","last_unstale":"2024-05-09T10:51:06.408469+0000","last_undegraded":"2024-05-09T10:51:06.408469+0000","last_fullsized":"2024-05-09T10:51:06.408469+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:56.016304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388231+0000","last_change":"2024-05-09T10:51:06.388231+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388231+0000","last_undegraded":"2024-05-09T10:51:06.388231+0000","last_fullsized":"2024-05-09T10:51:06.388231+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408273+0000","last_change":"2024-05-09T10:51:06.406795+0000","last_active":"2024-05-09T10:51:06.408273+0000","last_peered":"2024-05-09T10:51:06.408273+0000","last_clean":"2024-05-09T10:51:06.408273+0000","last_became_active":"2024-05-09T10:51:06.406516+0000","last_became_peered":"2024-05-09T10:51:06.406516+0000","last_unstale":"2024-05-09T10:51:06.408273+0000","last_undegraded":"2024-05-09T10:51:06.408273+0000","last_fullsized":"2024-05-09T10:51:06.408273+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:29.722036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393382+0000","last_change":"2024-05-09T10:51:06.393382+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393382+0000","last_undegraded":"2024-05-09T10:51:06.393382+0000","last_fullsized":"2024-05-09T10:51:06.393382+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.401982+0000","last_change":"2024-05-09T10:51:06.399077+0000","last_active":"2024-05-09T10:51:06.401982+0000","last_peered":"2024-05-09T10:51:06.401982+0000","last_clean":"2024-05-09T10:51:06.401982+0000","last_became_active":"2024-05-09T10:51:06.398760+0000","last_became_peered":"2024-05-09T10:51:06.398760+0000","last_unstale":"2024-05-09T10:51:06.401982+0000","last_undegraded":"2024-05-09T10:51:06.401982+0000","last_fullsized":"2024-05-09T10:51:06.401982+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:51.058627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.394715+0000","last_change":"2024-05-09T10:51:06.394715+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.394715+0000","last_undegraded":"2024-05-09T10:51:06.394715+0000","last_fullsized":"2024-05-09T10:51:06.394715+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408196+0000","last_change":"2024-05-09T10:51:06.402648+0000","last_active":"2024-05-09T10:51:06.408196+0000","last_peered":"2024-05-09T10:51:06.408196+0000","last_clean":"2024-05-09T10:51:06.408196+0000","last_became_active":"2024-05-09T10:51:06.402315+0000","last_became_peered":"2024-05-09T10:51:06.402315+0000","last_unstale":"2024-05-09T10:51:06.408196+0000","last_undegraded":"2024-05-09T10:51:06.408196+0000","last_fullsized":"2024-05-09T10:51:06.408196+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:29.722036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.389652+0000","last_change":"2024-05-09T10:51:06.389652+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.389652+0000","last_undegraded":"2024-05-09T10:51:06.389652+0000","last_fullsized":"2024-05-09T10:51:06.389652+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406190+0000","last_change":"2024-05-09T10:51:06.403735+0000","last_active":"2024-05-09T10:51:06.406190+0000","last_peered":"2024-05-09T10:51:06.406190+0000","last_clean":"2024-05-09T10:51:06.406190+0000","last_became_active":"2024-05-09T10:51:06.403469+0000","last_became_peered":"2024-05-09T10:51:06.403469+0000","last_unstale":"2024-05-09T10:51:06.406190+0000","last_undegraded":"2024-05-09T10:51:06.406190+0000","last_fullsized":"2024-05-09T10:51:06.406190+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:03.704175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393445+0000","last_change":"2024-05-09T10:51:06.393445+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393445+0000","last_undegraded":"2024-05-09T10:51:06.393445+0000","last_fullsized":"2024-05-09T10:51:06.393445+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.409330+0000","last_change":"2024-05-09T10:51:06.406522+0000","last_active":"2024-05-09T10:51:06.409330+0000","last_peered":"2024-05-09T10:51:06.409330+0000","last_clean":"2024-05-09T10:51:06.409330+0000","last_became_active":"2024-05-09T10:51:06.406242+0000","last_became_peered":"2024-05-09T10:51:06.406242+0000","last_unstale":"2024-05-09T10:51:06.409330+0000","last_undegraded":"2024-05-09T10:51:06.409330+0000","last_fullsized":"2024-05-09T10:51:06.409330+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:21.592545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395214+0000","last_change":"2024-05-09T10:51:06.395214+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395214+0000","last_undegraded":"2024-05-09T10:51:06.395214+0000","last_fullsized":"2024-05-09T10:51:06.395214+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406193+0000","last_change":"2024-05-09T10:51:06.402262+0000","last_active":"2024-05-09T10:51:06.406193+0000","last_peered":"2024-05-09T10:51:06.406193+0000","last_clean":"2024-05-09T10:51:06.406193+0000","last_became_active":"2024-05-09T10:51:06.401936+0000","last_became_peered":"2024-05-09T10:51:06.401936+0000","last_unstale":"2024-05-09T10:51:06.406193+0000","last_undegraded":"2024-05-09T10:51:06.406193+0000","last_fullsized":"2024-05-09T10:51:06.406193+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:05:03.035522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.397979+0000","last_change":"2024-05-09T10:51:06.397979+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.397979+0000","last_undegraded":"2024-05-09T10:51:06.397979+0000","last_fullsized":"2024-05-09T10:51:06.397979+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406601+0000","last_change":"2024-05-09T10:51:06.405127+0000","last_active":"2024-05-09T10:51:06.406601+0000","last_peered":"2024-05-09T10:51:06.406601+0000","last_clean":"2024-05-09T10:51:06.406601+0000","last_became_active":"2024-05-09T10:51:06.404682+0000","last_became_peered":"2024-05-09T10:51:06.404682+0000","last_unstale":"2024-05-09T10:51:06.406601+0000","last_undegraded":"2024-05-09T10:51:06.406601+0000","last_fullsized":"2024-05-09T10:51:06.406601+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:21.393250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392434+0000","last_change":"2024-05-09T10:51:06.392434+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392434+0000","last_undegraded":"2024-05-09T10:51:06.392434+0000","last_fullsized":"2024-05-09T10:51:06.392434+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.402616+0000","last_change":"2024-05-09T10:51:06.401431+0000","last_active":"2024-05-09T10:51:06.402616+0000","last_peered":"2024-05-09T10:51:06.402616+0000","last_clean":"2024-05-09T10:51:06.402616+0000","last_became_active":"2024-05-09T10:51:06.401097+0000","last_became_peered":"2024-05-09T10:51:06.401097+0000","last_unstale":"2024-05-09T10:51:06.402616+0000","last_undegraded":"2024-05-09T10:51:06.402616+0000","last_fullsized":"2024-05-09T10:51:06.402616+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:05:03.035522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.398088+0000","last_change":"2024-05-09T10:51:06.398088+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.398088+0000","last_undegraded":"2024-05-09T10:51:06.398088+0000","last_fullsized":"2024-05-09T10:51:06.398088+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406321+0000","last_change":"2024-05-09T10:51:06.403074+0000","last_active":"2024-05-09T10:51:06.406321+0000","last_peered":"2024-05-09T10:51:06.406321+0000","last_clean":"2024-05-09T10:51:06.406321+0000","last_became_active":"2024-05-09T10:51:06.402721+0000","last_became_peered":"2024-05-09T10:51:06.402721+0000","last_unstale":"2024-05-09T10:51:06.406321+0000","last_undegraded":"2024-05-09T10:51:06.406321+0000","last_fullsized":"2024-05-09T10:51:06.406321+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:03.704175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388885+0000","last_change":"2024-05-09T10:51:06.388885+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388885+0000","last_undegraded":"2024-05-09T10:51:06.388885+0000","last_fullsized":"2024-05-09T10:51:06.388885+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.409346+0000","last_change":"2024-05-09T10:51:06.407069+0000","last_active":"2024-05-09T10:51:06.409346+0000","last_peered":"2024-05-09T10:51:06.409346+0000","last_clean":"2024-05-09T10:51:06.409346+0000","last_became_active":"2024-05-09T10:51:06.406780+0000","last_became_peered":"2024-05-09T10:51:06.406780+0000","last_unstale":"2024-05-09T10:51:06.409346+0000","last_undegraded":"2024-05-09T10:51:06.409346+0000","last_fullsized":"2024-05-09T10:51:06.409346+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:01.273826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392827+0000","last_change":"2024-05-09T10:51:06.392827+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392827+0000","last_undegraded":"2024-05-09T10:51:06.392827+0000","last_fullsized":"2024-05-09T10:51:06.392827+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.411703+0000","last_change":"2024-05-09T10:51:06.411247+0000","last_active":"2024-05-09T10:51:06.411703+0000","last_peered":"2024-05-09T10:51:06.411703+0000","last_clean":"2024-05-09T10:51:06.411703+0000","last_became_active":"2024-05-09T10:51:06.411001+0000","last_became_peered":"2024-05-09T10:51:06.411001+0000","last_unstale":"2024-05-09T10:51:06.411703+0000","last_undegraded":"2024-05-09T10:51:06.411703+0000","last_fullsized":"2024-05-09T10:51:06.411703+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:35.165729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388204+0000","last_change":"2024-05-09T10:51:06.388204+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388204+0000","last_undegraded":"2024-05-09T10:51:06.388204+0000","last_fullsized":"2024-05-09T10:51:06.388204+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408650+0000","last_change":"2024-05-09T10:51:06.405035+0000","last_active":"2024-05-09T10:51:06.408650+0000","last_peered":"2024-05-09T10:51:06.408650+0000","last_clean":"2024-05-09T10:51:06.408650+0000","last_became_active":"2024-05-09T10:51:06.404712+0000","last_became_peered":"2024-05-09T10:51:06.404712+0000","last_unstale":"2024-05-09T10:51:06.408650+0000","last_undegraded":"2024-05-09T10:51:06.408650+0000","last_fullsized":"2024-05-09T10:51:06.408650+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:47:47.684710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388110+0000","last_change":"2024-05-09T10:51:06.388110+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388110+0000","last_undegraded":"2024-05-09T10:51:06.388110+0000","last_fullsized":"2024-05-09T10:51:06.388110+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.401280+0000","last_change":"2024-05-09T10:51:06.396633+0000","last_active":"2024-05-09T10:51:06.401280+0000","last_peered":"2024-05-09T10:51:06.401280+0000","last_clean":"2024-05-09T10:51:06.401280+0000","last_became_active":"2024-05-09T10:51:06.396277+0000","last_became_peered":"2024-05-09T10:51:06.396277+0000","last_unstale":"2024-05-09T10:51:06.401280+0000","last_undegraded":"2024-05-09T10:51:06.401280+0000","last_fullsized":"2024-05-09T10:51:06.401280+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:29.685531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388416+0000","last_change":"2024-05-09T10:51:06.388416+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388416+0000","last_undegraded":"2024-05-09T10:51:06.388416+0000","last_fullsized":"2024-05-09T10:51:06.388416+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408562+0000","last_change":"2024-05-09T10:51:06.405148+0000","last_active":"2024-05-09T10:51:06.408562+0000","last_peered":"2024-05-09T10:51:06.408562+0000","last_clean":"2024-05-09T10:51:06.408562+0000","last_became_active":"2024-05-09T10:51:06.403901+0000","last_became_peered":"2024-05-09T10:51:06.403901+0000","last_unstale":"2024-05-09T10:51:06.408562+0000","last_undegraded":"2024-05-09T10:51:06.408562+0000","last_fullsized":"2024-05-09T10:51:06.408562+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:29.685531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388109+0000","last_change":"2024-05-09T10:51:06.388109+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388109+0000","last_undegraded":"2024-05-09T10:51:06.388109+0000","last_fullsized":"2024-05-09T10:51:06.388109+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408642+0000","last_change":"2024-05-09T10:51:06.404951+0000","last_active":"2024-05-09T10:51:06.408642+0000","last_peered":"2024-05-09T10:51:06.408642+0000","last_clean":"2024-05-09T10:51:06.408642+0000","last_became_active":"2024-05-09T10:51:06.404574+0000","last_became_peered":"2024-05-09T10:51:06.404574+0000","last_unstale":"2024-05-09T10:51:06.408642+0000","last_undegraded":"2024-05-09T10:51:06.408642+0000","last_fullsized":"2024-05-09T10:51:06.408642+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:31:55.748009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.396433+0000","last_change":"2024-05-09T10:51:06.396433+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.396433+0000","last_undegraded":"2024-05-09T10:51:06.396433+0000","last_fullsized":"2024-05-09T10:51:06.396433+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408223+0000","last_change":"2024-05-09T10:51:06.404809+0000","last_active":"2024-05-09T10:51:06.408223+0000","last_peered":"2024-05-09T10:51:06.408223+0000","last_clean":"2024-05-09T10:51:06.408223+0000","last_became_active":"2024-05-09T10:51:06.404529+0000","last_became_peered":"2024-05-09T10:51:06.404529+0000","last_unstale":"2024-05-09T10:51:06.408223+0000","last_undegraded":"2024-05-09T10:51:06.408223+0000","last_fullsized":"2024-05-09T10:51:06.408223+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:30:44.917770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388071+0000","last_change":"2024-05-09T10:51:06.388071+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388071+0000","last_undegraded":"2024-05-09T10:51:06.388071+0000","last_fullsized":"2024-05-09T10:51:06.388071+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.402311+0000","last_change":"2024-05-09T10:51:06.398765+0000","last_active":"2024-05-09T10:51:06.402311+0000","last_peered":"2024-05-09T10:51:06.402311+0000","last_clean":"2024-05-09T10:51:06.402311+0000","last_became_active":"2024-05-09T10:51:06.397480+0000","last_became_peered":"2024-05-09T10:51:06.397480+0000","last_unstale":"2024-05-09T10:51:06.402311+0000","last_undegraded":"2024-05-09T10:51:06.402311+0000","last_fullsized":"2024-05-09T10:51:06.402311+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:29.685531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.387775+0000","last_change":"2024-05-09T10:51:06.387775+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.387775+0000","last_undegraded":"2024-05-09T10:51:06.387775+0000","last_fullsized":"2024-05-09T10:51:06.387775+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.402004+0000","last_change":"2024-05-09T10:51:06.399613+0000","last_active":"2024-05-09T10:51:06.402004+0000","last_peered":"2024-05-09T10:51:06.402004+0000","last_clean":"2024-05-09T10:51:06.402004+0000","last_became_active":"2024-05-09T10:51:06.399333+0000","last_became_peered":"2024-05-09T10:51:06.399333+0000","last_unstale":"2024-05-09T10:51:06.402004+0000","last_undegraded":"2024-05-09T10:51:06.402004+0000","last_fullsized":"2024-05-09T10:51:06.402004+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:05:03.035522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.389289+0000","last_change":"2024-05-09T10:51:06.389289+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.389289+0000","last_undegraded":"2024-05-09T10:51:06.389289+0000","last_fullsized":"2024-05-09T10:51:06.389289+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.400013+0000","last_change":"2024-05-09T10:51:06.396469+0000","last_active":"2024-05-09T10:51:06.400013+0000","last_peered":"2024-05-09T10:51:06.400013+0000","last_clean":"2024-05-09T10:51:06.400013+0000","last_became_active":"2024-05-09T10:51:06.396180+0000","last_became_peered":"2024-05-09T10:51:06.396180+0000","last_unstale":"2024-05-09T10:51:06.400013+0000","last_undegraded":"2024-05-09T10:51:06.400013+0000","last_fullsized":"2024-05-09T10:51:06.400013+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:21.607813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392198+0000","last_change":"2024-05-09T10:51:06.392198+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392198+0000","last_undegraded":"2024-05-09T10:51:06.392198+0000","last_fullsized":"2024-05-09T10:51:06.392198+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.395406+0000","last_change":"2024-05-09T10:51:06.394559+0000","last_active":"2024-05-09T10:51:06.395406+0000","last_peered":"2024-05-09T10:51:06.395406+0000","last_clean":"2024-05-09T10:51:06.395406+0000","last_became_active":"2024-05-09T10:51:06.394288+0000","last_became_peered":"2024-05-09T10:51:06.394288+0000","last_unstale":"2024-05-09T10:51:06.395406+0000","last_undegraded":"2024-05-09T10:51:06.395406+0000","last_fullsized":"2024-05-09T10:51:06.395406+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:21.607813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.389645+0000","last_change":"2024-05-09T10:51:06.389645+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.389645+0000","last_undegraded":"2024-05-09T10:51:06.389645+0000","last_fullsized":"2024-05-09T10:51:06.389645+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.404581+0000","last_change":"2024-05-09T10:51:06.400034+0000","last_active":"2024-05-09T10:51:06.404581+0000","last_peered":"2024-05-09T10:51:06.404581+0000","last_clean":"2024-05-09T10:51:06.404581+0000","last_became_active":"2024-05-09T10:51:06.397993+0000","last_became_peered":"2024-05-09T10:51:06.397993+0000","last_unstale":"2024-05-09T10:51:06.404581+0000","last_undegraded":"2024-05-09T10:51:06.404581+0000","last_fullsized":"2024-05-09T10:51:06.404581+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:51.058627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393340+0000","last_change":"2024-05-09T10:51:06.393340+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393340+0000","last_undegraded":"2024-05-09T10:51:06.393340+0000","last_fullsized":"2024-05-09T10:51:06.393340+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406276+0000","last_change":"2024-05-09T10:51:06.400625+0000","last_active":"2024-05-09T10:51:06.406276+0000","last_peered":"2024-05-09T10:51:06.406276+0000","last_clean":"2024-05-09T10:51:06.406276+0000","last_became_active":"2024-05-09T10:51:06.400119+0000","last_became_peered":"2024-05-09T10:51:06.400119+0000","last_unstale":"2024-05-09T10:51:06.406276+0000","last_undegraded":"2024-05-09T10:51:06.406276+0000","last_fullsized":"2024-05-09T10:51:06.406276+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:34:06.133956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395771+0000","last_change":"2024-05-09T10:51:06.395771+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395771+0000","last_undegraded":"2024-05-09T10:51:06.395771+0000","last_fullsized":"2024-05-09T10:51:06.395771+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408282+0000","last_change":"2024-05-09T10:51:06.406673+0000","last_active":"2024-05-09T10:51:06.408282+0000","last_peered":"2024-05-09T10:51:06.408282+0000","last_clean":"2024-05-09T10:51:06.408282+0000","last_became_active":"2024-05-09T10:51:06.406391+0000","last_became_peered":"2024-05-09T10:51:06.406391+0000","last_unstale":"2024-05-09T10:51:06.408282+0000","last_undegraded":"2024-05-09T10:51:06.408282+0000","last_fullsized":"2024-05-09T10:51:06.408282+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:45:51.854599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395869+0000","last_change":"2024-05-09T10:51:06.395869+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395869+0000","last_undegraded":"2024-05-09T10:51:06.395869+0000","last_fullsized":"2024-05-09T10:51:06.395869+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408907+0000","last_change":"2024-05-09T10:51:06.407600+0000","last_active":"2024-05-09T10:51:06.408907+0000","last_peered":"2024-05-09T10:51:06.408907+0000","last_clean":"2024-05-09T10:51:06.408907+0000","last_became_active":"2024-05-09T10:51:06.407354+0000","last_became_peered":"2024-05-09T10:51:06.407354+0000","last_unstale":"2024-05-09T10:51:06.408907+0000","last_undegraded":"2024-05-09T10:51:06.408907+0000","last_fullsized":"2024-05-09T10:51:06.408907+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:51.058627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":186,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.401241+0000","last_change":"2024-05-09T10:50:08.410364+0000","last_active":"2024-05-09T10:51:06.401241+0000","last_peered":"2024-05-09T10:51:06.401241+0000","last_clean":"2024-05-09T10:51:06.401241+0000","last_became_active":"2024-05-09T10:50:08.409457+0000","last_became_peered":"2024-05-09T10:50:08.409457+0000","last_unstale":"2024-05-09T10:51:06.401241+0000","last_undegraded":"2024-05-09T10:51:06.401241+0000","last_fullsized":"2024-05-09T10:51:06.401241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_clean_scrub_stamp":"2024-05-09T10:50:07.396510+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:03.566698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":917504,"data_stored":711419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":917504,"data_stored":711419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T10:51:10.697 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T10:51:10.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.840+0000 7f6796c15640 1 Processor -- start 2024-05-09T10:51:10.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.840+0000 7f6796c15640 1 -- start start 2024-05-09T10:51:10.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f6796c15640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f679015bdc0 0x7f67900cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:10.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f6796c15640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67900cc4c0 con 0x7f67900b4f10 2024-05-09T10:51:10.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f6796c15640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67900cf0e0 con 0x7f67900ceb70 2024-05-09T10:51:10.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f6796c15640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f679015c1a0 con 0x7f679015bdc0 2024-05-09T10:51:10.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f67951b2640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f67900b4f10 0x7f67900cfa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52818/0 (socket says 172.21.15.98:52818) 2024-05-09T10:51:10.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f67951b2640 1 -- 172.21.15.98:0/401036017 learned_addr learned my addr 172.21.15.98:0/401036017 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:10.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f67949b1640 1 --2- 172.21.15.98:0/401036017 >> v2:172.21.15.105:3300/0 conn(0x7f679015bdc0 0x7f67900cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:10.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f67877fe640 1 -- 172.21.15.98:0/401036017 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585201252 0 0) 0x7f67900cc4c0 con 0x7f67900b4f10 2024-05-09T10:51:10.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f67877fe640 1 -- 172.21.15.98:0/401036017 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6778003520 con 0x7f67900b4f10 2024-05-09T10:51:10.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f67877fe640 1 -- 172.21.15.98:0/401036017 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2533095286 0 0) 0x7f6778003520 con 0x7f67900b4f10 2024-05-09T10:51:10.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f67877fe640 1 -- 172.21.15.98:0/401036017 >> v2:172.21.15.105:3300/0 conn(0x7f679015bdc0 msgr2=0x7f67900cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:10.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f67877fe640 1 --2- 172.21.15.98:0/401036017 >> v2:172.21.15.105:3300/0 conn(0x7f679015bdc0 0x7f67900cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f67877fe640 1 -- 172.21.15.98:0/401036017 >> v1:172.21.15.105:6789/0 conn(0x7f67900ceb70 legacy=0x7f67900b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:10.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f67877fe640 1 -- 172.21.15.98:0/401036017 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67900cc760 con 0x7f67900b4f10 2024-05-09T10:51:10.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f67877fe640 1 -- 172.21.15.98:0/401036017 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f6788001670 con 0x7f67900b4f10 2024-05-09T10:51:10.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f67949b1640 1 --2- 172.21.15.98:0/401036017 >> v2:172.21.15.105:3300/0 conn(0x7f679015bdc0 0x7f67900cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:10.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f67877fe640 1 -- 172.21.15.98:0/401036017 >> v1:172.21.15.98:6789/0 conn(0x7f67900b4f10 legacy=0x7f67900cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:10.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f67877fe640 1 --2- 172.21.15.98:0/401036017 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67780039b0 0x7f6778003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:10.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f67877fe640 1 --2- 172.21.15.98:0/401036017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6778004800 0x7f6778004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:10.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f67877fe640 1 --2- 172.21.15.98:0/401036017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67780051c0 0x7f677800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:10.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f67877fe640 1 -- 172.21.15.98:0/401036017 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679015c1a0 con 0x7f6778004800 2024-05-09T10:51:10.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f67877fe640 1 -- 172.21.15.98:0/401036017 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677800c110 con 0x7f67780051c0 2024-05-09T10:51:10.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f67877fe640 1 -- 172.21.15.98:0/401036017 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f677800c310 con 0x7f67780039b0 2024-05-09T10:51:10.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.841+0000 7f6787fff640 1 --2- 172.21.15.98:0/401036017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6778004800 0x7f6778004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:10.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.842+0000 7f67949b1640 1 --2- 172.21.15.98:0/401036017 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67780039b0 0x7f6778003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:10.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.842+0000 7f67951b2640 1 --2- 172.21.15.98:0/401036017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67780051c0 0x7f677800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:10.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.842+0000 7f6787fff640 1 -- 172.21.15.98:0/401036017 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67780039b0 msgr2=0x7f6778003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:10.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.842+0000 7f6787fff640 1 --2- 172.21.15.98:0/401036017 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67780039b0 0x7f6778003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.842+0000 7f6787fff640 1 -- 172.21.15.98:0/401036017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67780051c0 msgr2=0x7f677800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:10.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.842+0000 7f6787fff640 1 --2- 172.21.15.98:0/401036017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67780051c0 0x7f677800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.842+0000 7f6787fff640 1 -- 172.21.15.98:0/401036017 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6778003520 con 0x7f6778004800 2024-05-09T10:51:10.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.842+0000 7f67951b2640 1 --2- 172.21.15.98:0/401036017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67780051c0 0x7f677800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:10.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.842+0000 7f67949b1640 1 --2- 172.21.15.98:0/401036017 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67780039b0 0x7f6778003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:10.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.842+0000 7f67951b2640 1 -- 172.21.15.98:0/401036017 reap_dead start 2024-05-09T10:51:10.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.842+0000 7f6787fff640 1 --2- 172.21.15.98:0/401036017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6778004800 0x7f6778004c80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6780001750 tx=0x7f6778005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c99eca9a8ceebd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:10.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.843+0000 7f67877fe640 1 -- 172.21.15.98:0/401036017 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780001370 con 0x7f6778004800 2024-05-09T10:51:10.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.843+0000 7f67877fe640 1 -- 172.21.15.98:0/401036017 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67800052d0 con 0x7f6778004800 2024-05-09T10:51:10.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.843+0000 7f67877fe640 1 -- 172.21.15.98:0/401036017 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678002abc0 con 0x7f6778004800 2024-05-09T10:51:10.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.843+0000 7f6796c15640 1 -- 172.21.15.98:0/401036017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6778004800 msgr2=0x7f6778004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:10.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.843+0000 7f6796c15640 1 --2- 172.21.15.98:0/401036017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6778004800 0x7f6778004c80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6780001750 tx=0x7f6778005830 comp rx=0 tx=0).stop 2024-05-09T10:51:10.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.843+0000 7f6796c15640 1 -- 172.21.15.98:0/401036017 shutdown_connections 2024-05-09T10:51:10.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.843+0000 7f6796c15640 1 --2- 172.21.15.98:0/401036017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6778004800 0x7f6778004c80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.843+0000 7f6796c15640 1 -- 172.21.15.98:0/401036017 >> 172.21.15.98:0/401036017 conn(0x7f67900b34e0 msgr2=0x7f67900b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:10.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.843+0000 7f6796c15640 1 -- 172.21.15.98:0/401036017 shutdown_connections 2024-05-09T10:51:10.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.843+0000 7f6796c15640 1 -- 172.21.15.98:0/401036017 wait complete. 2024-05-09T10:51:10.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.844+0000 7f6796c15640 1 Processor -- start 2024-05-09T10:51:10.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.844+0000 7f6796c15640 1 -- start start 2024-05-09T10:51:10.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.845+0000 7f6796c15640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6778004800 0x7f67901f4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:10.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.845+0000 7f6796c15640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67901f54f0 0x7f67900c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:10.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.845+0000 7f6796c15640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67900c8fc0 0x7f67900cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:10.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.845+0000 7f6796c15640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67900ccd20 con 0x7f6778004800 2024-05-09T10:51:10.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.845+0000 7f6796c15640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67900cf3c0 con 0x7f67900c8fc0 2024-05-09T10:51:10.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.845+0000 7f6796c15640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67900b41b0 con 0x7f67901f54f0 2024-05-09T10:51:10.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.845+0000 7f67949b1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6778004800 0x7f67901f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:10.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.845+0000 7f67949b1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6778004800 0x7f67901f4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49068/0 (socket says 172.21.15.98:49068) 2024-05-09T10:51:10.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.845+0000 7f67949b1640 1 -- 172.21.15.98:0/745396940 learned_addr learned my addr 172.21.15.98:0/745396940 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:10.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.846+0000 7f67949b1640 1 -- 172.21.15.98:0/745396940 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67901f54f0 msgr2=0x7f67900c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:10.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.846+0000 7f67949b1640 1 --2- 172.21.15.98:0/745396940 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67901f54f0 0x7f67900c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.846+0000 7f67951b2640 1 --2- 172.21.15.98:0/745396940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67900c8fc0 0x7f67900cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:10.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.846+0000 7f67949b1640 1 -- 172.21.15.98:0/745396940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67900c8fc0 msgr2=0x7f67900cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:10.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.846+0000 7f67949b1640 1 --2- 172.21.15.98:0/745396940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67900c8fc0 0x7f67900cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:10.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.846+0000 7f67949b1640 1 -- 172.21.15.98:0/745396940 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67900cb9f0 con 0x7f6778004800 2024-05-09T10:51:10.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.846+0000 7f67951b2640 1 --2- 172.21.15.98:0/745396940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67900c8fc0 0x7f67900cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:10.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.846+0000 7f67949b1640 1 --2- 172.21.15.98:0/745396940 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6778004800 0x7f67901f4cc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f678c004660 tx=0x7f678c007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:10.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.846+0000 7f67857fa640 1 -- 172.21.15.98:0/745396940 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c014610 con 0x7f6778004800 2024-05-09T10:51:10.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.846+0000 7f67857fa640 1 -- 172.21.15.98:0/745396940 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678c019070 con 0x7f6778004800 2024-05-09T10:51:10.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.846+0000 7f6796c15640 1 -- 172.21.15.98:0/745396940 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67900cbcd0 con 0x7f6778004800 2024-05-09T10:51:10.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.846+0000 7f67857fa640 1 -- 172.21.15.98:0/745396940 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c0149b0 con 0x7f6778004800 2024-05-09T10:51:10.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.846+0000 7f6796c15640 1 -- 172.21.15.98:0/745396940 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679006bfd0 con 0x7f6778004800 2024-05-09T10:51:10.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.847+0000 7f6796c15640 1 -- 172.21.15.98:0/745396940 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6790069cb0 con 0x7f6778004800 2024-05-09T10:51:10.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.847+0000 7f67857fa640 1 -- 172.21.15.98:0/745396940 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f678c014b50 con 0x7f6778004800 2024-05-09T10:51:10.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.848+0000 7f67857fa640 1 --2- 172.21.15.98:0/745396940 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f677c06b4d0 0x7f677c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:10.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.848+0000 7f67857fa640 1 -- 172.21.15.98:0/745396940 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f678c08aab0 con 0x7f6778004800 2024-05-09T10:51:10.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.848+0000 7f6787fff640 1 --2- 172.21.15.98:0/745396940 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f677c06b4d0 0x7f677c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:10.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.848+0000 7f6787fff640 1 --2- 172.21.15.98:0/745396940 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f677c06b4d0 0x7f677c06d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6780032000 tx=0x7f678002c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:10.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:10.852+0000 7f67857fa640 1 -- 172.21.15.98:0/745396940 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f678c0599c0 con 0x7f6778004800 2024-05-09T10:51:11.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.018+0000 7f6796c15640 1 -- 172.21.15.98:0/745396940 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f67900b7930 con 0x7f677c06b4d0 2024-05-09T10:51:11.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.024+0000 7f67857fa640 1 -- 172.21.15.98:0/745396940 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246634 (secure 0 0 0) 0x7f67900b7930 con 0x7f677c06b4d0 2024-05-09T10:51:11.021 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:11.023 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-05-09T10:51:11.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.027+0000 7f6796c15640 1 -- 172.21.15.98:0/745396940 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f677c06b4d0 msgr2=0x7f677c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:11.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.027+0000 7f6796c15640 1 --2- 172.21.15.98:0/745396940 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f677c06b4d0 0x7f677c06d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6780032000 tx=0x7f678002c000 comp rx=0 tx=0).stop 2024-05-09T10:51:11.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.027+0000 7f6796c15640 1 -- 172.21.15.98:0/745396940 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6778004800 msgr2=0x7f67901f4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:11.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.027+0000 7f6796c15640 1 --2- 172.21.15.98:0/745396940 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6778004800 0x7f67901f4cc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f678c004660 tx=0x7f678c007700 comp rx=0 tx=0).stop 2024-05-09T10:51:11.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.027+0000 7f6796c15640 1 -- 172.21.15.98:0/745396940 shutdown_connections 2024-05-09T10:51:11.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.027+0000 7f6796c15640 1 --2- 172.21.15.98:0/745396940 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f677c06b4d0 0x7f677c06d990 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:11.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.027+0000 7f6796c15640 1 --2- 172.21.15.98:0/745396940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67900c8fc0 0x7f67900cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:11.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.027+0000 7f6796c15640 1 --2- 172.21.15.98:0/745396940 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f67901f54f0 0x7f67900c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:11.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.027+0000 7f6796c15640 1 --2- 172.21.15.98:0/745396940 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6778004800 0x7f67901f4cc0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:11.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.027+0000 7f6796c15640 1 -- 172.21.15.98:0/745396940 >> 172.21.15.98:0/745396940 conn(0x7f67900b34e0 msgr2=0x7f679006ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:11.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.027+0000 7f6796c15640 1 -- 172.21.15.98:0/745396940 shutdown_connections 2024-05-09T10:51:11.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.027+0000 7f6796c15640 1 -- 172.21.15.98:0/745396940 wait complete. 2024-05-09T10:51:11.031 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-05-09T10:51:09.374919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219880,"kb_used_data":3688,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754840,"statfs":{"total":773094113280,"available":772868956160,"internally_reserved":0,"allocated":3776512,"data_stored":2149144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000826"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393755+0000","last_change":"2024-05-09T10:51:06.393755+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393755+0000","last_undegraded":"2024-05-09T10:51:06.393755+0000","last_fullsized":"2024-05-09T10:51:06.393755+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392812+0000","last_change":"2024-05-09T10:51:06.392812+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392812+0000","last_undegraded":"2024-05-09T10:51:06.392812+0000","last_fullsized":"2024-05-09T10:51:06.392812+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388265+0000","last_change":"2024-05-09T10:51:06.388265+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388265+0000","last_undegraded":"2024-05-09T10:51:06.388265+0000","last_fullsized":"2024-05-09T10:51:06.388265+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393261+0000","last_change":"2024-05-09T10:51:06.393261+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393261+0000","last_undegraded":"2024-05-09T10:51:06.393261+0000","last_fullsized":"2024-05-09T10:51:06.393261+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.389283+0000","last_change":"2024-05-09T10:51:06.389283+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.389283+0000","last_undegraded":"2024-05-09T10:51:06.389283+0000","last_fullsized":"2024-05-09T10:51:06.389283+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.394709+0000","last_change":"2024-05-09T10:51:06.394709+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.394709+0000","last_undegraded":"2024-05-09T10:51:06.394709+0000","last_fullsized":"2024-05-09T10:51:06.394709+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395480+0000","last_change":"2024-05-09T10:51:06.395480+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395480+0000","last_undegraded":"2024-05-09T10:51:06.395480+0000","last_fullsized":"2024-05-09T10:51:06.395480+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.398792+0000","last_change":"2024-05-09T10:51:06.398792+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.398792+0000","last_undegraded":"2024-05-09T10:51:06.398792+0000","last_fullsized":"2024-05-09T10:51:06.398792+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.394178+0000","last_change":"2024-05-09T10:51:06.394178+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.394178+0000","last_undegraded":"2024-05-09T10:51:06.394178+0000","last_fullsized":"2024-05-09T10:51:06.394178+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392823+0000","last_change":"2024-05-09T10:51:06.392823+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392823+0000","last_undegraded":"2024-05-09T10:51:06.392823+0000","last_fullsized":"2024-05-09T10:51:06.392823+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.396067+0000","last_change":"2024-05-09T10:51:06.396067+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.396067+0000","last_undegraded":"2024-05-09T10:51:06.396067+0000","last_fullsized":"2024-05-09T10:51:06.396067+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388701+0000","last_change":"2024-05-09T10:51:06.388701+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388701+0000","last_undegraded":"2024-05-09T10:51:06.388701+0000","last_fullsized":"2024-05-09T10:51:06.388701+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388834+0000","last_change":"2024-05-09T10:51:06.388834+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388834+0000","last_undegraded":"2024-05-09T10:51:06.388834+0000","last_fullsized":"2024-05-09T10:51:06.388834+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393806+0000","last_change":"2024-05-09T10:51:06.393806+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393806+0000","last_undegraded":"2024-05-09T10:51:06.393806+0000","last_fullsized":"2024-05-09T10:51:06.393806+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388334+0000","last_change":"2024-05-09T10:51:06.388334+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388334+0000","last_undegraded":"2024-05-09T10:51:06.388334+0000","last_fullsized":"2024-05-09T10:51:06.388334+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.396634+0000","last_change":"2024-05-09T10:51:06.396634+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.396634+0000","last_undegraded":"2024-05-09T10:51:06.396634+0000","last_fullsized":"2024-05-09T10:51:06.396634+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.397070+0000","last_change":"2024-05-09T10:51:06.397070+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.397070+0000","last_undegraded":"2024-05-09T10:51:06.397070+0000","last_fullsized":"2024-05-09T10:51:06.397070+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392398+0000","last_change":"2024-05-09T10:51:06.392398+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392398+0000","last_undegraded":"2024-05-09T10:51:06.392398+0000","last_fullsized":"2024-05-09T10:51:06.392398+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395656+0000","last_change":"2024-05-09T10:51:06.395656+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395656+0000","last_undegraded":"2024-05-09T10:51:06.395656+0000","last_fullsized":"2024-05-09T10:51:06.395656+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388193+0000","last_change":"2024-05-09T10:51:06.388193+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388193+0000","last_undegraded":"2024-05-09T10:51:06.388193+0000","last_fullsized":"2024-05-09T10:51:06.388193+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.398904+0000","last_change":"2024-05-09T10:51:06.398904+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.398904+0000","last_undegraded":"2024-05-09T10:51:06.398904+0000","last_fullsized":"2024-05-09T10:51:06.398904+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.396580+0000","last_change":"2024-05-09T10:51:06.396580+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.396580+0000","last_undegraded":"2024-05-09T10:51:06.396580+0000","last_fullsized":"2024-05-09T10:51:06.396580+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.391128+0000","last_change":"2024-05-09T10:51:06.391128+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.391128+0000","last_undegraded":"2024-05-09T10:51:06.391128+0000","last_fullsized":"2024-05-09T10:51:06.391128+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.387807+0000","last_change":"2024-05-09T10:51:06.387807+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.387807+0000","last_undegraded":"2024-05-09T10:51:06.387807+0000","last_fullsized":"2024-05-09T10:51:06.387807+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.391876+0000","last_change":"2024-05-09T10:51:06.391876+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.391876+0000","last_undegraded":"2024-05-09T10:51:06.391876+0000","last_fullsized":"2024-05-09T10:51:06.391876+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.391884+0000","last_change":"2024-05-09T10:51:06.391884+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.391884+0000","last_undegraded":"2024-05-09T10:51:06.391884+0000","last_fullsized":"2024-05-09T10:51:06.391884+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392175+0000","last_change":"2024-05-09T10:51:06.392175+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392175+0000","last_undegraded":"2024-05-09T10:51:06.392175+0000","last_fullsized":"2024-05-09T10:51:06.392175+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388098+0000","last_change":"2024-05-09T10:51:06.388098+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388098+0000","last_undegraded":"2024-05-09T10:51:06.388098+0000","last_fullsized":"2024-05-09T10:51:06.388098+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.398517+0000","last_change":"2024-05-09T10:51:06.398517+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.398517+0000","last_undegraded":"2024-05-09T10:51:06.398517+0000","last_fullsized":"2024-05-09T10:51:06.398517+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.396453+0000","last_change":"2024-05-09T10:51:06.396453+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.396453+0000","last_undegraded":"2024-05-09T10:51:06.396453+0000","last_fullsized":"2024-05-09T10:51:06.396453+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392904+0000","last_change":"2024-05-09T10:51:06.392904+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392904+0000","last_undegraded":"2024-05-09T10:51:06.392904+0000","last_fullsized":"2024-05-09T10:51:06.392904+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.398378+0000","last_change":"2024-05-09T10:51:06.398378+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.398378+0000","last_undegraded":"2024-05-09T10:51:06.398378+0000","last_fullsized":"2024-05-09T10:51:06.398378+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.390548+0000","last_change":"2024-05-09T10:51:06.390548+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.390548+0000","last_undegraded":"2024-05-09T10:51:06.390548+0000","last_fullsized":"2024-05-09T10:51:06.390548+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.403221+0000","last_change":"2024-05-09T10:51:06.399065+0000","last_active":"2024-05-09T10:51:06.403221+0000","last_peered":"2024-05-09T10:51:06.403221+0000","last_clean":"2024-05-09T10:51:06.403221+0000","last_became_active":"2024-05-09T10:51:06.398631+0000","last_became_peered":"2024-05-09T10:51:06.398631+0000","last_unstale":"2024-05-09T10:51:06.403221+0000","last_undegraded":"2024-05-09T10:51:06.403221+0000","last_fullsized":"2024-05-09T10:51:06.403221+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:21.607813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393181+0000","last_change":"2024-05-09T10:51:06.393181+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393181+0000","last_undegraded":"2024-05-09T10:51:06.393181+0000","last_fullsized":"2024-05-09T10:51:06.393181+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.405979+0000","last_change":"2024-05-09T10:51:06.405043+0000","last_active":"2024-05-09T10:51:06.405979+0000","last_peered":"2024-05-09T10:51:06.405979+0000","last_clean":"2024-05-09T10:51:06.405979+0000","last_became_active":"2024-05-09T10:51:06.404375+0000","last_became_peered":"2024-05-09T10:51:06.404375+0000","last_unstale":"2024-05-09T10:51:06.405979+0000","last_undegraded":"2024-05-09T10:51:06.405979+0000","last_fullsized":"2024-05-09T10:51:06.405979+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:03.704175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.399481+0000","last_change":"2024-05-09T10:51:06.399481+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.399481+0000","last_undegraded":"2024-05-09T10:51:06.399481+0000","last_fullsized":"2024-05-09T10:51:06.399481+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406256+0000","last_change":"2024-05-09T10:51:06.403754+0000","last_active":"2024-05-09T10:51:06.406256+0000","last_peered":"2024-05-09T10:51:06.406256+0000","last_clean":"2024-05-09T10:51:06.406256+0000","last_became_active":"2024-05-09T10:51:06.403476+0000","last_became_peered":"2024-05-09T10:51:06.403476+0000","last_unstale":"2024-05-09T10:51:06.406256+0000","last_undegraded":"2024-05-09T10:51:06.406256+0000","last_fullsized":"2024-05-09T10:51:06.406256+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:48:18.069829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.391713+0000","last_change":"2024-05-09T10:51:06.391713+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.391713+0000","last_undegraded":"2024-05-09T10:51:06.391713+0000","last_fullsized":"2024-05-09T10:51:06.391713+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.410669+0000","last_change":"2024-05-09T10:51:06.401479+0000","last_active":"2024-05-09T10:51:06.410669+0000","last_peered":"2024-05-09T10:51:06.410669+0000","last_clean":"2024-05-09T10:51:06.410669+0000","last_became_active":"2024-05-09T10:51:06.401079+0000","last_became_peered":"2024-05-09T10:51:06.401079+0000","last_unstale":"2024-05-09T10:51:06.410669+0000","last_undegraded":"2024-05-09T10:51:06.410669+0000","last_fullsized":"2024-05-09T10:51:06.410669+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:56.016304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395508+0000","last_change":"2024-05-09T10:51:06.395508+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395508+0000","last_undegraded":"2024-05-09T10:51:06.395508+0000","last_fullsized":"2024-05-09T10:51:06.395508+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408534+0000","last_change":"2024-05-09T10:51:06.403964+0000","last_active":"2024-05-09T10:51:06.408534+0000","last_peered":"2024-05-09T10:51:06.408534+0000","last_clean":"2024-05-09T10:51:06.408534+0000","last_became_active":"2024-05-09T10:51:06.403691+0000","last_became_peered":"2024-05-09T10:51:06.403691+0000","last_unstale":"2024-05-09T10:51:06.408534+0000","last_undegraded":"2024-05-09T10:51:06.408534+0000","last_fullsized":"2024-05-09T10:51:06.408534+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:26.083086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.387999+0000","last_change":"2024-05-09T10:51:06.387999+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.387999+0000","last_undegraded":"2024-05-09T10:51:06.387999+0000","last_fullsized":"2024-05-09T10:51:06.387999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408477+0000","last_change":"2024-05-09T10:51:06.403128+0000","last_active":"2024-05-09T10:51:06.408477+0000","last_peered":"2024-05-09T10:51:06.408477+0000","last_clean":"2024-05-09T10:51:06.408477+0000","last_became_active":"2024-05-09T10:51:06.402841+0000","last_became_peered":"2024-05-09T10:51:06.402841+0000","last_unstale":"2024-05-09T10:51:06.408477+0000","last_undegraded":"2024-05-09T10:51:06.408477+0000","last_fullsized":"2024-05-09T10:51:06.408477+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:33:59.577821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.397348+0000","last_change":"2024-05-09T10:51:06.397348+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.397348+0000","last_undegraded":"2024-05-09T10:51:06.397348+0000","last_fullsized":"2024-05-09T10:51:06.397348+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.414048+0000","last_change":"2024-05-09T10:51:06.410970+0000","last_active":"2024-05-09T10:51:06.414048+0000","last_peered":"2024-05-09T10:51:06.414048+0000","last_clean":"2024-05-09T10:51:06.414048+0000","last_became_active":"2024-05-09T10:51:06.410608+0000","last_became_peered":"2024-05-09T10:51:06.410608+0000","last_unstale":"2024-05-09T10:51:06.414048+0000","last_undegraded":"2024-05-09T10:51:06.414048+0000","last_fullsized":"2024-05-09T10:51:06.414048+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:30:44.917770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.391151+0000","last_change":"2024-05-09T10:51:06.391151+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.391151+0000","last_undegraded":"2024-05-09T10:51:06.391151+0000","last_fullsized":"2024-05-09T10:51:06.391151+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408469+0000","last_change":"2024-05-09T10:51:06.403392+0000","last_active":"2024-05-09T10:51:06.408469+0000","last_peered":"2024-05-09T10:51:06.408469+0000","last_clean":"2024-05-09T10:51:06.408469+0000","last_became_active":"2024-05-09T10:51:06.403111+0000","last_became_peered":"2024-05-09T10:51:06.403111+0000","last_unstale":"2024-05-09T10:51:06.408469+0000","last_undegraded":"2024-05-09T10:51:06.408469+0000","last_fullsized":"2024-05-09T10:51:06.408469+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:56.016304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388231+0000","last_change":"2024-05-09T10:51:06.388231+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388231+0000","last_undegraded":"2024-05-09T10:51:06.388231+0000","last_fullsized":"2024-05-09T10:51:06.388231+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408273+0000","last_change":"2024-05-09T10:51:06.406795+0000","last_active":"2024-05-09T10:51:06.408273+0000","last_peered":"2024-05-09T10:51:06.408273+0000","last_clean":"2024-05-09T10:51:06.408273+0000","last_became_active":"2024-05-09T10:51:06.406516+0000","last_became_peered":"2024-05-09T10:51:06.406516+0000","last_unstale":"2024-05-09T10:51:06.408273+0000","last_undegraded":"2024-05-09T10:51:06.408273+0000","last_fullsized":"2024-05-09T10:51:06.408273+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:29.722036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393382+0000","last_change":"2024-05-09T10:51:06.393382+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393382+0000","last_undegraded":"2024-05-09T10:51:06.393382+0000","last_fullsized":"2024-05-09T10:51:06.393382+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.401982+0000","last_change":"2024-05-09T10:51:06.399077+0000","last_active":"2024-05-09T10:51:06.401982+0000","last_peered":"2024-05-09T10:51:06.401982+0000","last_clean":"2024-05-09T10:51:06.401982+0000","last_became_active":"2024-05-09T10:51:06.398760+0000","last_became_peered":"2024-05-09T10:51:06.398760+0000","last_unstale":"2024-05-09T10:51:06.401982+0000","last_undegraded":"2024-05-09T10:51:06.401982+0000","last_fullsized":"2024-05-09T10:51:06.401982+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:51.058627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.394715+0000","last_change":"2024-05-09T10:51:06.394715+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.394715+0000","last_undegraded":"2024-05-09T10:51:06.394715+0000","last_fullsized":"2024-05-09T10:51:06.394715+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408196+0000","last_change":"2024-05-09T10:51:06.402648+0000","last_active":"2024-05-09T10:51:06.408196+0000","last_peered":"2024-05-09T10:51:06.408196+0000","last_clean":"2024-05-09T10:51:06.408196+0000","last_became_active":"2024-05-09T10:51:06.402315+0000","last_became_peered":"2024-05-09T10:51:06.402315+0000","last_unstale":"2024-05-09T10:51:06.408196+0000","last_undegraded":"2024-05-09T10:51:06.408196+0000","last_fullsized":"2024-05-09T10:51:06.408196+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:29.722036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.389652+0000","last_change":"2024-05-09T10:51:06.389652+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.389652+0000","last_undegraded":"2024-05-09T10:51:06.389652+0000","last_fullsized":"2024-05-09T10:51:06.389652+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406190+0000","last_change":"2024-05-09T10:51:06.403735+0000","last_active":"2024-05-09T10:51:06.406190+0000","last_peered":"2024-05-09T10:51:06.406190+0000","last_clean":"2024-05-09T10:51:06.406190+0000","last_became_active":"2024-05-09T10:51:06.403469+0000","last_became_peered":"2024-05-09T10:51:06.403469+0000","last_unstale":"2024-05-09T10:51:06.406190+0000","last_undegraded":"2024-05-09T10:51:06.406190+0000","last_fullsized":"2024-05-09T10:51:06.406190+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:03.704175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393445+0000","last_change":"2024-05-09T10:51:06.393445+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393445+0000","last_undegraded":"2024-05-09T10:51:06.393445+0000","last_fullsized":"2024-05-09T10:51:06.393445+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.409330+0000","last_change":"2024-05-09T10:51:06.406522+0000","last_active":"2024-05-09T10:51:06.409330+0000","last_peered":"2024-05-09T10:51:06.409330+0000","last_clean":"2024-05-09T10:51:06.409330+0000","last_became_active":"2024-05-09T10:51:06.406242+0000","last_became_peered":"2024-05-09T10:51:06.406242+0000","last_unstale":"2024-05-09T10:51:06.409330+0000","last_undegraded":"2024-05-09T10:51:06.409330+0000","last_fullsized":"2024-05-09T10:51:06.409330+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:21.592545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395214+0000","last_change":"2024-05-09T10:51:06.395214+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395214+0000","last_undegraded":"2024-05-09T10:51:06.395214+0000","last_fullsized":"2024-05-09T10:51:06.395214+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406193+0000","last_change":"2024-05-09T10:51:06.402262+0000","last_active":"2024-05-09T10:51:06.406193+0000","last_peered":"2024-05-09T10:51:06.406193+0000","last_clean":"2024-05-09T10:51:06.406193+0000","last_became_active":"2024-05-09T10:51:06.401936+0000","last_became_peered":"2024-05-09T10:51:06.401936+0000","last_unstale":"2024-05-09T10:51:06.406193+0000","last_undegraded":"2024-05-09T10:51:06.406193+0000","last_fullsized":"2024-05-09T10:51:06.406193+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:05:03.035522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.397979+0000","last_change":"2024-05-09T10:51:06.397979+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.397979+0000","last_undegraded":"2024-05-09T10:51:06.397979+0000","last_fullsized":"2024-05-09T10:51:06.397979+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406601+0000","last_change":"2024-05-09T10:51:06.405127+0000","last_active":"2024-05-09T10:51:06.406601+0000","last_peered":"2024-05-09T10:51:06.406601+0000","last_clean":"2024-05-09T10:51:06.406601+0000","last_became_active":"2024-05-09T10:51:06.404682+0000","last_became_peered":"2024-05-09T10:51:06.404682+0000","last_unstale":"2024-05-09T10:51:06.406601+0000","last_undegraded":"2024-05-09T10:51:06.406601+0000","last_fullsized":"2024-05-09T10:51:06.406601+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:21.393250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392434+0000","last_change":"2024-05-09T10:51:06.392434+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392434+0000","last_undegraded":"2024-05-09T10:51:06.392434+0000","last_fullsized":"2024-05-09T10:51:06.392434+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.402616+0000","last_change":"2024-05-09T10:51:06.401431+0000","last_active":"2024-05-09T10:51:06.402616+0000","last_peered":"2024-05-09T10:51:06.402616+0000","last_clean":"2024-05-09T10:51:06.402616+0000","last_became_active":"2024-05-09T10:51:06.401097+0000","last_became_peered":"2024-05-09T10:51:06.401097+0000","last_unstale":"2024-05-09T10:51:06.402616+0000","last_undegraded":"2024-05-09T10:51:06.402616+0000","last_fullsized":"2024-05-09T10:51:06.402616+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:05:03.035522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.398088+0000","last_change":"2024-05-09T10:51:06.398088+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.398088+0000","last_undegraded":"2024-05-09T10:51:06.398088+0000","last_fullsized":"2024-05-09T10:51:06.398088+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406321+0000","last_change":"2024-05-09T10:51:06.403074+0000","last_active":"2024-05-09T10:51:06.406321+0000","last_peered":"2024-05-09T10:51:06.406321+0000","last_clean":"2024-05-09T10:51:06.406321+0000","last_became_active":"2024-05-09T10:51:06.402721+0000","last_became_peered":"2024-05-09T10:51:06.402721+0000","last_unstale":"2024-05-09T10:51:06.406321+0000","last_undegraded":"2024-05-09T10:51:06.406321+0000","last_fullsized":"2024-05-09T10:51:06.406321+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:03.704175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388885+0000","last_change":"2024-05-09T10:51:06.388885+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388885+0000","last_undegraded":"2024-05-09T10:51:06.388885+0000","last_fullsized":"2024-05-09T10:51:06.388885+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.409346+0000","last_change":"2024-05-09T10:51:06.407069+0000","last_active":"2024-05-09T10:51:06.409346+0000","last_peered":"2024-05-09T10:51:06.409346+0000","last_clean":"2024-05-09T10:51:06.409346+0000","last_became_active":"2024-05-09T10:51:06.406780+0000","last_became_peered":"2024-05-09T10:51:06.406780+0000","last_unstale":"2024-05-09T10:51:06.409346+0000","last_undegraded":"2024-05-09T10:51:06.409346+0000","last_fullsized":"2024-05-09T10:51:06.409346+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:01.273826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392827+0000","last_change":"2024-05-09T10:51:06.392827+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392827+0000","last_undegraded":"2024-05-09T10:51:06.392827+0000","last_fullsized":"2024-05-09T10:51:06.392827+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.411703+0000","last_change":"2024-05-09T10:51:06.411247+0000","last_active":"2024-05-09T10:51:06.411703+0000","last_peered":"2024-05-09T10:51:06.411703+0000","last_clean":"2024-05-09T10:51:06.411703+0000","last_became_active":"2024-05-09T10:51:06.411001+0000","last_became_peered":"2024-05-09T10:51:06.411001+0000","last_unstale":"2024-05-09T10:51:06.411703+0000","last_undegraded":"2024-05-09T10:51:06.411703+0000","last_fullsized":"2024-05-09T10:51:06.411703+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:35.165729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388204+0000","last_change":"2024-05-09T10:51:06.388204+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388204+0000","last_undegraded":"2024-05-09T10:51:06.388204+0000","last_fullsized":"2024-05-09T10:51:06.388204+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408650+0000","last_change":"2024-05-09T10:51:06.405035+0000","last_active":"2024-05-09T10:51:06.408650+0000","last_peered":"2024-05-09T10:51:06.408650+0000","last_clean":"2024-05-09T10:51:06.408650+0000","last_became_active":"2024-05-09T10:51:06.404712+0000","last_became_peered":"2024-05-09T10:51:06.404712+0000","last_unstale":"2024-05-09T10:51:06.408650+0000","last_undegraded":"2024-05-09T10:51:06.408650+0000","last_fullsized":"2024-05-09T10:51:06.408650+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:47:47.684710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388110+0000","last_change":"2024-05-09T10:51:06.388110+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388110+0000","last_undegraded":"2024-05-09T10:51:06.388110+0000","last_fullsized":"2024-05-09T10:51:06.388110+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.401280+0000","last_change":"2024-05-09T10:51:06.396633+0000","last_active":"2024-05-09T10:51:06.401280+0000","last_peered":"2024-05-09T10:51:06.401280+0000","last_clean":"2024-05-09T10:51:06.401280+0000","last_became_active":"2024-05-09T10:51:06.396277+0000","last_became_peered":"2024-05-09T10:51:06.396277+0000","last_unstale":"2024-05-09T10:51:06.401280+0000","last_undegraded":"2024-05-09T10:51:06.401280+0000","last_fullsized":"2024-05-09T10:51:06.401280+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:29.685531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388416+0000","last_change":"2024-05-09T10:51:06.388416+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388416+0000","last_undegraded":"2024-05-09T10:51:06.388416+0000","last_fullsized":"2024-05-09T10:51:06.388416+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408562+0000","last_change":"2024-05-09T10:51:06.405148+0000","last_active":"2024-05-09T10:51:06.408562+0000","last_peered":"2024-05-09T10:51:06.408562+0000","last_clean":"2024-05-09T10:51:06.408562+0000","last_became_active":"2024-05-09T10:51:06.403901+0000","last_became_peered":"2024-05-09T10:51:06.403901+0000","last_unstale":"2024-05-09T10:51:06.408562+0000","last_undegraded":"2024-05-09T10:51:06.408562+0000","last_fullsized":"2024-05-09T10:51:06.408562+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:29.685531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388109+0000","last_change":"2024-05-09T10:51:06.388109+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388109+0000","last_undegraded":"2024-05-09T10:51:06.388109+0000","last_fullsized":"2024-05-09T10:51:06.388109+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408642+0000","last_change":"2024-05-09T10:51:06.404951+0000","last_active":"2024-05-09T10:51:06.408642+0000","last_peered":"2024-05-09T10:51:06.408642+0000","last_clean":"2024-05-09T10:51:06.408642+0000","last_became_active":"2024-05-09T10:51:06.404574+0000","last_became_peered":"2024-05-09T10:51:06.404574+0000","last_unstale":"2024-05-09T10:51:06.408642+0000","last_undegraded":"2024-05-09T10:51:06.408642+0000","last_fullsized":"2024-05-09T10:51:06.408642+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:31:55.748009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.396433+0000","last_change":"2024-05-09T10:51:06.396433+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.396433+0000","last_undegraded":"2024-05-09T10:51:06.396433+0000","last_fullsized":"2024-05-09T10:51:06.396433+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408223+0000","last_change":"2024-05-09T10:51:06.404809+0000","last_active":"2024-05-09T10:51:06.408223+0000","last_peered":"2024-05-09T10:51:06.408223+0000","last_clean":"2024-05-09T10:51:06.408223+0000","last_became_active":"2024-05-09T10:51:06.404529+0000","last_became_peered":"2024-05-09T10:51:06.404529+0000","last_unstale":"2024-05-09T10:51:06.408223+0000","last_undegraded":"2024-05-09T10:51:06.408223+0000","last_fullsized":"2024-05-09T10:51:06.408223+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:30:44.917770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388071+0000","last_change":"2024-05-09T10:51:06.388071+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388071+0000","last_undegraded":"2024-05-09T10:51:06.388071+0000","last_fullsized":"2024-05-09T10:51:06.388071+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.402311+0000","last_change":"2024-05-09T10:51:06.398765+0000","last_active":"2024-05-09T10:51:06.402311+0000","last_peered":"2024-05-09T10:51:06.402311+0000","last_clean":"2024-05-09T10:51:06.402311+0000","last_became_active":"2024-05-09T10:51:06.397480+0000","last_became_peered":"2024-05-09T10:51:06.397480+0000","last_unstale":"2024-05-09T10:51:06.402311+0000","last_undegraded":"2024-05-09T10:51:06.402311+0000","last_fullsized":"2024-05-09T10:51:06.402311+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:29.685531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.387775+0000","last_change":"2024-05-09T10:51:06.387775+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.387775+0000","last_undegraded":"2024-05-09T10:51:06.387775+0000","last_fullsized":"2024-05-09T10:51:06.387775+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.402004+0000","last_change":"2024-05-09T10:51:06.399613+0000","last_active":"2024-05-09T10:51:06.402004+0000","last_peered":"2024-05-09T10:51:06.402004+0000","last_clean":"2024-05-09T10:51:06.402004+0000","last_became_active":"2024-05-09T10:51:06.399333+0000","last_became_peered":"2024-05-09T10:51:06.399333+0000","last_unstale":"2024-05-09T10:51:06.402004+0000","last_undegraded":"2024-05-09T10:51:06.402004+0000","last_fullsized":"2024-05-09T10:51:06.402004+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:05:03.035522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.389289+0000","last_change":"2024-05-09T10:51:06.389289+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.389289+0000","last_undegraded":"2024-05-09T10:51:06.389289+0000","last_fullsized":"2024-05-09T10:51:06.389289+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.400013+0000","last_change":"2024-05-09T10:51:06.396469+0000","last_active":"2024-05-09T10:51:06.400013+0000","last_peered":"2024-05-09T10:51:06.400013+0000","last_clean":"2024-05-09T10:51:06.400013+0000","last_became_active":"2024-05-09T10:51:06.396180+0000","last_became_peered":"2024-05-09T10:51:06.396180+0000","last_unstale":"2024-05-09T10:51:06.400013+0000","last_undegraded":"2024-05-09T10:51:06.400013+0000","last_fullsized":"2024-05-09T10:51:06.400013+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:21.607813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392198+0000","last_change":"2024-05-09T10:51:06.392198+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392198+0000","last_undegraded":"2024-05-09T10:51:06.392198+0000","last_fullsized":"2024-05-09T10:51:06.392198+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.395406+0000","last_change":"2024-05-09T10:51:06.394559+0000","last_active":"2024-05-09T10:51:06.395406+0000","last_peered":"2024-05-09T10:51:06.395406+0000","last_clean":"2024-05-09T10:51:06.395406+0000","last_became_active":"2024-05-09T10:51:06.394288+0000","last_became_peered":"2024-05-09T10:51:06.394288+0000","last_unstale":"2024-05-09T10:51:06.395406+0000","last_undegraded":"2024-05-09T10:51:06.395406+0000","last_fullsized":"2024-05-09T10:51:06.395406+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:21.607813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.389645+0000","last_change":"2024-05-09T10:51:06.389645+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.389645+0000","last_undegraded":"2024-05-09T10:51:06.389645+0000","last_fullsized":"2024-05-09T10:51:06.389645+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.404581+0000","last_change":"2024-05-09T10:51:06.400034+0000","last_active":"2024-05-09T10:51:06.404581+0000","last_peered":"2024-05-09T10:51:06.404581+0000","last_clean":"2024-05-09T10:51:06.404581+0000","last_became_active":"2024-05-09T10:51:06.397993+0000","last_became_peered":"2024-05-09T10:51:06.397993+0000","last_unstale":"2024-05-09T10:51:06.404581+0000","last_undegraded":"2024-05-09T10:51:06.404581+0000","last_fullsized":"2024-05-09T10:51:06.404581+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:51.058627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393340+0000","last_change":"2024-05-09T10:51:06.393340+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393340+0000","last_undegraded":"2024-05-09T10:51:06.393340+0000","last_fullsized":"2024-05-09T10:51:06.393340+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406276+0000","last_change":"2024-05-09T10:51:06.400625+0000","last_active":"2024-05-09T10:51:06.406276+0000","last_peered":"2024-05-09T10:51:06.406276+0000","last_clean":"2024-05-09T10:51:06.406276+0000","last_became_active":"2024-05-09T10:51:06.400119+0000","last_became_peered":"2024-05-09T10:51:06.400119+0000","last_unstale":"2024-05-09T10:51:06.406276+0000","last_undegraded":"2024-05-09T10:51:06.406276+0000","last_fullsized":"2024-05-09T10:51:06.406276+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:34:06.133956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395771+0000","last_change":"2024-05-09T10:51:06.395771+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395771+0000","last_undegraded":"2024-05-09T10:51:06.395771+0000","last_fullsized":"2024-05-09T10:51:06.395771+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408282+0000","last_change":"2024-05-09T10:51:06.406673+0000","last_active":"2024-05-09T10:51:06.408282+0000","last_peered":"2024-05-09T10:51:06.408282+0000","last_clean":"2024-05-09T10:51:06.408282+0000","last_became_active":"2024-05-09T10:51:06.406391+0000","last_became_peered":"2024-05-09T10:51:06.406391+0000","last_unstale":"2024-05-09T10:51:06.408282+0000","last_undegraded":"2024-05-09T10:51:06.408282+0000","last_fullsized":"2024-05-09T10:51:06.408282+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:45:51.854599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395869+0000","last_change":"2024-05-09T10:51:06.395869+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395869+0000","last_undegraded":"2024-05-09T10:51:06.395869+0000","last_fullsized":"2024-05-09T10:51:06.395869+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408907+0000","last_change":"2024-05-09T10:51:06.407600+0000","last_active":"2024-05-09T10:51:06.408907+0000","last_peered":"2024-05-09T10:51:06.408907+0000","last_clean":"2024-05-09T10:51:06.408907+0000","last_became_active":"2024-05-09T10:51:06.407354+0000","last_became_peered":"2024-05-09T10:51:06.407354+0000","last_unstale":"2024-05-09T10:51:06.408907+0000","last_undegraded":"2024-05-09T10:51:06.408907+0000","last_fullsized":"2024-05-09T10:51:06.408907+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:51.058627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":186,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.401241+0000","last_change":"2024-05-09T10:50:08.410364+0000","last_active":"2024-05-09T10:51:06.401241+0000","last_peered":"2024-05-09T10:51:06.401241+0000","last_clean":"2024-05-09T10:51:06.401241+0000","last_became_active":"2024-05-09T10:50:08.409457+0000","last_became_peered":"2024-05-09T10:50:08.409457+0000","last_unstale":"2024-05-09T10:51:06.401241+0000","last_undegraded":"2024-05-09T10:51:06.401241+0000","last_fullsized":"2024-05-09T10:51:06.401241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_clean_scrub_stamp":"2024-05-09T10:50:07.396510+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:03.566698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":917504,"data_stored":711419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":917504,"data_stored":711419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T10:51:11.035 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-05-09T10:51:11.035 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.393755+0000', 'last_change': '2024-05-09T10:51:06.393755+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.393755+0000', 'last_undegraded': '2024-05-09T10:51:06.393755+0000', 'last_fullsized': '2024-05-09T10:51:06.393755+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-09T10:51:11.035 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-05-09T10:51:11.035 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.392812+0000', 'last_change': '2024-05-09T10:51:06.392812+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.392812+0000', 'last_undegraded': '2024-05-09T10:51:06.392812+0000', 'last_fullsized': '2024-05-09T10:51:06.392812+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T10:51:11.036 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-05-09T10:51:11.036 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.388265+0000', 'last_change': '2024-05-09T10:51:06.388265+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.388265+0000', 'last_undegraded': '2024-05-09T10:51:06.388265+0000', 'last_fullsized': '2024-05-09T10:51:06.388265+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T10:51:11.036 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-05-09T10:51:11.036 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.393261+0000', 'last_change': '2024-05-09T10:51:06.393261+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.393261+0000', 'last_undegraded': '2024-05-09T10:51:06.393261+0000', 'last_fullsized': '2024-05-09T10:51:06.393261+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T10:51:11.036 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-05-09T10:51:11.036 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.389283+0000', 'last_change': '2024-05-09T10:51:06.389283+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.389283+0000', 'last_undegraded': '2024-05-09T10:51:06.389283+0000', 'last_fullsized': '2024-05-09T10:51:06.389283+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T10:51:11.036 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-05-09T10:51:11.036 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.394709+0000', 'last_change': '2024-05-09T10:51:06.394709+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.394709+0000', 'last_undegraded': '2024-05-09T10:51:06.394709+0000', 'last_fullsized': '2024-05-09T10:51:06.394709+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T10:51:11.036 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-05-09T10:51:11.036 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.395480+0000', 'last_change': '2024-05-09T10:51:06.395480+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.395480+0000', 'last_undegraded': '2024-05-09T10:51:06.395480+0000', 'last_fullsized': '2024-05-09T10:51:06.395480+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T10:51:11.037 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-05-09T10:51:11.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.398792+0000', 'last_change': '2024-05-09T10:51:06.398792+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.398792+0000', 'last_undegraded': '2024-05-09T10:51:06.398792+0000', 'last_fullsized': '2024-05-09T10:51:06.398792+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T10:51:11.037 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-05-09T10:51:11.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.394178+0000', 'last_change': '2024-05-09T10:51:06.394178+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.394178+0000', 'last_undegraded': '2024-05-09T10:51:06.394178+0000', 'last_fullsized': '2024-05-09T10:51:06.394178+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T10:51:11.037 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-05-09T10:51:11.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.392823+0000', 'last_change': '2024-05-09T10:51:06.392823+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.392823+0000', 'last_undegraded': '2024-05-09T10:51:06.392823+0000', 'last_fullsized': '2024-05-09T10:51:06.392823+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T10:51:11.037 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-05-09T10:51:11.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.396067+0000', 'last_change': '2024-05-09T10:51:06.396067+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.396067+0000', 'last_undegraded': '2024-05-09T10:51:06.396067+0000', 'last_fullsized': '2024-05-09T10:51:06.396067+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T10:51:11.037 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-05-09T10:51:11.037 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.388701+0000', 'last_change': '2024-05-09T10:51:06.388701+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.388701+0000', 'last_undegraded': '2024-05-09T10:51:06.388701+0000', 'last_fullsized': '2024-05-09T10:51:06.388701+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T10:51:11.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-05-09T10:51:11.038 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.388834+0000', 'last_change': '2024-05-09T10:51:06.388834+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.388834+0000', 'last_undegraded': '2024-05-09T10:51:06.388834+0000', 'last_fullsized': '2024-05-09T10:51:06.388834+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T10:51:11.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-05-09T10:51:11.038 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.393806+0000', 'last_change': '2024-05-09T10:51:06.393806+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.393806+0000', 'last_undegraded': '2024-05-09T10:51:06.393806+0000', 'last_fullsized': '2024-05-09T10:51:06.393806+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T10:51:11.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-05-09T10:51:11.038 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.388334+0000', 'last_change': '2024-05-09T10:51:06.388334+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.388334+0000', 'last_undegraded': '2024-05-09T10:51:06.388334+0000', 'last_fullsized': '2024-05-09T10:51:06.388334+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T10:51:11.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-05-09T10:51:11.038 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.396634+0000', 'last_change': '2024-05-09T10:51:06.396634+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.396634+0000', 'last_undegraded': '2024-05-09T10:51:06.396634+0000', 'last_fullsized': '2024-05-09T10:51:06.396634+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T10:51:11.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-05-09T10:51:11.038 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.397070+0000', 'last_change': '2024-05-09T10:51:06.397070+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.397070+0000', 'last_undegraded': '2024-05-09T10:51:06.397070+0000', 'last_fullsized': '2024-05-09T10:51:06.397070+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T10:51:11.038 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-05-09T10:51:11.039 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.392398+0000', 'last_change': '2024-05-09T10:51:06.392398+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.392398+0000', 'last_undegraded': '2024-05-09T10:51:06.392398+0000', 'last_fullsized': '2024-05-09T10:51:06.392398+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T10:51:11.039 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-05-09T10:51:11.039 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.395656+0000', 'last_change': '2024-05-09T10:51:06.395656+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.395656+0000', 'last_undegraded': '2024-05-09T10:51:06.395656+0000', 'last_fullsized': '2024-05-09T10:51:06.395656+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T10:51:11.039 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-05-09T10:51:11.039 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.388193+0000', 'last_change': '2024-05-09T10:51:06.388193+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.388193+0000', 'last_undegraded': '2024-05-09T10:51:06.388193+0000', 'last_fullsized': '2024-05-09T10:51:06.388193+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T10:51:11.039 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-05-09T10:51:11.039 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.398904+0000', 'last_change': '2024-05-09T10:51:06.398904+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.398904+0000', 'last_undegraded': '2024-05-09T10:51:06.398904+0000', 'last_fullsized': '2024-05-09T10:51:06.398904+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T10:51:11.039 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-05-09T10:51:11.039 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.396580+0000', 'last_change': '2024-05-09T10:51:06.396580+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.396580+0000', 'last_undegraded': '2024-05-09T10:51:06.396580+0000', 'last_fullsized': '2024-05-09T10:51:06.396580+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T10:51:11.039 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-05-09T10:51:11.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.391128+0000', 'last_change': '2024-05-09T10:51:06.391128+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.391128+0000', 'last_undegraded': '2024-05-09T10:51:06.391128+0000', 'last_fullsized': '2024-05-09T10:51:06.391128+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T10:51:11.040 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-05-09T10:51:11.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.387807+0000', 'last_change': '2024-05-09T10:51:06.387807+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.387807+0000', 'last_undegraded': '2024-05-09T10:51:06.387807+0000', 'last_fullsized': '2024-05-09T10:51:06.387807+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T10:51:11.040 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-05-09T10:51:11.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.391876+0000', 'last_change': '2024-05-09T10:51:06.391876+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.391876+0000', 'last_undegraded': '2024-05-09T10:51:06.391876+0000', 'last_fullsized': '2024-05-09T10:51:06.391876+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T10:51:11.040 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-05-09T10:51:11.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.391884+0000', 'last_change': '2024-05-09T10:51:06.391884+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.391884+0000', 'last_undegraded': '2024-05-09T10:51:06.391884+0000', 'last_fullsized': '2024-05-09T10:51:06.391884+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T10:51:11.040 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-05-09T10:51:11.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.392175+0000', 'last_change': '2024-05-09T10:51:06.392175+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.392175+0000', 'last_undegraded': '2024-05-09T10:51:06.392175+0000', 'last_fullsized': '2024-05-09T10:51:06.392175+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T10:51:11.040 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-05-09T10:51:11.040 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.388098+0000', 'last_change': '2024-05-09T10:51:06.388098+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.388098+0000', 'last_undegraded': '2024-05-09T10:51:06.388098+0000', 'last_fullsized': '2024-05-09T10:51:06.388098+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T10:51:11.041 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-05-09T10:51:11.041 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.398517+0000', 'last_change': '2024-05-09T10:51:06.398517+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.398517+0000', 'last_undegraded': '2024-05-09T10:51:06.398517+0000', 'last_fullsized': '2024-05-09T10:51:06.398517+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T10:51:11.041 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-05-09T10:51:11.041 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.396453+0000', 'last_change': '2024-05-09T10:51:06.396453+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.396453+0000', 'last_undegraded': '2024-05-09T10:51:06.396453+0000', 'last_fullsized': '2024-05-09T10:51:06.396453+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T10:51:11.041 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-05-09T10:51:11.041 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.392904+0000', 'last_change': '2024-05-09T10:51:06.392904+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.392904+0000', 'last_undegraded': '2024-05-09T10:51:06.392904+0000', 'last_fullsized': '2024-05-09T10:51:06.392904+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-09T10:51:11.041 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-05-09T10:51:11.041 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.398378+0000', 'last_change': '2024-05-09T10:51:06.398378+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.398378+0000', 'last_undegraded': '2024-05-09T10:51:06.398378+0000', 'last_fullsized': '2024-05-09T10:51:06.398378+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T10:51:11.041 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-05-09T10:51:11.041 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.390548+0000', 'last_change': '2024-05-09T10:51:06.390548+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.390548+0000', 'last_undegraded': '2024-05-09T10:51:06.390548+0000', 'last_fullsized': '2024-05-09T10:51:06.390548+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T10:51:11.042 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-05-09T10:51:11.042 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.393181+0000', 'last_change': '2024-05-09T10:51:06.393181+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.393181+0000', 'last_undegraded': '2024-05-09T10:51:06.393181+0000', 'last_fullsized': '2024-05-09T10:51:06.393181+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-09T10:51:11.042 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-05-09T10:51:11.042 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.399481+0000', 'last_change': '2024-05-09T10:51:06.399481+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.399481+0000', 'last_undegraded': '2024-05-09T10:51:06.399481+0000', 'last_fullsized': '2024-05-09T10:51:06.399481+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T10:51:11.042 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-05-09T10:51:11.042 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.391713+0000', 'last_change': '2024-05-09T10:51:06.391713+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.391713+0000', 'last_undegraded': '2024-05-09T10:51:06.391713+0000', 'last_fullsized': '2024-05-09T10:51:06.391713+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T10:51:11.042 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-05-09T10:51:11.042 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.395508+0000', 'last_change': '2024-05-09T10:51:06.395508+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.395508+0000', 'last_undegraded': '2024-05-09T10:51:06.395508+0000', 'last_fullsized': '2024-05-09T10:51:06.395508+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T10:51:11.042 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-05-09T10:51:11.042 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.387999+0000', 'last_change': '2024-05-09T10:51:06.387999+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.387999+0000', 'last_undegraded': '2024-05-09T10:51:06.387999+0000', 'last_fullsized': '2024-05-09T10:51:06.387999+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T10:51:11.042 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-05-09T10:51:11.042 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.397348+0000', 'last_change': '2024-05-09T10:51:06.397348+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.397348+0000', 'last_undegraded': '2024-05-09T10:51:06.397348+0000', 'last_fullsized': '2024-05-09T10:51:06.397348+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T10:51:11.043 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-05-09T10:51:11.043 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.391151+0000', 'last_change': '2024-05-09T10:51:06.391151+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.391151+0000', 'last_undegraded': '2024-05-09T10:51:06.391151+0000', 'last_fullsized': '2024-05-09T10:51:06.391151+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T10:51:11.043 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-05-09T10:51:11.043 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.388231+0000', 'last_change': '2024-05-09T10:51:06.388231+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.388231+0000', 'last_undegraded': '2024-05-09T10:51:06.388231+0000', 'last_fullsized': '2024-05-09T10:51:06.388231+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T10:51:11.043 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-05-09T10:51:11.043 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.393382+0000', 'last_change': '2024-05-09T10:51:06.393382+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.393382+0000', 'last_undegraded': '2024-05-09T10:51:06.393382+0000', 'last_fullsized': '2024-05-09T10:51:06.393382+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T10:51:11.043 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-05-09T10:51:11.043 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.394715+0000', 'last_change': '2024-05-09T10:51:06.394715+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.394715+0000', 'last_undegraded': '2024-05-09T10:51:06.394715+0000', 'last_fullsized': '2024-05-09T10:51:06.394715+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T10:51:11.043 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-05-09T10:51:11.043 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.389652+0000', 'last_change': '2024-05-09T10:51:06.389652+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.389652+0000', 'last_undegraded': '2024-05-09T10:51:06.389652+0000', 'last_fullsized': '2024-05-09T10:51:06.389652+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T10:51:11.043 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-05-09T10:51:11.044 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.393445+0000', 'last_change': '2024-05-09T10:51:06.393445+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.393445+0000', 'last_undegraded': '2024-05-09T10:51:06.393445+0000', 'last_fullsized': '2024-05-09T10:51:06.393445+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T10:51:11.044 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-05-09T10:51:11.044 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.395214+0000', 'last_change': '2024-05-09T10:51:06.395214+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.395214+0000', 'last_undegraded': '2024-05-09T10:51:06.395214+0000', 'last_fullsized': '2024-05-09T10:51:06.395214+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T10:51:11.044 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-05-09T10:51:11.044 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.397979+0000', 'last_change': '2024-05-09T10:51:06.397979+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.397979+0000', 'last_undegraded': '2024-05-09T10:51:06.397979+0000', 'last_fullsized': '2024-05-09T10:51:06.397979+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T10:51:11.044 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-05-09T10:51:11.044 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.392434+0000', 'last_change': '2024-05-09T10:51:06.392434+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.392434+0000', 'last_undegraded': '2024-05-09T10:51:06.392434+0000', 'last_fullsized': '2024-05-09T10:51:06.392434+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-09T10:51:11.044 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-05-09T10:51:11.044 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.398088+0000', 'last_change': '2024-05-09T10:51:06.398088+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.398088+0000', 'last_undegraded': '2024-05-09T10:51:06.398088+0000', 'last_fullsized': '2024-05-09T10:51:06.398088+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T10:51:11.044 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-05-09T10:51:11.044 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.388885+0000', 'last_change': '2024-05-09T10:51:06.388885+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.388885+0000', 'last_undegraded': '2024-05-09T10:51:06.388885+0000', 'last_fullsized': '2024-05-09T10:51:06.388885+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T10:51:11.044 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-05-09T10:51:11.045 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.392827+0000', 'last_change': '2024-05-09T10:51:06.392827+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.392827+0000', 'last_undegraded': '2024-05-09T10:51:06.392827+0000', 'last_fullsized': '2024-05-09T10:51:06.392827+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-09T10:51:11.045 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-05-09T10:51:11.045 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.388204+0000', 'last_change': '2024-05-09T10:51:06.388204+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.388204+0000', 'last_undegraded': '2024-05-09T10:51:06.388204+0000', 'last_fullsized': '2024-05-09T10:51:06.388204+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T10:51:11.045 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-05-09T10:51:11.045 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.388110+0000', 'last_change': '2024-05-09T10:51:06.388110+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.388110+0000', 'last_undegraded': '2024-05-09T10:51:06.388110+0000', 'last_fullsized': '2024-05-09T10:51:06.388110+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T10:51:11.045 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-05-09T10:51:11.045 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.388416+0000', 'last_change': '2024-05-09T10:51:06.388416+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.388416+0000', 'last_undegraded': '2024-05-09T10:51:06.388416+0000', 'last_fullsized': '2024-05-09T10:51:06.388416+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T10:51:11.045 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-05-09T10:51:11.045 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.388109+0000', 'last_change': '2024-05-09T10:51:06.388109+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.388109+0000', 'last_undegraded': '2024-05-09T10:51:06.388109+0000', 'last_fullsized': '2024-05-09T10:51:06.388109+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T10:51:11.045 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-05-09T10:51:11.045 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.396433+0000', 'last_change': '2024-05-09T10:51:06.396433+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.396433+0000', 'last_undegraded': '2024-05-09T10:51:06.396433+0000', 'last_fullsized': '2024-05-09T10:51:06.396433+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T10:51:11.046 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-05-09T10:51:11.046 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.388071+0000', 'last_change': '2024-05-09T10:51:06.388071+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.388071+0000', 'last_undegraded': '2024-05-09T10:51:06.388071+0000', 'last_fullsized': '2024-05-09T10:51:06.388071+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-09T10:51:11.046 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-05-09T10:51:11.046 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.387775+0000', 'last_change': '2024-05-09T10:51:06.387775+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.387775+0000', 'last_undegraded': '2024-05-09T10:51:06.387775+0000', 'last_fullsized': '2024-05-09T10:51:06.387775+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-09T10:51:11.046 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-05-09T10:51:11.046 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.389289+0000', 'last_change': '2024-05-09T10:51:06.389289+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.389289+0000', 'last_undegraded': '2024-05-09T10:51:06.389289+0000', 'last_fullsized': '2024-05-09T10:51:06.389289+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T10:51:11.046 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-05-09T10:51:11.046 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.392198+0000', 'last_change': '2024-05-09T10:51:06.392198+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.392198+0000', 'last_undegraded': '2024-05-09T10:51:06.392198+0000', 'last_fullsized': '2024-05-09T10:51:06.392198+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-09T10:51:11.046 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-05-09T10:51:11.046 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.389645+0000', 'last_change': '2024-05-09T10:51:06.389645+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.389645+0000', 'last_undegraded': '2024-05-09T10:51:06.389645+0000', 'last_fullsized': '2024-05-09T10:51:06.389645+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-09T10:51:11.046 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-05-09T10:51:11.046 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.393340+0000', 'last_change': '2024-05-09T10:51:06.393340+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.393340+0000', 'last_undegraded': '2024-05-09T10:51:06.393340+0000', 'last_fullsized': '2024-05-09T10:51:06.393340+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-09T10:51:11.047 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-05-09T10:51:11.047 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.395771+0000', 'last_change': '2024-05-09T10:51:06.395771+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.395771+0000', 'last_undegraded': '2024-05-09T10:51:06.395771+0000', 'last_fullsized': '2024-05-09T10:51:06.395771+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-09T10:51:11.047 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-05-09T10:51:11.047 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-05-09T10:51:06.395869+0000', 'last_change': '2024-05-09T10:51:06.395869+0000', 'last_active': '2024-05-09T10:51:06.382079+0000', 'last_peered': '2024-05-09T10:51:06.382079+0000', 'last_clean': '2024-05-09T10:51:06.382079+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-09T10:51:06.395869+0000', 'last_undegraded': '2024-05-09T10:51:06.395869+0000', 'last_fullsized': '2024-05-09T10:51:06.395869+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'last_clean_scrub_stamp': '2024-05-09T10:51:06.382079+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-09T10:51:11.047 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-09T10:51:11.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.176+0000 7fea32a12640 1 Processor -- start 2024-05-09T10:51:11.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.176+0000 7fea32a12640 1 -- start start 2024-05-09T10:51:11.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.177+0000 7fea32a12640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fea2c15f0e0 0x7fea2c0cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:11.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.177+0000 7fea32a12640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea2c15c190 con 0x7fea2c06c300 2024-05-09T10:51:11.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.177+0000 7fea32a12640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea2c15c470 con 0x7fea2c0cfae0 2024-05-09T10:51:11.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.177+0000 7fea32a12640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c15cda0 con 0x7fea2c15f0e0 2024-05-09T10:51:11.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.177+0000 7fea30faf640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fea2c06c300 0x7fea2c152150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52830/0 (socket says 172.21.15.98:52830) 2024-05-09T10:51:11.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.177+0000 7fea30faf640 1 -- 172.21.15.98:0/2787531772 learned_addr learned my addr 172.21.15.98:0/2787531772 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:11.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.177+0000 7fea2bfff640 1 --2- 172.21.15.98:0/2787531772 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fea2c15f0e0 0x7fea2c0cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:11.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.177+0000 7fea2affd640 1 -- 172.21.15.98:0/2787531772 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4052934728 0 0) 0x7fea2c15c190 con 0x7fea2c06c300 2024-05-09T10:51:11.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.177+0000 7fea2affd640 1 -- 172.21.15.98:0/2787531772 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea14003520 con 0x7fea2c06c300 2024-05-09T10:51:11.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.178+0000 7fea2affd640 1 -- 172.21.15.98:0/2787531772 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3955821158 0 0) 0x7fea14003520 con 0x7fea2c06c300 2024-05-09T10:51:11.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.178+0000 7fea2affd640 1 -- 172.21.15.98:0/2787531772 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fea2c15f0e0 msgr2=0x7fea2c0cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:11.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.178+0000 7fea2affd640 1 --2- 172.21.15.98:0/2787531772 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fea2c15f0e0 0x7fea2c0cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:11.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.178+0000 7fea2affd640 1 -- 172.21.15.98:0/2787531772 >> v1:172.21.15.105:6789/0 conn(0x7fea2c0cfae0 legacy=0x7fea2c06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:11.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.178+0000 7fea2affd640 1 -- 172.21.15.98:0/2787531772 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2c15c750 con 0x7fea2c06c300 2024-05-09T10:51:11.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.178+0000 7fea2affd640 1 -- 172.21.15.98:0/2787531772 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7fea240019f0 con 0x7fea2c06c300 2024-05-09T10:51:11.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.178+0000 7fea2affd640 1 -- 172.21.15.98:0/2787531772 >> v1:172.21.15.98:6789/0 conn(0x7fea2c06c300 legacy=0x7fea2c152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:11.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.178+0000 7fea2affd640 1 --2- 172.21.15.98:0/2787531772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea140039e0 0x7fea14003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:11.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.178+0000 7fea2affd640 1 --2- 172.21.15.98:0/2787531772 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fea2c15f0e0 0x7fea14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:11.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.178+0000 7fea2affd640 1 --2- 172.21.15.98:0/2787531772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea14004d40 0x7fea140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:11.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.178+0000 7fea2affd640 1 -- 172.21.15.98:0/2787531772 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c15cda0 con 0x7fea140039e0 2024-05-09T10:51:11.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.178+0000 7fea2affd640 1 -- 172.21.15.98:0/2787531772 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1400ba70 con 0x7fea14004d40 2024-05-09T10:51:11.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.178+0000 7fea2affd640 1 -- 172.21.15.98:0/2787531772 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1400bbf0 con 0x7fea2c15f0e0 2024-05-09T10:51:11.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.178+0000 7fea2bfff640 1 --2- 172.21.15.98:0/2787531772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea140039e0 0x7fea14003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:11.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.178+0000 7fea2b7fe640 1 --2- 172.21.15.98:0/2787531772 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fea2c15f0e0 0x7fea14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:11.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.178+0000 7fea30faf640 1 --2- 172.21.15.98:0/2787531772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea14004d40 0x7fea140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:11.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.178+0000 7fea2bfff640 1 -- 172.21.15.98:0/2787531772 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fea2c15f0e0 msgr2=0x7fea14004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:11.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.178+0000 7fea2bfff640 1 --2- 172.21.15.98:0/2787531772 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fea2c15f0e0 0x7fea14004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:11.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.178+0000 7fea2bfff640 1 -- 172.21.15.98:0/2787531772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea14004d40 msgr2=0x7fea140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:11.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.178+0000 7fea2bfff640 1 --2- 172.21.15.98:0/2787531772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea14004d40 0x7fea140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:11.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.178+0000 7fea2bfff640 1 -- 172.21.15.98:0/2787531772 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea14003520 con 0x7fea140039e0 2024-05-09T10:51:11.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.179+0000 7fea2bfff640 1 --2- 172.21.15.98:0/2787531772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea140039e0 0x7fea14003de0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009ac0 tx=0x7fea14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e196006b77969975 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:11.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.179+0000 7fea2affd640 1 -- 172.21.15.98:0/2787531772 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c036070 con 0x7fea140039e0 2024-05-09T10:51:11.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.179+0000 7fea2affd640 1 -- 172.21.15.98:0/2787531772 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1c031400 con 0x7fea140039e0 2024-05-09T10:51:11.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.179+0000 7fea2affd640 1 -- 172.21.15.98:0/2787531772 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c009690 con 0x7fea140039e0 2024-05-09T10:51:11.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.179+0000 7fea32a12640 1 -- 172.21.15.98:0/2787531772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea140039e0 msgr2=0x7fea14003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:11.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.179+0000 7fea32a12640 1 --2- 172.21.15.98:0/2787531772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea140039e0 0x7fea14003de0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009ac0 tx=0x7fea14005830 comp rx=0 tx=0).stop 2024-05-09T10:51:11.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.179+0000 7fea30faf640 1 -- 172.21.15.98:0/2787531772 reap_dead start 2024-05-09T10:51:11.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.179+0000 7fea32a12640 1 -- 172.21.15.98:0/2787531772 shutdown_connections 2024-05-09T10:51:11.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.179+0000 7fea32a12640 1 -- 172.21.15.98:0/2787531772 >> 172.21.15.98:0/2787531772 conn(0x7fea2c06cae0 msgr2=0x7fea2c06cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:11.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.179+0000 7fea32a12640 1 -- 172.21.15.98:0/2787531772 shutdown_connections 2024-05-09T10:51:11.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.179+0000 7fea32a12640 1 -- 172.21.15.98:0/2787531772 wait complete. 2024-05-09T10:51:11.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.181+0000 7fea32a12640 1 Processor -- start 2024-05-09T10:51:11.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.181+0000 7fea32a12640 1 -- start start 2024-05-09T10:51:11.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.181+0000 7fea32a12640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fea140039e0 0x7fea2c14e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:11.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.181+0000 7fea32a12640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea2c14e9a0 0x7fea2c1f1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:11.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.181+0000 7fea32a12640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea2c1f1960 0x7fea2c1f1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:11.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.182+0000 7fea32a12640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c0afa70 con 0x7fea2c1f1960 2024-05-09T10:51:11.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.182+0000 7fea32a12640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c0af8f0 con 0x7fea2c14e9a0 2024-05-09T10:51:11.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.182+0000 7fea32a12640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c15e2c0 con 0x7fea140039e0 2024-05-09T10:51:11.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.182+0000 7fea30faf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea2c1f1960 0x7fea2c1f1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:11.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.182+0000 7fea30faf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea2c1f1960 0x7fea2c1f1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49098/0 (socket says 172.21.15.98:49098) 2024-05-09T10:51:11.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.182+0000 7fea30faf640 1 -- 172.21.15.98:0/905625319 learned_addr learned my addr 172.21.15.98:0/905625319 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:11.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.182+0000 7fea2bfff640 1 --2- 172.21.15.98:0/905625319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fea140039e0 0x7fea2c14e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:11.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.182+0000 7fea2b7fe640 1 --2- 172.21.15.98:0/905625319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea2c14e9a0 0x7fea2c1f1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:11.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.182+0000 7fea30faf640 1 -- 172.21.15.98:0/905625319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fea140039e0 msgr2=0x7fea2c14e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:11.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.182+0000 7fea30faf640 1 --2- 172.21.15.98:0/905625319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fea140039e0 0x7fea2c14e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:11.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.182+0000 7fea30faf640 1 -- 172.21.15.98:0/905625319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea2c14e9a0 msgr2=0x7fea2c1f1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:11.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.182+0000 7fea30faf640 1 --2- 172.21.15.98:0/905625319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea2c14e9a0 0x7fea2c1f1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:11.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.182+0000 7fea30faf640 1 -- 172.21.15.98:0/905625319 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2c14eeb0 con 0x7fea2c1f1960 2024-05-09T10:51:11.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.182+0000 7fea30faf640 1 --2- 172.21.15.98:0/905625319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea2c1f1960 0x7fea2c1f1de0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fea2c070560 tx=0x7fea2400a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:11.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.182+0000 7fea28ff9640 1 -- 172.21.15.98:0/905625319 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2400aca0 con 0x7fea2c1f1960 2024-05-09T10:51:11.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.182+0000 7fea28ff9640 1 -- 172.21.15.98:0/905625319 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea24019070 con 0x7fea2c1f1960 2024-05-09T10:51:11.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.182+0000 7fea32a12640 1 -- 172.21.15.98:0/905625319 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2c14f190 con 0x7fea2c1f1960 2024-05-09T10:51:11.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.182+0000 7fea28ff9640 1 -- 172.21.15.98:0/905625319 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea24013630 con 0x7fea2c1f1960 2024-05-09T10:51:11.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.182+0000 7fea32a12640 1 -- 172.21.15.98:0/905625319 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea2c156830 con 0x7fea2c1f1960 2024-05-09T10:51:11.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.183+0000 7fea32a12640 1 -- 172.21.15.98:0/905625319 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea2c069cb0 con 0x7fea2c1f1960 2024-05-09T10:51:11.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.184+0000 7fea28ff9640 1 -- 172.21.15.98:0/905625319 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea240137d0 con 0x7fea2c1f1960 2024-05-09T10:51:11.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.184+0000 7fea28ff9640 1 --2- 172.21.15.98:0/905625319 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fea1806b500 0x7fea1806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:11.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.184+0000 7fea28ff9640 1 -- 172.21.15.98:0/905625319 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea2408a5f0 con 0x7fea2c1f1960 2024-05-09T10:51:11.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.184+0000 7fea2bfff640 1 --2- 172.21.15.98:0/905625319 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fea1806b500 0x7fea1806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:11.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.185+0000 7fea2bfff640 1 --2- 172.21.15.98:0/905625319 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fea1806b500 0x7fea1806d9c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fea1c033000 tx=0x7fea1c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:11.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.188+0000 7fea28ff9640 1 -- 172.21.15.98:0/905625319 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fea24058e20 con 0x7fea2c1f1960 2024-05-09T10:51:11.358 INFO:tasks.ceph.mds.a.smithi098.stderr:2024-05-09T10:51:11.365+0000 7f7557619640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T10:51:11.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.427+0000 7fea32a12640 1 -- 172.21.15.98:0/905625319 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fea2c14f360 con 0x7fea2c1f1960 2024-05-09T10:51:11.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.427+0000 7fea28ff9640 1 -- 172.21.15.98:0/905625319 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1581 (secure 0 0 0) 0x7fea2405cec0 con 0x7fea2c1f1960 2024-05-09T10:51:11.421 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:11.421 INFO:teuthology.orchestra.run.smithi098.stdout:{"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":76,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":31,"num_osds":8,"num_up_osds":8,"osd_up_since":1715251806,"num_in_osds":8,"osd_in_since":1715251797,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225157120,"bytes_avail":772868956160,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":13,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5127}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-05-09T10:51:09.374981+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-09T10:51:11.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.429+0000 7fea32a12640 1 -- 172.21.15.98:0/905625319 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fea1806b500 msgr2=0x7fea1806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:11.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.429+0000 7fea32a12640 1 --2- 172.21.15.98:0/905625319 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fea1806b500 0x7fea1806d9c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fea1c033000 tx=0x7fea1c008030 comp rx=0 tx=0).stop 2024-05-09T10:51:11.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.429+0000 7fea32a12640 1 -- 172.21.15.98:0/905625319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea2c1f1960 msgr2=0x7fea2c1f1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:11.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.429+0000 7fea32a12640 1 --2- 172.21.15.98:0/905625319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea2c1f1960 0x7fea2c1f1de0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fea2c070560 tx=0x7fea2400a6a0 comp rx=0 tx=0).stop 2024-05-09T10:51:11.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.429+0000 7fea32a12640 1 -- 172.21.15.98:0/905625319 shutdown_connections 2024-05-09T10:51:11.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.429+0000 7fea32a12640 1 --2- 172.21.15.98:0/905625319 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fea1806b500 0x7fea1806d9c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:11.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.429+0000 7fea32a12640 1 --2- 172.21.15.98:0/905625319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea2c1f1960 0x7fea2c1f1de0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:11.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.429+0000 7fea32a12640 1 --2- 172.21.15.98:0/905625319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea2c14e9a0 0x7fea2c1f1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:11.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.429+0000 7fea32a12640 1 --2- 172.21.15.98:0/905625319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fea140039e0 0x7fea2c14e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:11.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.429+0000 7fea32a12640 1 -- 172.21.15.98:0/905625319 >> 172.21.15.98:0/905625319 conn(0x7fea2c06cae0 msgr2=0x7fea2c0b0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:11.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.429+0000 7fea32a12640 1 -- 172.21.15.98:0/905625319 shutdown_connections 2024-05-09T10:51:11.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.429+0000 7fea32a12640 1 -- 172.21.15.98:0/905625319 wait complete. 2024-05-09T10:51:11.433 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-09T10:51:11.433 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T10:51:11.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.540+0000 7f66937f5640 1 Processor -- start 2024-05-09T10:51:11.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.540+0000 7f66937f5640 1 -- start start 2024-05-09T10:51:11.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.541+0000 7f66937f5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f668c15bdc0 0x7f668c0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:11.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.541+0000 7f66937f5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f668c0b4f10 0x7f668c0cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:11.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.541+0000 7f66937f5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f668c0cc4c0 con 0x7f668c0ceb70 2024-05-09T10:51:11.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.541+0000 7f66937f5640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f668c15c1a0 con 0x7f668c0b4f10 2024-05-09T10:51:11.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.541+0000 7f66937f5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f668c0cf3c0 con 0x7f668c15bdc0 2024-05-09T10:51:11.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.541+0000 7f6691d92640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f668c0b4f10 0x7f668c0cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:11.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.541+0000 7f6690d90640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f668c0ceb70 0x7f668c0b4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52836/0 (socket says 172.21.15.98:52836) 2024-05-09T10:51:11.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.541+0000 7f6690d90640 1 -- 172.21.15.98:0/1588546790 learned_addr learned my addr 172.21.15.98:0/1588546790 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:11.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.541+0000 7f6691591640 1 --2- 172.21.15.98:0/1588546790 >> v2:172.21.15.105:3300/0 conn(0x7f668c15bdc0 0x7f668c0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:11.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.541+0000 7f6691d92640 1 -- 172.21.15.98:0/1588546790 >> v2:172.21.15.105:3300/0 conn(0x7f668c15bdc0 msgr2=0x7f668c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:11.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.541+0000 7f6691d92640 1 --2- 172.21.15.98:0/1588546790 >> v2:172.21.15.105:3300/0 conn(0x7f668c15bdc0 0x7f668c0cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:11.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.541+0000 7f6691d92640 1 -- 172.21.15.98:0/1588546790 >> v1:172.21.15.98:6789/0 conn(0x7f668c0ceb70 legacy=0x7f668c0b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:11.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.541+0000 7f6691d92640 1 -- 172.21.15.98:0/1588546790 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c0cf0e0 con 0x7f668c0b4f10 2024-05-09T10:51:11.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.541+0000 7f667ffff640 1 -- 172.21.15.98:0/1588546790 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3349834921 0 0) 0x7f668c0cc4c0 con 0x7f668c0ceb70 2024-05-09T10:51:11.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.541+0000 7f6691591640 1 --2- 172.21.15.98:0/1588546790 >> v2:172.21.15.105:3300/0 conn(0x7f668c15bdc0 0x7f668c0cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:11.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.542+0000 7f6691d92640 1 --2- 172.21.15.98:0/1588546790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f668c0b4f10 0x7f668c0cfa30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f668c070f70 tx=0x7f668402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fa6e0f08b6cfd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:11.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.542+0000 7f667ffff640 1 -- 172.21.15.98:0/1588546790 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6684007740 con 0x7f668c0b4f10 2024-05-09T10:51:11.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.542+0000 7f667ffff640 1 -- 172.21.15.98:0/1588546790 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66840078e0 con 0x7f668c0b4f10 2024-05-09T10:51:11.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.542+0000 7f667ffff640 1 -- 172.21.15.98:0/1588546790 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6684007cb0 con 0x7f668c0b4f10 2024-05-09T10:51:11.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.543+0000 7f66937f5640 1 -- 172.21.15.98:0/1588546790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f668c0b4f10 msgr2=0x7f668c0cfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:11.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.543+0000 7f66937f5640 1 --2- 172.21.15.98:0/1588546790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f668c0b4f10 0x7f668c0cfa30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f668c070f70 tx=0x7f668402a680 comp rx=0 tx=0).stop 2024-05-09T10:51:11.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.543+0000 7f66937f5640 1 -- 172.21.15.98:0/1588546790 shutdown_connections 2024-05-09T10:51:11.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.543+0000 7f66937f5640 1 --2- 172.21.15.98:0/1588546790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f668c0b4f10 0x7f668c0cfa30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:11.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.543+0000 7f66937f5640 1 --2- 172.21.15.98:0/1588546790 >> v2:172.21.15.105:3300/0 conn(0x7f668c15bdc0 0x7f668c0cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:11.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.543+0000 7f66937f5640 1 -- 172.21.15.98:0/1588546790 >> 172.21.15.98:0/1588546790 conn(0x7f668c0b34e0 msgr2=0x7f668c0b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:11.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.543+0000 7f66937f5640 1 -- 172.21.15.98:0/1588546790 shutdown_connections 2024-05-09T10:51:11.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.543+0000 7f66937f5640 1 -- 172.21.15.98:0/1588546790 wait complete. 2024-05-09T10:51:11.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.544+0000 7f66937f5640 1 Processor -- start 2024-05-09T10:51:11.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.544+0000 7f66937f5640 1 -- start start 2024-05-09T10:51:11.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.545+0000 7f66937f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f668c0b4f10 0x7f668c14ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:11.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.545+0000 7f66937f5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f668c0ceb70 0x7f668c0c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:11.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.545+0000 7f66937f5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f668c15bdc0 0x7f668c0c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:11.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.545+0000 7f66937f5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c0ccd20 con 0x7f668c0ceb70 2024-05-09T10:51:11.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.545+0000 7f66937f5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c0cca40 con 0x7f668c0b4f10 2024-05-09T10:51:11.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.545+0000 7f66937f5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f668c0b41b0 con 0x7f668c15bdc0 2024-05-09T10:51:11.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.545+0000 7f6690d90640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f668c0ceb70 0x7f668c0c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:11.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.545+0000 7f6690d90640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f668c0ceb70 0x7f668c0c6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49108/0 (socket says 172.21.15.98:49108) 2024-05-09T10:51:11.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.545+0000 7f6690d90640 1 -- 172.21.15.98:0/3845281081 learned_addr learned my addr 172.21.15.98:0/3845281081 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:11.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.545+0000 7f6691d92640 1 --2- 172.21.15.98:0/3845281081 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f668c15bdc0 0x7f668c0c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:11.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.545+0000 7f6691591640 1 --2- 172.21.15.98:0/3845281081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f668c0b4f10 0x7f668c14ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:11.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.545+0000 7f6690d90640 1 -- 172.21.15.98:0/3845281081 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f668c15bdc0 msgr2=0x7f668c0c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:11.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.545+0000 7f6690d90640 1 --2- 172.21.15.98:0/3845281081 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f668c15bdc0 0x7f668c0c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:11.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.545+0000 7f6690d90640 1 -- 172.21.15.98:0/3845281081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f668c0b4f10 msgr2=0x7f668c14ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:11.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.545+0000 7f6690d90640 1 --2- 172.21.15.98:0/3845281081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f668c0b4f10 0x7f668c14ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:11.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.545+0000 7f6690d90640 1 -- 172.21.15.98:0/3845281081 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c0c5580 con 0x7f668c0ceb70 2024-05-09T10:51:11.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.545+0000 7f6690d90640 1 --2- 172.21.15.98:0/3845281081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f668c0ceb70 0x7f668c0c6300 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f668000c4d0 tx=0x7f668000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:11.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.545+0000 7f667dffb640 1 -- 172.21.15.98:0/3845281081 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6680018070 con 0x7f668c0ceb70 2024-05-09T10:51:11.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.545+0000 7f667dffb640 1 -- 172.21.15.98:0/3845281081 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6680007500 con 0x7f668c0ceb70 2024-05-09T10:51:11.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.545+0000 7f667dffb640 1 -- 172.21.15.98:0/3845281081 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6680004a20 con 0x7f668c0ceb70 2024-05-09T10:51:11.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.545+0000 7f66937f5640 1 -- 172.21.15.98:0/3845281081 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f668c0c5820 con 0x7f668c0ceb70 2024-05-09T10:51:11.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.546+0000 7f66937f5640 1 -- 172.21.15.98:0/3845281081 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668c15edb0 con 0x7f668c0ceb70 2024-05-09T10:51:11.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.547+0000 7f66937f5640 1 -- 172.21.15.98:0/3845281081 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f668c0cc480 con 0x7f668c0ceb70 2024-05-09T10:51:11.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.547+0000 7f667dffb640 1 -- 172.21.15.98:0/3845281081 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f668001b070 con 0x7f668c0ceb70 2024-05-09T10:51:11.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.547+0000 7f667dffb640 1 --2- 172.21.15.98:0/3845281081 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f667006b4d0 0x7f667006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:11.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.548+0000 7f667dffb640 1 -- 172.21.15.98:0/3845281081 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6680089480 con 0x7f668c0ceb70 2024-05-09T10:51:11.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.548+0000 7f6691591640 1 --2- 172.21.15.98:0/3845281081 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f667006b4d0 0x7f667006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:11.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.548+0000 7f6691591640 1 --2- 172.21.15.98:0/3845281081 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f667006b4d0 0x7f667006d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6688006fd0 tx=0x7f6688008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:11.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.552+0000 7f667dffb640 1 -- 172.21.15.98:0/3845281081 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f668c0cc480 con 0x7f668c0ceb70 2024-05-09T10:51:11.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.719+0000 7f66937f5640 1 -- 172.21.15.98:0/3845281081 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f668c0b7930 con 0x7f667006b4d0 2024-05-09T10:51:11.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.725+0000 7f667dffb640 1 -- 172.21.15.98:0/3845281081 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246634 (secure 0 0 0) 0x7f668c0b7930 con 0x7f667006b4d0 2024-05-09T10:51:11.721 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:11.723 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-05-09T10:51:11.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.728+0000 7f66937f5640 1 -- 172.21.15.98:0/3845281081 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f667006b4d0 msgr2=0x7f667006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:11.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.728+0000 7f66937f5640 1 --2- 172.21.15.98:0/3845281081 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f667006b4d0 0x7f667006d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6688006fd0 tx=0x7f6688008040 comp rx=0 tx=0).stop 2024-05-09T10:51:11.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.728+0000 7f66937f5640 1 -- 172.21.15.98:0/3845281081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f668c0ceb70 msgr2=0x7f668c0c6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:11.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.728+0000 7f66937f5640 1 --2- 172.21.15.98:0/3845281081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f668c0ceb70 0x7f668c0c6300 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f668000c4d0 tx=0x7f668000c990 comp rx=0 tx=0).stop 2024-05-09T10:51:11.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.728+0000 7f66937f5640 1 -- 172.21.15.98:0/3845281081 shutdown_connections 2024-05-09T10:51:11.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.728+0000 7f66937f5640 1 --2- 172.21.15.98:0/3845281081 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f667006b4d0 0x7f667006d990 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:11.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.728+0000 7f66937f5640 1 --2- 172.21.15.98:0/3845281081 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f668c15bdc0 0x7f668c0c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:11.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.728+0000 7f66937f5640 1 --2- 172.21.15.98:0/3845281081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f668c0ceb70 0x7f668c0c6300 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:11.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.728+0000 7f66937f5640 1 --2- 172.21.15.98:0/3845281081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f668c0b4f10 0x7f668c14ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:11.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.728+0000 7f66937f5640 1 -- 172.21.15.98:0/3845281081 >> 172.21.15.98:0/3845281081 conn(0x7f668c0b34e0 msgr2=0x7f668c06e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:11.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.729+0000 7f66937f5640 1 -- 172.21.15.98:0/3845281081 shutdown_connections 2024-05-09T10:51:11.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:11.729+0000 7f66937f5640 1 -- 172.21.15.98:0/3845281081 wait complete. 2024-05-09T10:51:11.732 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-05-09T10:51:11.375229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219880,"kb_used_data":3688,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754840,"statfs":{"total":773094113280,"available":772868956160,"internally_reserved":0,"allocated":3776512,"data_stored":2149144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.992741"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393755+0000","last_change":"2024-05-09T10:51:06.393755+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393755+0000","last_undegraded":"2024-05-09T10:51:06.393755+0000","last_fullsized":"2024-05-09T10:51:06.393755+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392812+0000","last_change":"2024-05-09T10:51:06.392812+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392812+0000","last_undegraded":"2024-05-09T10:51:06.392812+0000","last_fullsized":"2024-05-09T10:51:06.392812+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388265+0000","last_change":"2024-05-09T10:51:06.388265+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388265+0000","last_undegraded":"2024-05-09T10:51:06.388265+0000","last_fullsized":"2024-05-09T10:51:06.388265+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393261+0000","last_change":"2024-05-09T10:51:06.393261+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393261+0000","last_undegraded":"2024-05-09T10:51:06.393261+0000","last_fullsized":"2024-05-09T10:51:06.393261+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.389283+0000","last_change":"2024-05-09T10:51:06.389283+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.389283+0000","last_undegraded":"2024-05-09T10:51:06.389283+0000","last_fullsized":"2024-05-09T10:51:06.389283+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.394709+0000","last_change":"2024-05-09T10:51:06.394709+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.394709+0000","last_undegraded":"2024-05-09T10:51:06.394709+0000","last_fullsized":"2024-05-09T10:51:06.394709+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395480+0000","last_change":"2024-05-09T10:51:06.395480+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395480+0000","last_undegraded":"2024-05-09T10:51:06.395480+0000","last_fullsized":"2024-05-09T10:51:06.395480+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.398792+0000","last_change":"2024-05-09T10:51:06.398792+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.398792+0000","last_undegraded":"2024-05-09T10:51:06.398792+0000","last_fullsized":"2024-05-09T10:51:06.398792+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.394178+0000","last_change":"2024-05-09T10:51:06.394178+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.394178+0000","last_undegraded":"2024-05-09T10:51:06.394178+0000","last_fullsized":"2024-05-09T10:51:06.394178+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392823+0000","last_change":"2024-05-09T10:51:06.392823+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392823+0000","last_undegraded":"2024-05-09T10:51:06.392823+0000","last_fullsized":"2024-05-09T10:51:06.392823+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.396067+0000","last_change":"2024-05-09T10:51:06.396067+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.396067+0000","last_undegraded":"2024-05-09T10:51:06.396067+0000","last_fullsized":"2024-05-09T10:51:06.396067+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388701+0000","last_change":"2024-05-09T10:51:06.388701+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388701+0000","last_undegraded":"2024-05-09T10:51:06.388701+0000","last_fullsized":"2024-05-09T10:51:06.388701+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388834+0000","last_change":"2024-05-09T10:51:06.388834+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388834+0000","last_undegraded":"2024-05-09T10:51:06.388834+0000","last_fullsized":"2024-05-09T10:51:06.388834+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393806+0000","last_change":"2024-05-09T10:51:06.393806+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393806+0000","last_undegraded":"2024-05-09T10:51:06.393806+0000","last_fullsized":"2024-05-09T10:51:06.393806+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388334+0000","last_change":"2024-05-09T10:51:06.388334+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388334+0000","last_undegraded":"2024-05-09T10:51:06.388334+0000","last_fullsized":"2024-05-09T10:51:06.388334+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.396634+0000","last_change":"2024-05-09T10:51:06.396634+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.396634+0000","last_undegraded":"2024-05-09T10:51:06.396634+0000","last_fullsized":"2024-05-09T10:51:06.396634+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.397070+0000","last_change":"2024-05-09T10:51:06.397070+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.397070+0000","last_undegraded":"2024-05-09T10:51:06.397070+0000","last_fullsized":"2024-05-09T10:51:06.397070+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392398+0000","last_change":"2024-05-09T10:51:06.392398+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392398+0000","last_undegraded":"2024-05-09T10:51:06.392398+0000","last_fullsized":"2024-05-09T10:51:06.392398+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395656+0000","last_change":"2024-05-09T10:51:06.395656+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395656+0000","last_undegraded":"2024-05-09T10:51:06.395656+0000","last_fullsized":"2024-05-09T10:51:06.395656+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388193+0000","last_change":"2024-05-09T10:51:06.388193+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388193+0000","last_undegraded":"2024-05-09T10:51:06.388193+0000","last_fullsized":"2024-05-09T10:51:06.388193+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.398904+0000","last_change":"2024-05-09T10:51:06.398904+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.398904+0000","last_undegraded":"2024-05-09T10:51:06.398904+0000","last_fullsized":"2024-05-09T10:51:06.398904+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.396580+0000","last_change":"2024-05-09T10:51:06.396580+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.396580+0000","last_undegraded":"2024-05-09T10:51:06.396580+0000","last_fullsized":"2024-05-09T10:51:06.396580+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.391128+0000","last_change":"2024-05-09T10:51:06.391128+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.391128+0000","last_undegraded":"2024-05-09T10:51:06.391128+0000","last_fullsized":"2024-05-09T10:51:06.391128+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.387807+0000","last_change":"2024-05-09T10:51:06.387807+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.387807+0000","last_undegraded":"2024-05-09T10:51:06.387807+0000","last_fullsized":"2024-05-09T10:51:06.387807+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.391876+0000","last_change":"2024-05-09T10:51:06.391876+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.391876+0000","last_undegraded":"2024-05-09T10:51:06.391876+0000","last_fullsized":"2024-05-09T10:51:06.391876+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.391884+0000","last_change":"2024-05-09T10:51:06.391884+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.391884+0000","last_undegraded":"2024-05-09T10:51:06.391884+0000","last_fullsized":"2024-05-09T10:51:06.391884+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392175+0000","last_change":"2024-05-09T10:51:06.392175+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392175+0000","last_undegraded":"2024-05-09T10:51:06.392175+0000","last_fullsized":"2024-05-09T10:51:06.392175+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388098+0000","last_change":"2024-05-09T10:51:06.388098+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388098+0000","last_undegraded":"2024-05-09T10:51:06.388098+0000","last_fullsized":"2024-05-09T10:51:06.388098+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.398517+0000","last_change":"2024-05-09T10:51:06.398517+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.398517+0000","last_undegraded":"2024-05-09T10:51:06.398517+0000","last_fullsized":"2024-05-09T10:51:06.398517+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.396453+0000","last_change":"2024-05-09T10:51:06.396453+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.396453+0000","last_undegraded":"2024-05-09T10:51:06.396453+0000","last_fullsized":"2024-05-09T10:51:06.396453+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392904+0000","last_change":"2024-05-09T10:51:06.392904+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392904+0000","last_undegraded":"2024-05-09T10:51:06.392904+0000","last_fullsized":"2024-05-09T10:51:06.392904+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.398378+0000","last_change":"2024-05-09T10:51:06.398378+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.398378+0000","last_undegraded":"2024-05-09T10:51:06.398378+0000","last_fullsized":"2024-05-09T10:51:06.398378+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.390548+0000","last_change":"2024-05-09T10:51:06.390548+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.390548+0000","last_undegraded":"2024-05-09T10:51:06.390548+0000","last_fullsized":"2024-05-09T10:51:06.390548+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.403221+0000","last_change":"2024-05-09T10:51:06.399065+0000","last_active":"2024-05-09T10:51:06.403221+0000","last_peered":"2024-05-09T10:51:06.403221+0000","last_clean":"2024-05-09T10:51:06.403221+0000","last_became_active":"2024-05-09T10:51:06.398631+0000","last_became_peered":"2024-05-09T10:51:06.398631+0000","last_unstale":"2024-05-09T10:51:06.403221+0000","last_undegraded":"2024-05-09T10:51:06.403221+0000","last_fullsized":"2024-05-09T10:51:06.403221+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:21.607813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393181+0000","last_change":"2024-05-09T10:51:06.393181+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393181+0000","last_undegraded":"2024-05-09T10:51:06.393181+0000","last_fullsized":"2024-05-09T10:51:06.393181+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.405979+0000","last_change":"2024-05-09T10:51:06.405043+0000","last_active":"2024-05-09T10:51:06.405979+0000","last_peered":"2024-05-09T10:51:06.405979+0000","last_clean":"2024-05-09T10:51:06.405979+0000","last_became_active":"2024-05-09T10:51:06.404375+0000","last_became_peered":"2024-05-09T10:51:06.404375+0000","last_unstale":"2024-05-09T10:51:06.405979+0000","last_undegraded":"2024-05-09T10:51:06.405979+0000","last_fullsized":"2024-05-09T10:51:06.405979+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:03.704175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.399481+0000","last_change":"2024-05-09T10:51:06.399481+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.399481+0000","last_undegraded":"2024-05-09T10:51:06.399481+0000","last_fullsized":"2024-05-09T10:51:06.399481+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406256+0000","last_change":"2024-05-09T10:51:06.403754+0000","last_active":"2024-05-09T10:51:06.406256+0000","last_peered":"2024-05-09T10:51:06.406256+0000","last_clean":"2024-05-09T10:51:06.406256+0000","last_became_active":"2024-05-09T10:51:06.403476+0000","last_became_peered":"2024-05-09T10:51:06.403476+0000","last_unstale":"2024-05-09T10:51:06.406256+0000","last_undegraded":"2024-05-09T10:51:06.406256+0000","last_fullsized":"2024-05-09T10:51:06.406256+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:48:18.069829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.391713+0000","last_change":"2024-05-09T10:51:06.391713+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.391713+0000","last_undegraded":"2024-05-09T10:51:06.391713+0000","last_fullsized":"2024-05-09T10:51:06.391713+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.410669+0000","last_change":"2024-05-09T10:51:06.401479+0000","last_active":"2024-05-09T10:51:06.410669+0000","last_peered":"2024-05-09T10:51:06.410669+0000","last_clean":"2024-05-09T10:51:06.410669+0000","last_became_active":"2024-05-09T10:51:06.401079+0000","last_became_peered":"2024-05-09T10:51:06.401079+0000","last_unstale":"2024-05-09T10:51:06.410669+0000","last_undegraded":"2024-05-09T10:51:06.410669+0000","last_fullsized":"2024-05-09T10:51:06.410669+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:56.016304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395508+0000","last_change":"2024-05-09T10:51:06.395508+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395508+0000","last_undegraded":"2024-05-09T10:51:06.395508+0000","last_fullsized":"2024-05-09T10:51:06.395508+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408534+0000","last_change":"2024-05-09T10:51:06.403964+0000","last_active":"2024-05-09T10:51:06.408534+0000","last_peered":"2024-05-09T10:51:06.408534+0000","last_clean":"2024-05-09T10:51:06.408534+0000","last_became_active":"2024-05-09T10:51:06.403691+0000","last_became_peered":"2024-05-09T10:51:06.403691+0000","last_unstale":"2024-05-09T10:51:06.408534+0000","last_undegraded":"2024-05-09T10:51:06.408534+0000","last_fullsized":"2024-05-09T10:51:06.408534+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:26.083086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.387999+0000","last_change":"2024-05-09T10:51:06.387999+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.387999+0000","last_undegraded":"2024-05-09T10:51:06.387999+0000","last_fullsized":"2024-05-09T10:51:06.387999+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408477+0000","last_change":"2024-05-09T10:51:06.403128+0000","last_active":"2024-05-09T10:51:06.408477+0000","last_peered":"2024-05-09T10:51:06.408477+0000","last_clean":"2024-05-09T10:51:06.408477+0000","last_became_active":"2024-05-09T10:51:06.402841+0000","last_became_peered":"2024-05-09T10:51:06.402841+0000","last_unstale":"2024-05-09T10:51:06.408477+0000","last_undegraded":"2024-05-09T10:51:06.408477+0000","last_fullsized":"2024-05-09T10:51:06.408477+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:33:59.577821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.397348+0000","last_change":"2024-05-09T10:51:06.397348+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.397348+0000","last_undegraded":"2024-05-09T10:51:06.397348+0000","last_fullsized":"2024-05-09T10:51:06.397348+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.414048+0000","last_change":"2024-05-09T10:51:06.410970+0000","last_active":"2024-05-09T10:51:06.414048+0000","last_peered":"2024-05-09T10:51:06.414048+0000","last_clean":"2024-05-09T10:51:06.414048+0000","last_became_active":"2024-05-09T10:51:06.410608+0000","last_became_peered":"2024-05-09T10:51:06.410608+0000","last_unstale":"2024-05-09T10:51:06.414048+0000","last_undegraded":"2024-05-09T10:51:06.414048+0000","last_fullsized":"2024-05-09T10:51:06.414048+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:30:44.917770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.391151+0000","last_change":"2024-05-09T10:51:06.391151+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.391151+0000","last_undegraded":"2024-05-09T10:51:06.391151+0000","last_fullsized":"2024-05-09T10:51:06.391151+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408469+0000","last_change":"2024-05-09T10:51:06.403392+0000","last_active":"2024-05-09T10:51:06.408469+0000","last_peered":"2024-05-09T10:51:06.408469+0000","last_clean":"2024-05-09T10:51:06.408469+0000","last_became_active":"2024-05-09T10:51:06.403111+0000","last_became_peered":"2024-05-09T10:51:06.403111+0000","last_unstale":"2024-05-09T10:51:06.408469+0000","last_undegraded":"2024-05-09T10:51:06.408469+0000","last_fullsized":"2024-05-09T10:51:06.408469+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:56.016304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388231+0000","last_change":"2024-05-09T10:51:06.388231+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388231+0000","last_undegraded":"2024-05-09T10:51:06.388231+0000","last_fullsized":"2024-05-09T10:51:06.388231+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408273+0000","last_change":"2024-05-09T10:51:06.406795+0000","last_active":"2024-05-09T10:51:06.408273+0000","last_peered":"2024-05-09T10:51:06.408273+0000","last_clean":"2024-05-09T10:51:06.408273+0000","last_became_active":"2024-05-09T10:51:06.406516+0000","last_became_peered":"2024-05-09T10:51:06.406516+0000","last_unstale":"2024-05-09T10:51:06.408273+0000","last_undegraded":"2024-05-09T10:51:06.408273+0000","last_fullsized":"2024-05-09T10:51:06.408273+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:29.722036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393382+0000","last_change":"2024-05-09T10:51:06.393382+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393382+0000","last_undegraded":"2024-05-09T10:51:06.393382+0000","last_fullsized":"2024-05-09T10:51:06.393382+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.401982+0000","last_change":"2024-05-09T10:51:06.399077+0000","last_active":"2024-05-09T10:51:06.401982+0000","last_peered":"2024-05-09T10:51:06.401982+0000","last_clean":"2024-05-09T10:51:06.401982+0000","last_became_active":"2024-05-09T10:51:06.398760+0000","last_became_peered":"2024-05-09T10:51:06.398760+0000","last_unstale":"2024-05-09T10:51:06.401982+0000","last_undegraded":"2024-05-09T10:51:06.401982+0000","last_fullsized":"2024-05-09T10:51:06.401982+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:51.058627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.394715+0000","last_change":"2024-05-09T10:51:06.394715+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.394715+0000","last_undegraded":"2024-05-09T10:51:06.394715+0000","last_fullsized":"2024-05-09T10:51:06.394715+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408196+0000","last_change":"2024-05-09T10:51:06.402648+0000","last_active":"2024-05-09T10:51:06.408196+0000","last_peered":"2024-05-09T10:51:06.408196+0000","last_clean":"2024-05-09T10:51:06.408196+0000","last_became_active":"2024-05-09T10:51:06.402315+0000","last_became_peered":"2024-05-09T10:51:06.402315+0000","last_unstale":"2024-05-09T10:51:06.408196+0000","last_undegraded":"2024-05-09T10:51:06.408196+0000","last_fullsized":"2024-05-09T10:51:06.408196+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:29.722036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.389652+0000","last_change":"2024-05-09T10:51:06.389652+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.389652+0000","last_undegraded":"2024-05-09T10:51:06.389652+0000","last_fullsized":"2024-05-09T10:51:06.389652+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406190+0000","last_change":"2024-05-09T10:51:06.403735+0000","last_active":"2024-05-09T10:51:06.406190+0000","last_peered":"2024-05-09T10:51:06.406190+0000","last_clean":"2024-05-09T10:51:06.406190+0000","last_became_active":"2024-05-09T10:51:06.403469+0000","last_became_peered":"2024-05-09T10:51:06.403469+0000","last_unstale":"2024-05-09T10:51:06.406190+0000","last_undegraded":"2024-05-09T10:51:06.406190+0000","last_fullsized":"2024-05-09T10:51:06.406190+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:03.704175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393445+0000","last_change":"2024-05-09T10:51:06.393445+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393445+0000","last_undegraded":"2024-05-09T10:51:06.393445+0000","last_fullsized":"2024-05-09T10:51:06.393445+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.409330+0000","last_change":"2024-05-09T10:51:06.406522+0000","last_active":"2024-05-09T10:51:06.409330+0000","last_peered":"2024-05-09T10:51:06.409330+0000","last_clean":"2024-05-09T10:51:06.409330+0000","last_became_active":"2024-05-09T10:51:06.406242+0000","last_became_peered":"2024-05-09T10:51:06.406242+0000","last_unstale":"2024-05-09T10:51:06.409330+0000","last_undegraded":"2024-05-09T10:51:06.409330+0000","last_fullsized":"2024-05-09T10:51:06.409330+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:21.592545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395214+0000","last_change":"2024-05-09T10:51:06.395214+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395214+0000","last_undegraded":"2024-05-09T10:51:06.395214+0000","last_fullsized":"2024-05-09T10:51:06.395214+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406193+0000","last_change":"2024-05-09T10:51:06.402262+0000","last_active":"2024-05-09T10:51:06.406193+0000","last_peered":"2024-05-09T10:51:06.406193+0000","last_clean":"2024-05-09T10:51:06.406193+0000","last_became_active":"2024-05-09T10:51:06.401936+0000","last_became_peered":"2024-05-09T10:51:06.401936+0000","last_unstale":"2024-05-09T10:51:06.406193+0000","last_undegraded":"2024-05-09T10:51:06.406193+0000","last_fullsized":"2024-05-09T10:51:06.406193+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:05:03.035522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.397979+0000","last_change":"2024-05-09T10:51:06.397979+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.397979+0000","last_undegraded":"2024-05-09T10:51:06.397979+0000","last_fullsized":"2024-05-09T10:51:06.397979+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406601+0000","last_change":"2024-05-09T10:51:06.405127+0000","last_active":"2024-05-09T10:51:06.406601+0000","last_peered":"2024-05-09T10:51:06.406601+0000","last_clean":"2024-05-09T10:51:06.406601+0000","last_became_active":"2024-05-09T10:51:06.404682+0000","last_became_peered":"2024-05-09T10:51:06.404682+0000","last_unstale":"2024-05-09T10:51:06.406601+0000","last_undegraded":"2024-05-09T10:51:06.406601+0000","last_fullsized":"2024-05-09T10:51:06.406601+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:21.393250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392434+0000","last_change":"2024-05-09T10:51:06.392434+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392434+0000","last_undegraded":"2024-05-09T10:51:06.392434+0000","last_fullsized":"2024-05-09T10:51:06.392434+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.402616+0000","last_change":"2024-05-09T10:51:06.401431+0000","last_active":"2024-05-09T10:51:06.402616+0000","last_peered":"2024-05-09T10:51:06.402616+0000","last_clean":"2024-05-09T10:51:06.402616+0000","last_became_active":"2024-05-09T10:51:06.401097+0000","last_became_peered":"2024-05-09T10:51:06.401097+0000","last_unstale":"2024-05-09T10:51:06.402616+0000","last_undegraded":"2024-05-09T10:51:06.402616+0000","last_fullsized":"2024-05-09T10:51:06.402616+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:05:03.035522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.398088+0000","last_change":"2024-05-09T10:51:06.398088+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.398088+0000","last_undegraded":"2024-05-09T10:51:06.398088+0000","last_fullsized":"2024-05-09T10:51:06.398088+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406321+0000","last_change":"2024-05-09T10:51:06.403074+0000","last_active":"2024-05-09T10:51:06.406321+0000","last_peered":"2024-05-09T10:51:06.406321+0000","last_clean":"2024-05-09T10:51:06.406321+0000","last_became_active":"2024-05-09T10:51:06.402721+0000","last_became_peered":"2024-05-09T10:51:06.402721+0000","last_unstale":"2024-05-09T10:51:06.406321+0000","last_undegraded":"2024-05-09T10:51:06.406321+0000","last_fullsized":"2024-05-09T10:51:06.406321+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:03.704175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388885+0000","last_change":"2024-05-09T10:51:06.388885+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388885+0000","last_undegraded":"2024-05-09T10:51:06.388885+0000","last_fullsized":"2024-05-09T10:51:06.388885+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.409346+0000","last_change":"2024-05-09T10:51:06.407069+0000","last_active":"2024-05-09T10:51:06.409346+0000","last_peered":"2024-05-09T10:51:06.409346+0000","last_clean":"2024-05-09T10:51:06.409346+0000","last_became_active":"2024-05-09T10:51:06.406780+0000","last_became_peered":"2024-05-09T10:51:06.406780+0000","last_unstale":"2024-05-09T10:51:06.409346+0000","last_undegraded":"2024-05-09T10:51:06.409346+0000","last_fullsized":"2024-05-09T10:51:06.409346+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:01.273826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392827+0000","last_change":"2024-05-09T10:51:06.392827+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392827+0000","last_undegraded":"2024-05-09T10:51:06.392827+0000","last_fullsized":"2024-05-09T10:51:06.392827+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.411703+0000","last_change":"2024-05-09T10:51:06.411247+0000","last_active":"2024-05-09T10:51:06.411703+0000","last_peered":"2024-05-09T10:51:06.411703+0000","last_clean":"2024-05-09T10:51:06.411703+0000","last_became_active":"2024-05-09T10:51:06.411001+0000","last_became_peered":"2024-05-09T10:51:06.411001+0000","last_unstale":"2024-05-09T10:51:06.411703+0000","last_undegraded":"2024-05-09T10:51:06.411703+0000","last_fullsized":"2024-05-09T10:51:06.411703+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:35.165729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388204+0000","last_change":"2024-05-09T10:51:06.388204+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388204+0000","last_undegraded":"2024-05-09T10:51:06.388204+0000","last_fullsized":"2024-05-09T10:51:06.388204+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408650+0000","last_change":"2024-05-09T10:51:06.405035+0000","last_active":"2024-05-09T10:51:06.408650+0000","last_peered":"2024-05-09T10:51:06.408650+0000","last_clean":"2024-05-09T10:51:06.408650+0000","last_became_active":"2024-05-09T10:51:06.404712+0000","last_became_peered":"2024-05-09T10:51:06.404712+0000","last_unstale":"2024-05-09T10:51:06.408650+0000","last_undegraded":"2024-05-09T10:51:06.408650+0000","last_fullsized":"2024-05-09T10:51:06.408650+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:47:47.684710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388110+0000","last_change":"2024-05-09T10:51:06.388110+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388110+0000","last_undegraded":"2024-05-09T10:51:06.388110+0000","last_fullsized":"2024-05-09T10:51:06.388110+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.401280+0000","last_change":"2024-05-09T10:51:06.396633+0000","last_active":"2024-05-09T10:51:06.401280+0000","last_peered":"2024-05-09T10:51:06.401280+0000","last_clean":"2024-05-09T10:51:06.401280+0000","last_became_active":"2024-05-09T10:51:06.396277+0000","last_became_peered":"2024-05-09T10:51:06.396277+0000","last_unstale":"2024-05-09T10:51:06.401280+0000","last_undegraded":"2024-05-09T10:51:06.401280+0000","last_fullsized":"2024-05-09T10:51:06.401280+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:29.685531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388416+0000","last_change":"2024-05-09T10:51:06.388416+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388416+0000","last_undegraded":"2024-05-09T10:51:06.388416+0000","last_fullsized":"2024-05-09T10:51:06.388416+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408562+0000","last_change":"2024-05-09T10:51:06.405148+0000","last_active":"2024-05-09T10:51:06.408562+0000","last_peered":"2024-05-09T10:51:06.408562+0000","last_clean":"2024-05-09T10:51:06.408562+0000","last_became_active":"2024-05-09T10:51:06.403901+0000","last_became_peered":"2024-05-09T10:51:06.403901+0000","last_unstale":"2024-05-09T10:51:06.408562+0000","last_undegraded":"2024-05-09T10:51:06.408562+0000","last_fullsized":"2024-05-09T10:51:06.408562+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:29.685531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388109+0000","last_change":"2024-05-09T10:51:06.388109+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388109+0000","last_undegraded":"2024-05-09T10:51:06.388109+0000","last_fullsized":"2024-05-09T10:51:06.388109+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408642+0000","last_change":"2024-05-09T10:51:06.404951+0000","last_active":"2024-05-09T10:51:06.408642+0000","last_peered":"2024-05-09T10:51:06.408642+0000","last_clean":"2024-05-09T10:51:06.408642+0000","last_became_active":"2024-05-09T10:51:06.404574+0000","last_became_peered":"2024-05-09T10:51:06.404574+0000","last_unstale":"2024-05-09T10:51:06.408642+0000","last_undegraded":"2024-05-09T10:51:06.408642+0000","last_fullsized":"2024-05-09T10:51:06.408642+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:31:55.748009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.396433+0000","last_change":"2024-05-09T10:51:06.396433+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.396433+0000","last_undegraded":"2024-05-09T10:51:06.396433+0000","last_fullsized":"2024-05-09T10:51:06.396433+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408223+0000","last_change":"2024-05-09T10:51:06.404809+0000","last_active":"2024-05-09T10:51:06.408223+0000","last_peered":"2024-05-09T10:51:06.408223+0000","last_clean":"2024-05-09T10:51:06.408223+0000","last_became_active":"2024-05-09T10:51:06.404529+0000","last_became_peered":"2024-05-09T10:51:06.404529+0000","last_unstale":"2024-05-09T10:51:06.408223+0000","last_undegraded":"2024-05-09T10:51:06.408223+0000","last_fullsized":"2024-05-09T10:51:06.408223+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:30:44.917770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.388071+0000","last_change":"2024-05-09T10:51:06.388071+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.388071+0000","last_undegraded":"2024-05-09T10:51:06.388071+0000","last_fullsized":"2024-05-09T10:51:06.388071+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.402311+0000","last_change":"2024-05-09T10:51:06.398765+0000","last_active":"2024-05-09T10:51:06.402311+0000","last_peered":"2024-05-09T10:51:06.402311+0000","last_clean":"2024-05-09T10:51:06.402311+0000","last_became_active":"2024-05-09T10:51:06.397480+0000","last_became_peered":"2024-05-09T10:51:06.397480+0000","last_unstale":"2024-05-09T10:51:06.402311+0000","last_undegraded":"2024-05-09T10:51:06.402311+0000","last_fullsized":"2024-05-09T10:51:06.402311+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:29.685531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.387775+0000","last_change":"2024-05-09T10:51:06.387775+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.387775+0000","last_undegraded":"2024-05-09T10:51:06.387775+0000","last_fullsized":"2024-05-09T10:51:06.387775+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.402004+0000","last_change":"2024-05-09T10:51:06.399613+0000","last_active":"2024-05-09T10:51:06.402004+0000","last_peered":"2024-05-09T10:51:06.402004+0000","last_clean":"2024-05-09T10:51:06.402004+0000","last_became_active":"2024-05-09T10:51:06.399333+0000","last_became_peered":"2024-05-09T10:51:06.399333+0000","last_unstale":"2024-05-09T10:51:06.402004+0000","last_undegraded":"2024-05-09T10:51:06.402004+0000","last_fullsized":"2024-05-09T10:51:06.402004+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:05:03.035522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.389289+0000","last_change":"2024-05-09T10:51:06.389289+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.389289+0000","last_undegraded":"2024-05-09T10:51:06.389289+0000","last_fullsized":"2024-05-09T10:51:06.389289+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.400013+0000","last_change":"2024-05-09T10:51:06.396469+0000","last_active":"2024-05-09T10:51:06.400013+0000","last_peered":"2024-05-09T10:51:06.400013+0000","last_clean":"2024-05-09T10:51:06.400013+0000","last_became_active":"2024-05-09T10:51:06.396180+0000","last_became_peered":"2024-05-09T10:51:06.396180+0000","last_unstale":"2024-05-09T10:51:06.400013+0000","last_undegraded":"2024-05-09T10:51:06.400013+0000","last_fullsized":"2024-05-09T10:51:06.400013+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:21.607813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.392198+0000","last_change":"2024-05-09T10:51:06.392198+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.392198+0000","last_undegraded":"2024-05-09T10:51:06.392198+0000","last_fullsized":"2024-05-09T10:51:06.392198+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.395406+0000","last_change":"2024-05-09T10:51:06.394559+0000","last_active":"2024-05-09T10:51:06.395406+0000","last_peered":"2024-05-09T10:51:06.395406+0000","last_clean":"2024-05-09T10:51:06.395406+0000","last_became_active":"2024-05-09T10:51:06.394288+0000","last_became_peered":"2024-05-09T10:51:06.394288+0000","last_unstale":"2024-05-09T10:51:06.395406+0000","last_undegraded":"2024-05-09T10:51:06.395406+0000","last_fullsized":"2024-05-09T10:51:06.395406+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:21.607813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.389645+0000","last_change":"2024-05-09T10:51:06.389645+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.389645+0000","last_undegraded":"2024-05-09T10:51:06.389645+0000","last_fullsized":"2024-05-09T10:51:06.389645+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.404581+0000","last_change":"2024-05-09T10:51:06.400034+0000","last_active":"2024-05-09T10:51:06.404581+0000","last_peered":"2024-05-09T10:51:06.404581+0000","last_clean":"2024-05-09T10:51:06.404581+0000","last_became_active":"2024-05-09T10:51:06.397993+0000","last_became_peered":"2024-05-09T10:51:06.397993+0000","last_unstale":"2024-05-09T10:51:06.404581+0000","last_undegraded":"2024-05-09T10:51:06.404581+0000","last_fullsized":"2024-05-09T10:51:06.404581+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:51.058627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.393340+0000","last_change":"2024-05-09T10:51:06.393340+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.393340+0000","last_undegraded":"2024-05-09T10:51:06.393340+0000","last_fullsized":"2024-05-09T10:51:06.393340+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.406276+0000","last_change":"2024-05-09T10:51:06.400625+0000","last_active":"2024-05-09T10:51:06.406276+0000","last_peered":"2024-05-09T10:51:06.406276+0000","last_clean":"2024-05-09T10:51:06.406276+0000","last_became_active":"2024-05-09T10:51:06.400119+0000","last_became_peered":"2024-05-09T10:51:06.400119+0000","last_unstale":"2024-05-09T10:51:06.406276+0000","last_undegraded":"2024-05-09T10:51:06.406276+0000","last_fullsized":"2024-05-09T10:51:06.406276+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:34:06.133956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395771+0000","last_change":"2024-05-09T10:51:06.395771+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395771+0000","last_undegraded":"2024-05-09T10:51:06.395771+0000","last_fullsized":"2024-05-09T10:51:06.395771+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408282+0000","last_change":"2024-05-09T10:51:06.406673+0000","last_active":"2024-05-09T10:51:06.408282+0000","last_peered":"2024-05-09T10:51:06.408282+0000","last_clean":"2024-05-09T10:51:06.408282+0000","last_became_active":"2024-05-09T10:51:06.406391+0000","last_became_peered":"2024-05-09T10:51:06.406391+0000","last_unstale":"2024-05-09T10:51:06.408282+0000","last_undegraded":"2024-05-09T10:51:06.408282+0000","last_fullsized":"2024-05-09T10:51:06.408282+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:45:51.854599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-05-09T10:51:06.395869+0000","last_change":"2024-05-09T10:51:06.395869+0000","last_active":"2024-05-09T10:51:06.382079+0000","last_peered":"2024-05-09T10:51:06.382079+0000","last_clean":"2024-05-09T10:51:06.382079+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-09T10:51:06.395869+0000","last_undegraded":"2024-05-09T10:51:06.395869+0000","last_fullsized":"2024-05-09T10:51:06.395869+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.408907+0000","last_change":"2024-05-09T10:51:06.407600+0000","last_active":"2024-05-09T10:51:06.408907+0000","last_peered":"2024-05-09T10:51:06.408907+0000","last_clean":"2024-05-09T10:51:06.408907+0000","last_became_active":"2024-05-09T10:51:06.407354+0000","last_became_peered":"2024-05-09T10:51:06.407354+0000","last_unstale":"2024-05-09T10:51:06.408907+0000","last_undegraded":"2024-05-09T10:51:06.408907+0000","last_fullsized":"2024-05-09T10:51:06.408907+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:51.058627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":186,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-09T10:51:06.401241+0000","last_change":"2024-05-09T10:50:08.410364+0000","last_active":"2024-05-09T10:51:06.401241+0000","last_peered":"2024-05-09T10:51:06.401241+0000","last_clean":"2024-05-09T10:51:06.401241+0000","last_became_active":"2024-05-09T10:50:08.409457+0000","last_became_peered":"2024-05-09T10:50:08.409457+0000","last_unstale":"2024-05-09T10:51:06.401241+0000","last_undegraded":"2024-05-09T10:51:06.401241+0000","last_fullsized":"2024-05-09T10:51:06.401241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_clean_scrub_stamp":"2024-05-09T10:50:07.396510+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:03.566698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":917504,"data_stored":711419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":917504,"data_stored":711419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T10:51:14.737 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T10:51:14.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.854+0000 7f83f80fc640 1 Processor -- start 2024-05-09T10:51:14.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.855+0000 7f83f80fc640 1 -- start start 2024-05-09T10:51:14.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f80fc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f83f015f4e0 0x7f83f00ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:14.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f80fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83f00b09e0 0x7f83f0165c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:14.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f80fc640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f00aeda0 con 0x7f83f00afe40 2024-05-09T10:51:14.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f80fc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f00af080 con 0x7f83f015f4e0 2024-05-09T10:51:14.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f80fc640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f00af360 con 0x7f83f00b09e0 2024-05-09T10:51:14.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f5697640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f83f00afe40 0x7f83f00b02d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36828/0 (socket says 172.21.15.98:36828) 2024-05-09T10:51:14.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f5697640 1 -- 172.21.15.98:0/1605189092 learned_addr learned my addr 172.21.15.98:0/1605189092 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:14.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f6699640 1 --2- 172.21.15.98:0/1605189092 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83f00b09e0 0x7f83f0165c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:14.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f5e98640 1 --2- 172.21.15.98:0/1605189092 >> v2:172.21.15.105:3300/0 conn(0x7f83f015f4e0 0x7f83f00ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:14.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f4e96640 1 -- 172.21.15.98:0/1605189092 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4166959946 0 0) 0x7f83f00aeda0 con 0x7f83f00afe40 2024-05-09T10:51:14.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f4e96640 1 -- 172.21.15.98:0/1605189092 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83dc003520 con 0x7f83f00afe40 2024-05-09T10:51:14.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f4e96640 1 -- 172.21.15.98:0/1605189092 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4188552764 0 0) 0x7f83dc003520 con 0x7f83f00afe40 2024-05-09T10:51:14.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f4e96640 1 -- 172.21.15.98:0/1605189092 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83f00b09e0 msgr2=0x7f83f0165c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:14.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f4e96640 1 --2- 172.21.15.98:0/1605189092 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83f00b09e0 0x7f83f0165c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:14.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f4e96640 1 -- 172.21.15.98:0/1605189092 >> v2:172.21.15.105:3300/0 conn(0x7f83f015f4e0 msgr2=0x7f83f00ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:14.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f4e96640 1 --2- 172.21.15.98:0/1605189092 >> v2:172.21.15.105:3300/0 conn(0x7f83f015f4e0 0x7f83f00ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:14.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f4e96640 1 -- 172.21.15.98:0/1605189092 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f00af640 con 0x7f83f00afe40 2024-05-09T10:51:14.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f6699640 1 --2- 172.21.15.98:0/1605189092 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83f00b09e0 0x7f83f0165c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:51:14.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f4e96640 1 -- 172.21.15.98:0/1605189092 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f83e4001670 con 0x7f83f00afe40 2024-05-09T10:51:14.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f4e96640 1 -- 172.21.15.98:0/1605189092 >> v1:172.21.15.98:6789/0 conn(0x7f83f00afe40 legacy=0x7f83f00b02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:14.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f4e96640 1 --2- 172.21.15.98:0/1605189092 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83f00b09e0 0x7f83dc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:14.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f4e96640 1 --2- 172.21.15.98:0/1605189092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83dc003ef0 0x7f83dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:14.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f4e96640 1 --2- 172.21.15.98:0/1605189092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83dc004d40 0x7f83dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:14.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f4e96640 1 -- 172.21.15.98:0/1605189092 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f00af080 con 0x7f83dc004d40 2024-05-09T10:51:14.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f4e96640 1 -- 172.21.15.98:0/1605189092 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f00af360 con 0x7f83dc003ef0 2024-05-09T10:51:14.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.856+0000 7f83f4e96640 1 -- 172.21.15.98:0/1605189092 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc00b9c0 con 0x7f83f00b09e0 2024-05-09T10:51:14.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.857+0000 7f83f6699640 1 --2- 172.21.15.98:0/1605189092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83dc004d40 0x7f83dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:14.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.857+0000 7f83f5e98640 1 --2- 172.21.15.98:0/1605189092 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83f00b09e0 0x7f83dc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:14.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.857+0000 7f83f5697640 1 --2- 172.21.15.98:0/1605189092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83dc003ef0 0x7f83dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:14.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.857+0000 7f83f6699640 1 -- 172.21.15.98:0/1605189092 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83f00b09e0 msgr2=0x7f83dc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:14.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.857+0000 7f83f6699640 1 --2- 172.21.15.98:0/1605189092 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83f00b09e0 0x7f83dc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:14.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.857+0000 7f83f6699640 1 -- 172.21.15.98:0/1605189092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83dc003ef0 msgr2=0x7f83dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:14.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.857+0000 7f83f6699640 1 --2- 172.21.15.98:0/1605189092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83dc003ef0 0x7f83dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:14.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.857+0000 7f83f6699640 1 -- 172.21.15.98:0/1605189092 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83dc003520 con 0x7f83dc004d40 2024-05-09T10:51:14.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.858+0000 7f83f6699640 1 --2- 172.21.15.98:0/1605189092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83dc004d40 0x7f83dc0051c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f83e800e350 tx=0x7f83dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02fed4114462d6a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:14.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.858+0000 7f83f4e96640 1 -- 172.21.15.98:0/1605189092 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e8033070 con 0x7f83dc004d40 2024-05-09T10:51:14.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.858+0000 7f83f4e96640 1 -- 172.21.15.98:0/1605189092 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e802f660 con 0x7f83dc004d40 2024-05-09T10:51:14.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.858+0000 7f83f4e96640 1 -- 172.21.15.98:0/1605189092 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e8009de0 con 0x7f83dc004d40 2024-05-09T10:51:14.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.858+0000 7f83f80fc640 1 -- 172.21.15.98:0/1605189092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83dc004d40 msgr2=0x7f83dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:14.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.858+0000 7f83f80fc640 1 --2- 172.21.15.98:0/1605189092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83dc004d40 0x7f83dc0051c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f83e800e350 tx=0x7f83dc005830 comp rx=0 tx=0).stop 2024-05-09T10:51:14.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.858+0000 7f83f6699640 1 -- 172.21.15.98:0/1605189092 reap_dead start 2024-05-09T10:51:14.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.858+0000 7f83f80fc640 1 -- 172.21.15.98:0/1605189092 shutdown_connections 2024-05-09T10:51:14.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.858+0000 7f83f80fc640 1 -- 172.21.15.98:0/1605189092 >> 172.21.15.98:0/1605189092 conn(0x7f83f00b34e0 msgr2=0x7f83f00b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:14.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.858+0000 7f83f80fc640 1 -- 172.21.15.98:0/1605189092 shutdown_connections 2024-05-09T10:51:14.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.858+0000 7f83f80fc640 1 -- 172.21.15.98:0/1605189092 wait complete. 2024-05-09T10:51:14.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.860+0000 7f83f80fc640 1 Processor -- start 2024-05-09T10:51:14.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.860+0000 7f83f80fc640 1 -- start start 2024-05-09T10:51:14.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.860+0000 7f83f80fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83dc004d40 0x7f83f01591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:14.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.860+0000 7f83f80fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83f0158c60 0x7f83f014fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:14.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.860+0000 7f83f80fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83f015d030 0x7f83f015d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:14.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.860+0000 7f83f80fc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f015f8c0 con 0x7f83f0158c60 2024-05-09T10:51:14.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.860+0000 7f83f80fc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f00cbcf0 con 0x7f83f015d030 2024-05-09T10:51:14.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.860+0000 7f83f80fc640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f00b0e30 con 0x7f83dc004d40 2024-05-09T10:51:14.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.860+0000 7f83f5697640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83f0158c60 0x7f83f014fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:14.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.861+0000 7f83f5697640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83f0158c60 0x7f83f014fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57472/0 (socket says 172.21.15.98:57472) 2024-05-09T10:51:14.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.861+0000 7f83f5697640 1 -- 172.21.15.98:0/3769640350 learned_addr learned my addr 172.21.15.98:0/3769640350 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:14.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.861+0000 7f83f5e98640 1 --2- 172.21.15.98:0/3769640350 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83dc004d40 0x7f83f01591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:14.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.861+0000 7f83f6699640 1 --2- 172.21.15.98:0/3769640350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83f015d030 0x7f83f015d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:14.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.861+0000 7f83f5697640 1 -- 172.21.15.98:0/3769640350 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83dc004d40 msgr2=0x7f83f01591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:14.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.861+0000 7f83f5697640 1 --2- 172.21.15.98:0/3769640350 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83dc004d40 0x7f83f01591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:14.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.861+0000 7f83f5697640 1 -- 172.21.15.98:0/3769640350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83f015d030 msgr2=0x7f83f015d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:14.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.861+0000 7f83f5697640 1 --2- 172.21.15.98:0/3769640350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83f015d030 0x7f83f015d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:14.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.861+0000 7f83f5697640 1 -- 172.21.15.98:0/3769640350 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f0150400 con 0x7f83f0158c60 2024-05-09T10:51:14.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.861+0000 7f83f5697640 1 --2- 172.21.15.98:0/3769640350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83f0158c60 0x7f83f014fec0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f83e40073a0 tx=0x7f83e4007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:14.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.861+0000 7f83e27fc640 1 -- 172.21.15.98:0/3769640350 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e401a070 con 0x7f83f0158c60 2024-05-09T10:51:14.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.861+0000 7f83e27fc640 1 -- 172.21.15.98:0/3769640350 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e400f9c0 con 0x7f83f0158c60 2024-05-09T10:51:14.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.861+0000 7f83e27fc640 1 -- 172.21.15.98:0/3769640350 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e400fd70 con 0x7f83f0158c60 2024-05-09T10:51:14.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.861+0000 7f83f80fc640 1 -- 172.21.15.98:0/3769640350 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f00b5c40 con 0x7f83f0158c60 2024-05-09T10:51:14.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.861+0000 7f83f80fc640 1 -- 172.21.15.98:0/3769640350 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f00b6200 con 0x7f83f0158c60 2024-05-09T10:51:14.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.862+0000 7f83f80fc640 1 -- 172.21.15.98:0/3769640350 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f015b4e0 con 0x7f83f0158c60 2024-05-09T10:51:14.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.863+0000 7f83e27fc640 1 -- 172.21.15.98:0/3769640350 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f83e401d360 con 0x7f83f0158c60 2024-05-09T10:51:14.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.863+0000 7f83e27fc640 1 --2- 172.21.15.98:0/3769640350 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f83d406b500 0x7f83d406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:14.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.863+0000 7f83e27fc640 1 -- 172.21.15.98:0/3769640350 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83e4089da0 con 0x7f83f0158c60 2024-05-09T10:51:14.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.863+0000 7f83f5e98640 1 --2- 172.21.15.98:0/3769640350 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f83d406b500 0x7f83d406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:14.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.863+0000 7f83f5e98640 1 --2- 172.21.15.98:0/3769640350 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f83d406b500 0x7f83d406d9c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009b80 tx=0x7f83ec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:14.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:14.867+0000 7f83e27fc640 1 -- 172.21.15.98:0/3769640350 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f83f0150400 con 0x7f83f0158c60 2024-05-09T10:51:15.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.035+0000 7f83f80fc640 1 -- 172.21.15.98:0/3769640350 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f83f00b6850 con 0x7f83d406b500 2024-05-09T10:51:15.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.041+0000 7f83e27fc640 1 -- 172.21.15.98:0/3769640350 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251951 (secure 0 0 0) 0x7f83f00b6850 con 0x7f83d406b500 2024-05-09T10:51:15.037 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:15.039 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-05-09T10:51:15.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.044+0000 7f83f80fc640 1 -- 172.21.15.98:0/3769640350 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f83d406b500 msgr2=0x7f83d406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:15.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.044+0000 7f83f80fc640 1 --2- 172.21.15.98:0/3769640350 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f83d406b500 0x7f83d406d9c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009b80 tx=0x7f83ec008040 comp rx=0 tx=0).stop 2024-05-09T10:51:15.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.044+0000 7f83f80fc640 1 -- 172.21.15.98:0/3769640350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83f0158c60 msgr2=0x7f83f014fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:15.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.044+0000 7f83f80fc640 1 --2- 172.21.15.98:0/3769640350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83f0158c60 0x7f83f014fec0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f83e40073a0 tx=0x7f83e4007860 comp rx=0 tx=0).stop 2024-05-09T10:51:15.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.044+0000 7f83f80fc640 1 -- 172.21.15.98:0/3769640350 shutdown_connections 2024-05-09T10:51:15.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.044+0000 7f83f80fc640 1 --2- 172.21.15.98:0/3769640350 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f83d406b500 0x7f83d406d9c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.044+0000 7f83f80fc640 1 --2- 172.21.15.98:0/3769640350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83f015d030 0x7f83f015d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.044+0000 7f83f80fc640 1 --2- 172.21.15.98:0/3769640350 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f83f0158c60 0x7f83f014fec0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.044+0000 7f83f80fc640 1 --2- 172.21.15.98:0/3769640350 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f83dc004d40 0x7f83f01591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.044+0000 7f83f80fc640 1 -- 172.21.15.98:0/3769640350 >> 172.21.15.98:0/3769640350 conn(0x7f83f00b34e0 msgr2=0x7f83f015add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:15.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.044+0000 7f83f80fc640 1 -- 172.21.15.98:0/3769640350 shutdown_connections 2024-05-09T10:51:15.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.044+0000 7f83f80fc640 1 -- 172.21.15.98:0/3769640350 wait complete. 2024-05-09T10:51:15.049 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-05-09T10:51:13.375591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220108,"kb_used_data":3944,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754612,"statfs":{"total":773094113280,"available":772868722688,"internally_reserved":0,"allocated":4038656,"data_stored":2258564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.990537"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.032436+0000","last_change":"2024-05-09T10:51:07.419735+0000","last_active":"2024-05-09T10:51:09.032436+0000","last_peered":"2024-05-09T10:51:09.032436+0000","last_clean":"2024-05-09T10:51:09.032436+0000","last_became_active":"2024-05-09T10:51:07.418990+0000","last_became_peered":"2024-05-09T10:51:07.418990+0000","last_unstale":"2024-05-09T10:51:09.032436+0000","last_undegraded":"2024-05-09T10:51:09.032436+0000","last_fullsized":"2024-05-09T10:51:09.032436+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:56.646594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.031876+0000","last_change":"2024-05-09T10:51:07.423987+0000","last_active":"2024-05-09T10:51:09.031876+0000","last_peered":"2024-05-09T10:51:09.031876+0000","last_clean":"2024-05-09T10:51:09.031876+0000","last_became_active":"2024-05-09T10:51:07.423622+0000","last_became_peered":"2024-05-09T10:51:07.423622+0000","last_unstale":"2024-05-09T10:51:09.031876+0000","last_undegraded":"2024-05-09T10:51:09.031876+0000","last_fullsized":"2024-05-09T10:51:09.031876+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:01:37.963319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.034535+0000","last_change":"2024-05-09T10:51:07.414484+0000","last_active":"2024-05-09T10:51:09.034535+0000","last_peered":"2024-05-09T10:51:09.034535+0000","last_clean":"2024-05-09T10:51:09.034535+0000","last_became_active":"2024-05-09T10:51:07.414202+0000","last_became_peered":"2024-05-09T10:51:07.414202+0000","last_unstale":"2024-05-09T10:51:09.034535+0000","last_undegraded":"2024-05-09T10:51:09.034535+0000","last_fullsized":"2024-05-09T10:51:09.034535+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:48.715739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.413261+0000","last_change":"2024-05-09T10:51:07.420204+0000","last_active":"2024-05-09T10:51:08.413261+0000","last_peered":"2024-05-09T10:51:08.413261+0000","last_clean":"2024-05-09T10:51:08.413261+0000","last_became_active":"2024-05-09T10:51:07.419906+0000","last_became_peered":"2024-05-09T10:51:07.419906+0000","last_unstale":"2024-05-09T10:51:08.413261+0000","last_undegraded":"2024-05-09T10:51:08.413261+0000","last_fullsized":"2024-05-09T10:51:08.413261+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:15:52.271016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.412768+0000","last_change":"2024-05-09T10:51:07.421257+0000","last_active":"2024-05-09T10:51:08.412768+0000","last_peered":"2024-05-09T10:51:08.412768+0000","last_clean":"2024-05-09T10:51:08.412768+0000","last_became_active":"2024-05-09T10:51:07.420995+0000","last_became_peered":"2024-05-09T10:51:07.420995+0000","last_unstale":"2024-05-09T10:51:08.412768+0000","last_undegraded":"2024-05-09T10:51:08.412768+0000","last_fullsized":"2024-05-09T10:51:08.412768+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:52.062626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.031628+0000","last_change":"2024-05-09T10:51:07.412543+0000","last_active":"2024-05-09T10:51:09.031628+0000","last_peered":"2024-05-09T10:51:09.031628+0000","last_clean":"2024-05-09T10:51:09.031628+0000","last_became_active":"2024-05-09T10:51:07.412327+0000","last_became_peered":"2024-05-09T10:51:07.412327+0000","last_unstale":"2024-05-09T10:51:09.031628+0000","last_undegraded":"2024-05-09T10:51:09.031628+0000","last_fullsized":"2024-05-09T10:51:09.031628+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:48.715739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.033294+0000","last_change":"2024-05-09T10:51:07.414953+0000","last_active":"2024-05-09T10:51:09.033294+0000","last_peered":"2024-05-09T10:51:09.033294+0000","last_clean":"2024-05-09T10:51:09.033294+0000","last_became_active":"2024-05-09T10:51:07.414441+0000","last_became_peered":"2024-05-09T10:51:07.414441+0000","last_unstale":"2024-05-09T10:51:09.033294+0000","last_undegraded":"2024-05-09T10:51:09.033294+0000","last_fullsized":"2024-05-09T10:51:09.033294+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:13:39.637223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.032286+0000","last_change":"2024-05-09T10:51:07.411747+0000","last_active":"2024-05-09T10:51:09.032286+0000","last_peered":"2024-05-09T10:51:09.032286+0000","last_clean":"2024-05-09T10:51:09.032286+0000","last_became_active":"2024-05-09T10:51:07.411482+0000","last_became_peered":"2024-05-09T10:51:07.411482+0000","last_unstale":"2024-05-09T10:51:09.032286+0000","last_undegraded":"2024-05-09T10:51:09.032286+0000","last_fullsized":"2024-05-09T10:51:09.032286+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:41.634984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.034000+0000","last_change":"2024-05-09T10:51:07.413423+0000","last_active":"2024-05-09T10:51:09.034000+0000","last_peered":"2024-05-09T10:51:09.034000+0000","last_clean":"2024-05-09T10:51:09.034000+0000","last_became_active":"2024-05-09T10:51:07.412825+0000","last_became_peered":"2024-05-09T10:51:07.412825+0000","last_unstale":"2024-05-09T10:51:09.034000+0000","last_undegraded":"2024-05-09T10:51:09.034000+0000","last_fullsized":"2024-05-09T10:51:09.034000+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:35:51.265075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.413349+0000","last_change":"2024-05-09T10:51:07.423308+0000","last_active":"2024-05-09T10:51:08.413349+0000","last_peered":"2024-05-09T10:51:08.413349+0000","last_clean":"2024-05-09T10:51:08.413349+0000","last_became_active":"2024-05-09T10:51:07.423024+0000","last_became_peered":"2024-05-09T10:51:07.423024+0000","last_unstale":"2024-05-09T10:51:08.413349+0000","last_undegraded":"2024-05-09T10:51:08.413349+0000","last_fullsized":"2024-05-09T10:51:08.413349+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:35:51.265075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.033221+0000","last_change":"2024-05-09T10:51:07.415838+0000","last_active":"2024-05-09T10:51:09.033221+0000","last_peered":"2024-05-09T10:51:09.033221+0000","last_clean":"2024-05-09T10:51:09.033221+0000","last_became_active":"2024-05-09T10:51:07.415532+0000","last_became_peered":"2024-05-09T10:51:07.415532+0000","last_unstale":"2024-05-09T10:51:09.033221+0000","last_undegraded":"2024-05-09T10:51:09.033221+0000","last_fullsized":"2024-05-09T10:51:09.033221+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:48.715739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.034222+0000","last_change":"2024-05-09T10:51:07.423110+0000","last_active":"2024-05-09T10:51:09.034222+0000","last_peered":"2024-05-09T10:51:09.034222+0000","last_clean":"2024-05-09T10:51:09.034222+0000","last_became_active":"2024-05-09T10:51:07.422629+0000","last_became_peered":"2024-05-09T10:51:07.422629+0000","last_unstale":"2024-05-09T10:51:09.034222+0000","last_undegraded":"2024-05-09T10:51:09.034222+0000","last_fullsized":"2024-05-09T10:51:09.034222+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:35:24.776616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.412900+0000","last_change":"2024-05-09T10:51:07.414416+0000","last_active":"2024-05-09T10:51:08.412900+0000","last_peered":"2024-05-09T10:51:08.412900+0000","last_clean":"2024-05-09T10:51:08.412900+0000","last_became_active":"2024-05-09T10:51:07.414157+0000","last_became_peered":"2024-05-09T10:51:07.414157+0000","last_unstale":"2024-05-09T10:51:08.412900+0000","last_undegraded":"2024-05-09T10:51:08.412900+0000","last_fullsized":"2024-05-09T10:51:08.412900+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:05:04.039522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.031655+0000","last_change":"2024-05-09T10:51:07.420087+0000","last_active":"2024-05-09T10:51:09.031655+0000","last_peered":"2024-05-09T10:51:09.031655+0000","last_clean":"2024-05-09T10:51:09.031655+0000","last_became_active":"2024-05-09T10:51:07.419460+0000","last_became_peered":"2024-05-09T10:51:07.419460+0000","last_unstale":"2024-05-09T10:51:09.031655+0000","last_undegraded":"2024-05-09T10:51:09.031655+0000","last_fullsized":"2024-05-09T10:51:09.031655+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:52:15.329752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.033721+0000","last_change":"2024-05-09T10:51:07.414884+0000","last_active":"2024-05-09T10:51:09.033721+0000","last_peered":"2024-05-09T10:51:09.033721+0000","last_clean":"2024-05-09T10:51:09.033721+0000","last_became_active":"2024-05-09T10:51:07.414620+0000","last_became_peered":"2024-05-09T10:51:07.414620+0000","last_unstale":"2024-05-09T10:51:09.033721+0000","last_undegraded":"2024-05-09T10:51:09.033721+0000","last_fullsized":"2024-05-09T10:51:09.033721+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:58:29.190468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.032743+0000","last_change":"2024-05-09T10:51:07.421318+0000","last_active":"2024-05-09T10:51:09.032743+0000","last_peered":"2024-05-09T10:51:09.032743+0000","last_clean":"2024-05-09T10:51:09.032743+0000","last_became_active":"2024-05-09T10:51:07.420529+0000","last_became_peered":"2024-05-09T10:51:07.420529+0000","last_unstale":"2024-05-09T10:51:09.032743+0000","last_undegraded":"2024-05-09T10:51:09.032743+0000","last_fullsized":"2024-05-09T10:51:09.032743+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:37.530751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.033154+0000","last_change":"2024-05-09T10:51:07.417210+0000","last_active":"2024-05-09T10:51:09.033154+0000","last_peered":"2024-05-09T10:51:09.033154+0000","last_clean":"2024-05-09T10:51:09.033154+0000","last_became_active":"2024-05-09T10:51:07.416921+0000","last_became_peered":"2024-05-09T10:51:07.416921+0000","last_unstale":"2024-05-09T10:51:09.033154+0000","last_undegraded":"2024-05-09T10:51:09.033154+0000","last_fullsized":"2024-05-09T10:51:09.033154+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:19:22.881499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.031881+0000","last_change":"2024-05-09T10:51:07.427528+0000","last_active":"2024-05-09T10:51:09.031881+0000","last_peered":"2024-05-09T10:51:09.031881+0000","last_clean":"2024-05-09T10:51:09.031881+0000","last_became_active":"2024-05-09T10:51:07.427094+0000","last_became_peered":"2024-05-09T10:51:07.427094+0000","last_unstale":"2024-05-09T10:51:09.031881+0000","last_undegraded":"2024-05-09T10:51:09.031881+0000","last_fullsized":"2024-05-09T10:51:09.031881+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:59:21.253776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.031661+0000","last_change":"2024-05-09T10:51:07.421444+0000","last_active":"2024-05-09T10:51:09.031661+0000","last_peered":"2024-05-09T10:51:09.031661+0000","last_clean":"2024-05-09T10:51:09.031661+0000","last_became_active":"2024-05-09T10:51:07.420954+0000","last_became_peered":"2024-05-09T10:51:07.420954+0000","last_unstale":"2024-05-09T10:51:09.031661+0000","last_undegraded":"2024-05-09T10:51:09.031661+0000","last_fullsized":"2024-05-09T10:51:09.031661+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:19:22.881499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.412751+0000","last_change":"2024-05-09T10:51:07.413672+0000","last_active":"2024-05-09T10:51:08.412751+0000","last_peered":"2024-05-09T10:51:08.412751+0000","last_clean":"2024-05-09T10:51:08.412751+0000","last_became_active":"2024-05-09T10:51:07.413427+0000","last_became_peered":"2024-05-09T10:51:07.413427+0000","last_unstale":"2024-05-09T10:51:08.412751+0000","last_undegraded":"2024-05-09T10:51:08.412751+0000","last_fullsized":"2024-05-09T10:51:08.412751+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:30.689530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.034021+0000","last_change":"2024-05-09T10:51:07.412973+0000","last_active":"2024-05-09T10:51:09.034021+0000","last_peered":"2024-05-09T10:51:09.034021+0000","last_clean":"2024-05-09T10:51:09.034021+0000","last_became_active":"2024-05-09T10:51:07.412666+0000","last_became_peered":"2024-05-09T10:51:07.412666+0000","last_unstale":"2024-05-09T10:51:09.034021+0000","last_undegraded":"2024-05-09T10:51:09.034021+0000","last_fullsized":"2024-05-09T10:51:09.034021+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:13:12.976022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.033812+0000","last_change":"2024-05-09T10:51:07.418252+0000","last_active":"2024-05-09T10:51:09.033812+0000","last_peered":"2024-05-09T10:51:09.033812+0000","last_clean":"2024-05-09T10:51:09.033812+0000","last_became_active":"2024-05-09T10:51:07.417877+0000","last_became_peered":"2024-05-09T10:51:07.417877+0000","last_unstale":"2024-05-09T10:51:09.033812+0000","last_undegraded":"2024-05-09T10:51:09.033812+0000","last_fullsized":"2024-05-09T10:51:09.033812+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:15:52.271016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.411403+0000","last_change":"2024-05-09T10:51:07.410748+0000","last_active":"2024-05-09T10:51:08.411403+0000","last_peered":"2024-05-09T10:51:08.411403+0000","last_clean":"2024-05-09T10:51:08.411403+0000","last_became_active":"2024-05-09T10:51:07.410477+0000","last_became_peered":"2024-05-09T10:51:07.410477+0000","last_unstale":"2024-05-09T10:51:08.411403+0000","last_undegraded":"2024-05-09T10:51:08.411403+0000","last_fullsized":"2024-05-09T10:51:08.411403+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:52.418635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.033440+0000","last_change":"2024-05-09T10:51:07.411744+0000","last_active":"2024-05-09T10:51:09.033440+0000","last_peered":"2024-05-09T10:51:09.033440+0000","last_clean":"2024-05-09T10:51:09.033440+0000","last_became_active":"2024-05-09T10:51:07.411479+0000","last_became_peered":"2024-05-09T10:51:07.411479+0000","last_unstale":"2024-05-09T10:51:09.033440+0000","last_undegraded":"2024-05-09T10:51:09.033440+0000","last_fullsized":"2024-05-09T10:51:09.033440+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:33:40.854778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.413525+0000","last_change":"2024-05-09T10:51:07.414161+0000","last_active":"2024-05-09T10:51:08.413525+0000","last_peered":"2024-05-09T10:51:08.413525+0000","last_clean":"2024-05-09T10:51:08.413525+0000","last_became_active":"2024-05-09T10:51:07.413621+0000","last_became_peered":"2024-05-09T10:51:07.413621+0000","last_unstale":"2024-05-09T10:51:08.413525+0000","last_undegraded":"2024-05-09T10:51:08.413525+0000","last_fullsized":"2024-05-09T10:51:08.413525+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:16:41.634984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.031992+0000","last_change":"2024-05-09T10:51:07.424548+0000","last_active":"2024-05-09T10:51:09.031992+0000","last_peered":"2024-05-09T10:51:09.031992+0000","last_clean":"2024-05-09T10:51:09.031992+0000","last_became_active":"2024-05-09T10:51:07.424276+0000","last_became_peered":"2024-05-09T10:51:07.424276+0000","last_unstale":"2024-05-09T10:51:09.031992+0000","last_undegraded":"2024-05-09T10:51:09.031992+0000","last_fullsized":"2024-05-09T10:51:09.031992+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:22:02.388352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.413764+0000","last_change":"2024-05-09T10:51:07.420614+0000","last_active":"2024-05-09T10:51:08.413764+0000","last_peered":"2024-05-09T10:51:08.413764+0000","last_clean":"2024-05-09T10:51:08.413764+0000","last_became_active":"2024-05-09T10:51:07.420147+0000","last_became_peered":"2024-05-09T10:51:07.420147+0000","last_unstale":"2024-05-09T10:51:08.413764+0000","last_undegraded":"2024-05-09T10:51:08.413764+0000","last_fullsized":"2024-05-09T10:51:08.413764+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:58:29.190468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.034358+0000","last_change":"2024-05-09T10:51:07.413989+0000","last_active":"2024-05-09T10:51:09.034358+0000","last_peered":"2024-05-09T10:51:09.034358+0000","last_clean":"2024-05-09T10:51:09.034358+0000","last_became_active":"2024-05-09T10:51:07.413259+0000","last_became_peered":"2024-05-09T10:51:07.413259+0000","last_unstale":"2024-05-09T10:51:09.034358+0000","last_undegraded":"2024-05-09T10:51:09.034358+0000","last_fullsized":"2024-05-09T10:51:09.034358+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:14:32.486025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.034016+0000","last_change":"2024-05-09T10:51:07.417629+0000","last_active":"2024-05-09T10:51:09.034016+0000","last_peered":"2024-05-09T10:51:09.034016+0000","last_clean":"2024-05-09T10:51:09.034016+0000","last_became_active":"2024-05-09T10:51:07.417250+0000","last_became_peered":"2024-05-09T10:51:07.417250+0000","last_unstale":"2024-05-09T10:51:09.034016+0000","last_undegraded":"2024-05-09T10:51:09.034016+0000","last_fullsized":"2024-05-09T10:51:09.034016+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:36:58.548147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.032477+0000","last_change":"2024-05-09T10:51:07.417288+0000","last_active":"2024-05-09T10:51:09.032477+0000","last_peered":"2024-05-09T10:51:09.032477+0000","last_clean":"2024-05-09T10:51:09.032477+0000","last_became_active":"2024-05-09T10:51:07.416907+0000","last_became_peered":"2024-05-09T10:51:07.416907+0000","last_unstale":"2024-05-09T10:51:09.032477+0000","last_undegraded":"2024-05-09T10:51:09.032477+0000","last_fullsized":"2024-05-09T10:51:09.032477+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:33:40.854778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.032937+0000","last_change":"2024-05-09T10:51:07.415546+0000","last_active":"2024-05-09T10:51:09.032937+0000","last_peered":"2024-05-09T10:51:09.032937+0000","last_clean":"2024-05-09T10:51:09.032937+0000","last_became_active":"2024-05-09T10:51:07.415273+0000","last_became_peered":"2024-05-09T10:51:07.415273+0000","last_unstale":"2024-05-09T10:51:09.032937+0000","last_undegraded":"2024-05-09T10:51:09.032937+0000","last_fullsized":"2024-05-09T10:51:09.032937+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:32:51.515139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.032989+0000","last_change":"2024-05-09T10:51:07.420854+0000","last_active":"2024-05-09T10:51:09.032989+0000","last_peered":"2024-05-09T10:51:09.032989+0000","last_clean":"2024-05-09T10:51:09.032989+0000","last_became_active":"2024-05-09T10:51:07.420028+0000","last_became_peered":"2024-05-09T10:51:07.420028+0000","last_unstale":"2024-05-09T10:51:09.032989+0000","last_undegraded":"2024-05-09T10:51:09.032989+0000","last_fullsized":"2024-05-09T10:51:09.032989+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T21:35:24.776616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.411671+0000","last_change":"2024-05-09T10:51:07.410852+0000","last_active":"2024-05-09T10:51:08.411671+0000","last_peered":"2024-05-09T10:51:08.411671+0000","last_clean":"2024-05-09T10:51:08.411671+0000","last_became_active":"2024-05-09T10:51:07.410541+0000","last_became_peered":"2024-05-09T10:51:07.410541+0000","last_unstale":"2024-05-09T10:51:08.411671+0000","last_undegraded":"2024-05-09T10:51:08.411671+0000","last_fullsized":"2024-05-09T10:51:08.411671+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:52:57.065333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'4","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.032711+0000","last_change":"2024-05-09T10:51:06.399065+0000","last_active":"2024-05-09T10:51:09.032711+0000","last_peered":"2024-05-09T10:51:09.032711+0000","last_clean":"2024-05-09T10:51:09.032711+0000","last_became_active":"2024-05-09T10:51:06.398631+0000","last_became_peered":"2024-05-09T10:51:06.398631+0000","last_unstale":"2024-05-09T10:51:09.032711+0000","last_undegraded":"2024-05-09T10:51:09.032711+0000","last_fullsized":"2024-05-09T10:51:09.032711+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:48:18.069829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.032610+0000","last_change":"2024-05-09T10:51:07.422192+0000","last_active":"2024-05-09T10:51:09.032610+0000","last_peered":"2024-05-09T10:51:09.032610+0000","last_clean":"2024-05-09T10:51:09.032610+0000","last_became_active":"2024-05-09T10:51:07.421898+0000","last_became_peered":"2024-05-09T10:51:07.421898+0000","last_unstale":"2024-05-09T10:51:09.032610+0000","last_undegraded":"2024-05-09T10:51:09.032610+0000","last_fullsized":"2024-05-09T10:51:09.032610+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:05.168800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.411760+0000","last_change":"2024-05-09T10:51:06.405043+0000","last_active":"2024-05-09T10:51:08.411760+0000","last_peered":"2024-05-09T10:51:08.411760+0000","last_clean":"2024-05-09T10:51:08.411760+0000","last_became_active":"2024-05-09T10:51:06.404375+0000","last_became_peered":"2024-05-09T10:51:06.404375+0000","last_unstale":"2024-05-09T10:51:08.411760+0000","last_undegraded":"2024-05-09T10:51:08.411760+0000","last_fullsized":"2024-05-09T10:51:08.411760+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:45:51.854599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.033678+0000","last_change":"2024-05-09T10:51:07.422891+0000","last_active":"2024-05-09T10:51:09.033678+0000","last_peered":"2024-05-09T10:51:09.033678+0000","last_clean":"2024-05-09T10:51:09.033678+0000","last_became_active":"2024-05-09T10:51:07.422599+0000","last_became_peered":"2024-05-09T10:51:07.422599+0000","last_unstale":"2024-05-09T10:51:09.033678+0000","last_undegraded":"2024-05-09T10:51:09.033678+0000","last_fullsized":"2024-05-09T10:51:09.033678+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:24:07.909942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.031774+0000","last_change":"2024-05-09T10:51:06.403754+0000","last_active":"2024-05-09T10:51:09.031774+0000","last_peered":"2024-05-09T10:51:09.031774+0000","last_clean":"2024-05-09T10:51:09.031774+0000","last_became_active":"2024-05-09T10:51:06.403476+0000","last_became_peered":"2024-05-09T10:51:06.403476+0000","last_unstale":"2024-05-09T10:51:09.031774+0000","last_undegraded":"2024-05-09T10:51:09.031774+0000","last_fullsized":"2024-05-09T10:51:09.031774+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:48:48.946936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.413782+0000","last_change":"2024-05-09T10:51:07.420586+0000","last_active":"2024-05-09T10:51:08.413782+0000","last_peered":"2024-05-09T10:51:08.413782+0000","last_clean":"2024-05-09T10:51:08.413782+0000","last_became_active":"2024-05-09T10:51:07.419749+0000","last_became_peered":"2024-05-09T10:51:07.419749+0000","last_unstale":"2024-05-09T10:51:08.413782+0000","last_undegraded":"2024-05-09T10:51:08.413782+0000","last_fullsized":"2024-05-09T10:51:08.413782+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:37.530751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.412834+0000","last_change":"2024-05-09T10:51:06.401479+0000","last_active":"2024-05-09T10:51:08.412834+0000","last_peered":"2024-05-09T10:51:08.412834+0000","last_clean":"2024-05-09T10:51:08.412834+0000","last_became_active":"2024-05-09T10:51:06.401079+0000","last_became_peered":"2024-05-09T10:51:06.401079+0000","last_unstale":"2024-05-09T10:51:08.412834+0000","last_undegraded":"2024-05-09T10:51:08.412834+0000","last_fullsized":"2024-05-09T10:51:08.412834+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:26.083086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.032774+0000","last_change":"2024-05-09T10:51:07.419226+0000","last_active":"2024-05-09T10:51:09.032774+0000","last_peered":"2024-05-09T10:51:09.032774+0000","last_clean":"2024-05-09T10:51:09.032774+0000","last_became_active":"2024-05-09T10:51:07.418925+0000","last_became_peered":"2024-05-09T10:51:07.418925+0000","last_unstale":"2024-05-09T10:51:09.032774+0000","last_undegraded":"2024-05-09T10:51:09.032774+0000","last_fullsized":"2024-05-09T10:51:09.032774+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:51:10.642407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.034127+0000","last_change":"2024-05-09T10:51:06.403964+0000","last_active":"2024-05-09T10:51:09.034127+0000","last_peered":"2024-05-09T10:51:09.034127+0000","last_clean":"2024-05-09T10:51:09.034127+0000","last_became_active":"2024-05-09T10:51:06.403691+0000","last_became_peered":"2024-05-09T10:51:06.403691+0000","last_unstale":"2024-05-09T10:51:09.034127+0000","last_undegraded":"2024-05-09T10:51:09.034127+0000","last_fullsized":"2024-05-09T10:51:09.034127+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:51.058627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.033981+0000","last_change":"2024-05-09T10:51:07.411733+0000","last_active":"2024-05-09T10:51:09.033981+0000","last_peered":"2024-05-09T10:51:09.033981+0000","last_clean":"2024-05-09T10:51:09.033981+0000","last_became_active":"2024-05-09T10:51:07.411042+0000","last_became_peered":"2024-05-09T10:51:07.411042+0000","last_unstale":"2024-05-09T10:51:09.033981+0000","last_undegraded":"2024-05-09T10:51:09.033981+0000","last_fullsized":"2024-05-09T10:51:09.033981+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:37.530751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.033790+0000","last_change":"2024-05-09T10:51:06.403128+0000","last_active":"2024-05-09T10:51:09.033790+0000","last_peered":"2024-05-09T10:51:09.033790+0000","last_clean":"2024-05-09T10:51:09.033790+0000","last_became_active":"2024-05-09T10:51:06.402841+0000","last_became_peered":"2024-05-09T10:51:06.402841+0000","last_unstale":"2024-05-09T10:51:09.033790+0000","last_undegraded":"2024-05-09T10:51:09.033790+0000","last_fullsized":"2024-05-09T10:51:09.033790+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:48:18.069829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.033308+0000","last_change":"2024-05-09T10:51:07.414659+0000","last_active":"2024-05-09T10:51:09.033308+0000","last_peered":"2024-05-09T10:51:09.033308+0000","last_clean":"2024-05-09T10:51:09.033308+0000","last_became_active":"2024-05-09T10:51:07.414396+0000","last_became_peered":"2024-05-09T10:51:07.414396+0000","last_unstale":"2024-05-09T10:51:09.033308+0000","last_undegraded":"2024-05-09T10:51:09.033308+0000","last_fullsized":"2024-05-09T10:51:09.033308+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:52.418635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.031391+0000","last_change":"2024-05-09T10:51:06.410970+0000","last_active":"2024-05-09T10:51:09.031391+0000","last_peered":"2024-05-09T10:51:09.031391+0000","last_clean":"2024-05-09T10:51:09.031391+0000","last_became_active":"2024-05-09T10:51:06.410608+0000","last_became_peered":"2024-05-09T10:51:06.410608+0000","last_unstale":"2024-05-09T10:51:09.031391+0000","last_undegraded":"2024-05-09T10:51:09.031391+0000","last_fullsized":"2024-05-09T10:51:09.031391+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:55:35.891421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.413419+0000","last_change":"2024-05-09T10:51:07.414692+0000","last_active":"2024-05-09T10:51:08.413419+0000","last_peered":"2024-05-09T10:51:08.413419+0000","last_clean":"2024-05-09T10:51:08.413419+0000","last_became_active":"2024-05-09T10:51:07.414457+0000","last_became_peered":"2024-05-09T10:51:07.414457+0000","last_unstale":"2024-05-09T10:51:08.413419+0000","last_undegraded":"2024-05-09T10:51:08.413419+0000","last_fullsized":"2024-05-09T10:51:08.413419+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:36:58.548147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.033279+0000","last_change":"2024-05-09T10:51:06.403392+0000","last_active":"2024-05-09T10:51:09.033279+0000","last_peered":"2024-05-09T10:51:09.033279+0000","last_clean":"2024-05-09T10:51:09.033279+0000","last_became_active":"2024-05-09T10:51:06.403111+0000","last_became_peered":"2024-05-09T10:51:06.403111+0000","last_unstale":"2024-05-09T10:51:09.033279+0000","last_undegraded":"2024-05-09T10:51:09.033279+0000","last_fullsized":"2024-05-09T10:51:09.033279+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:04:00.333373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.034108+0000","last_change":"2024-05-09T10:51:07.422640+0000","last_active":"2024-05-09T10:51:09.034108+0000","last_peered":"2024-05-09T10:51:09.034108+0000","last_clean":"2024-05-09T10:51:09.034108+0000","last_became_active":"2024-05-09T10:51:07.422332+0000","last_became_peered":"2024-05-09T10:51:07.422332+0000","last_unstale":"2024-05-09T10:51:09.034108+0000","last_undegraded":"2024-05-09T10:51:09.034108+0000","last_fullsized":"2024-05-09T10:51:09.034108+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:51:10.642407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.031866+0000","last_change":"2024-05-09T10:51:06.406795+0000","last_active":"2024-05-09T10:51:09.031866+0000","last_peered":"2024-05-09T10:51:09.031866+0000","last_clean":"2024-05-09T10:51:09.031866+0000","last_became_active":"2024-05-09T10:51:06.406516+0000","last_became_peered":"2024-05-09T10:51:06.406516+0000","last_unstale":"2024-05-09T10:51:09.031866+0000","last_undegraded":"2024-05-09T10:51:09.031866+0000","last_fullsized":"2024-05-09T10:51:09.031866+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:04.164801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.031865+0000","last_change":"2024-05-09T10:51:07.421032+0000","last_active":"2024-05-09T10:51:09.031865+0000","last_peered":"2024-05-09T10:51:09.031865+0000","last_clean":"2024-05-09T10:51:09.031865+0000","last_became_active":"2024-05-09T10:51:07.420750+0000","last_became_peered":"2024-05-09T10:51:07.420750+0000","last_unstale":"2024-05-09T10:51:09.031865+0000","last_undegraded":"2024-05-09T10:51:09.031865+0000","last_fullsized":"2024-05-09T10:51:09.031865+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:24:46.373182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.411852+0000","last_change":"2024-05-09T10:51:06.399077+0000","last_active":"2024-05-09T10:51:08.411852+0000","last_peered":"2024-05-09T10:51:08.411852+0000","last_clean":"2024-05-09T10:51:08.411852+0000","last_became_active":"2024-05-09T10:51:06.398760+0000","last_became_peered":"2024-05-09T10:51:06.398760+0000","last_unstale":"2024-05-09T10:51:08.411852+0000","last_undegraded":"2024-05-09T10:51:08.411852+0000","last_fullsized":"2024-05-09T10:51:08.411852+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:29.722036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.031747+0000","last_change":"2024-05-09T10:51:07.421000+0000","last_active":"2024-05-09T10:51:09.031747+0000","last_peered":"2024-05-09T10:51:09.031747+0000","last_clean":"2024-05-09T10:51:09.031747+0000","last_became_active":"2024-05-09T10:51:07.420715+0000","last_became_peered":"2024-05-09T10:51:07.420715+0000","last_unstale":"2024-05-09T10:51:09.031747+0000","last_undegraded":"2024-05-09T10:51:09.031747+0000","last_fullsized":"2024-05-09T10:51:09.031747+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:02:08.562027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"30'3","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.034534+0000","last_change":"2024-05-09T10:51:06.402648+0000","last_active":"2024-05-09T10:51:09.034534+0000","last_peered":"2024-05-09T10:51:09.034534+0000","last_clean":"2024-05-09T10:51:09.034534+0000","last_became_active":"2024-05-09T10:51:06.402315+0000","last_became_peered":"2024-05-09T10:51:06.402315+0000","last_unstale":"2024-05-09T10:51:09.034534+0000","last_undegraded":"2024-05-09T10:51:09.034534+0000","last_fullsized":"2024-05-09T10:51:09.034534+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:55.642595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.034386+0000","last_change":"2024-05-09T10:51:07.416940+0000","last_active":"2024-05-09T10:51:09.034386+0000","last_peered":"2024-05-09T10:51:09.034386+0000","last_clean":"2024-05-09T10:51:09.034386+0000","last_became_active":"2024-05-09T10:51:07.416562+0000","last_became_peered":"2024-05-09T10:51:07.416562+0000","last_unstale":"2024-05-09T10:51:09.034386+0000","last_undegraded":"2024-05-09T10:51:09.034386+0000","last_fullsized":"2024-05-09T10:51:09.034386+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:52:15.329752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"30'6","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.031675+0000","last_change":"2024-05-09T10:51:06.403735+0000","last_active":"2024-05-09T10:51:09.031675+0000","last_peered":"2024-05-09T10:51:09.031675+0000","last_clean":"2024-05-09T10:51:09.031675+0000","last_became_active":"2024-05-09T10:51:06.403469+0000","last_became_peered":"2024-05-09T10:51:06.403469+0000","last_unstale":"2024-05-09T10:51:09.031675+0000","last_undegraded":"2024-05-09T10:51:09.031675+0000","last_fullsized":"2024-05-09T10:51:09.031675+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:24:06.905943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.413629+0000","last_change":"2024-05-09T10:51:07.417021+0000","last_active":"2024-05-09T10:51:08.413629+0000","last_peered":"2024-05-09T10:51:08.413629+0000","last_clean":"2024-05-09T10:51:08.413629+0000","last_became_active":"2024-05-09T10:51:07.416722+0000","last_became_peered":"2024-05-09T10:51:07.416722+0000","last_unstale":"2024-05-09T10:51:08.413629+0000","last_undegraded":"2024-05-09T10:51:08.413629+0000","last_fullsized":"2024-05-09T10:51:08.413629+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:33:40.854778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.034137+0000","last_change":"2024-05-09T10:51:06.406522+0000","last_active":"2024-05-09T10:51:09.034137+0000","last_peered":"2024-05-09T10:51:09.034137+0000","last_clean":"2024-05-09T10:51:09.034137+0000","last_became_active":"2024-05-09T10:51:06.406242+0000","last_became_peered":"2024-05-09T10:51:06.406242+0000","last_unstale":"2024-05-09T10:51:09.034137+0000","last_undegraded":"2024-05-09T10:51:09.034137+0000","last_fullsized":"2024-05-09T10:51:09.034137+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:03.704175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.031737+0000","last_change":"2024-05-09T10:51:07.418792+0000","last_active":"2024-05-09T10:51:09.031737+0000","last_peered":"2024-05-09T10:51:09.031737+0000","last_clean":"2024-05-09T10:51:09.031737+0000","last_became_active":"2024-05-09T10:51:07.418533+0000","last_became_peered":"2024-05-09T10:51:07.418533+0000","last_unstale":"2024-05-09T10:51:09.031737+0000","last_undegraded":"2024-05-09T10:51:09.031737+0000","last_fullsized":"2024-05-09T10:51:09.031737+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:13:39.637223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.413590+0000","last_change":"2024-05-09T10:51:06.402262+0000","last_active":"2024-05-09T10:51:08.413590+0000","last_peered":"2024-05-09T10:51:08.413590+0000","last_clean":"2024-05-09T10:51:08.413590+0000","last_became_active":"2024-05-09T10:51:06.401936+0000","last_became_peered":"2024-05-09T10:51:06.401936+0000","last_unstale":"2024-05-09T10:51:08.413590+0000","last_undegraded":"2024-05-09T10:51:08.413590+0000","last_fullsized":"2024-05-09T10:51:08.413590+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:35:29.722036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.032467+0000","last_change":"2024-05-09T10:51:07.421105+0000","last_active":"2024-05-09T10:51:09.032467+0000","last_peered":"2024-05-09T10:51:09.032467+0000","last_clean":"2024-05-09T10:51:09.032467+0000","last_became_active":"2024-05-09T10:51:07.419826+0000","last_became_peered":"2024-05-09T10:51:07.419826+0000","last_unstale":"2024-05-09T10:51:09.032467+0000","last_undegraded":"2024-05-09T10:51:09.032467+0000","last_fullsized":"2024-05-09T10:51:09.032467+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:58:29.190468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.031480+0000","last_change":"2024-05-09T10:51:06.405127+0000","last_active":"2024-05-09T10:51:09.031480+0000","last_peered":"2024-05-09T10:51:09.031480+0000","last_clean":"2024-05-09T10:51:09.031480+0000","last_became_active":"2024-05-09T10:51:06.404682+0000","last_became_peered":"2024-05-09T10:51:06.404682+0000","last_unstale":"2024-05-09T10:51:09.031480+0000","last_undegraded":"2024-05-09T10:51:09.031480+0000","last_fullsized":"2024-05-09T10:51:09.031480+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:55.642595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.032516+0000","last_change":"2024-05-09T10:51:07.421251+0000","last_active":"2024-05-09T10:51:09.032516+0000","last_peered":"2024-05-09T10:51:09.032516+0000","last_clean":"2024-05-09T10:51:09.032516+0000","last_became_active":"2024-05-09T10:51:07.420771+0000","last_became_peered":"2024-05-09T10:51:07.420771+0000","last_unstale":"2024-05-09T10:51:09.032516+0000","last_undegraded":"2024-05-09T10:51:09.032516+0000","last_fullsized":"2024-05-09T10:51:09.032516+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:48:49.950935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"30'8","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.031444+0000","last_change":"2024-05-09T10:51:06.401431+0000","last_active":"2024-05-09T10:51:09.031444+0000","last_peered":"2024-05-09T10:51:09.031444+0000","last_clean":"2024-05-09T10:51:09.031444+0000","last_became_active":"2024-05-09T10:51:06.401097+0000","last_became_peered":"2024-05-09T10:51:06.401097+0000","last_unstale":"2024-05-09T10:51:09.031444+0000","last_undegraded":"2024-05-09T10:51:09.031444+0000","last_fullsized":"2024-05-09T10:51:09.031444+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:51.414636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.033626+0000","last_change":"2024-05-09T10:51:07.413716+0000","last_active":"2024-05-09T10:51:09.033626+0000","last_peered":"2024-05-09T10:51:09.033626+0000","last_clean":"2024-05-09T10:51:09.033626+0000","last_became_active":"2024-05-09T10:51:07.412889+0000","last_became_peered":"2024-05-09T10:51:07.412889+0000","last_unstale":"2024-05-09T10:51:09.033626+0000","last_undegraded":"2024-05-09T10:51:09.033626+0000","last_fullsized":"2024-05-09T10:51:09.033626+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:52:57.065333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.413581+0000","last_change":"2024-05-09T10:51:06.403074+0000","last_active":"2024-05-09T10:51:08.413581+0000","last_peered":"2024-05-09T10:51:08.413581+0000","last_clean":"2024-05-09T10:51:08.413581+0000","last_became_active":"2024-05-09T10:51:06.402721+0000","last_became_peered":"2024-05-09T10:51:06.402721+0000","last_unstale":"2024-05-09T10:51:08.413581+0000","last_undegraded":"2024-05-09T10:51:08.413581+0000","last_fullsized":"2024-05-09T10:51:08.413581+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:21.393250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.034503+0000","last_change":"2024-05-09T10:51:07.411319+0000","last_active":"2024-05-09T10:51:09.034503+0000","last_peered":"2024-05-09T10:51:09.034503+0000","last_clean":"2024-05-09T10:51:09.034503+0000","last_became_active":"2024-05-09T10:51:07.411076+0000","last_became_peered":"2024-05-09T10:51:07.411076+0000","last_unstale":"2024-05-09T10:51:09.034503+0000","last_undegraded":"2024-05-09T10:51:09.034503+0000","last_fullsized":"2024-05-09T10:51:09.034503+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:19:22.881499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.033742+0000","last_change":"2024-05-09T10:51:06.407069+0000","last_active":"2024-05-09T10:51:09.033742+0000","last_peered":"2024-05-09T10:51:09.033742+0000","last_clean":"2024-05-09T10:51:09.033742+0000","last_became_active":"2024-05-09T10:51:06.406780+0000","last_became_peered":"2024-05-09T10:51:06.406780+0000","last_unstale":"2024-05-09T10:51:09.033742+0000","last_undegraded":"2024-05-09T10:51:09.033742+0000","last_fullsized":"2024-05-09T10:51:09.033742+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:21.607813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.413466+0000","last_change":"2024-05-09T10:51:07.417473+0000","last_active":"2024-05-09T10:51:08.413466+0000","last_peered":"2024-05-09T10:51:08.413466+0000","last_clean":"2024-05-09T10:51:08.413466+0000","last_became_active":"2024-05-09T10:51:07.417234+0000","last_became_peered":"2024-05-09T10:51:07.417234+0000","last_unstale":"2024-05-09T10:51:08.413466+0000","last_undegraded":"2024-05-09T10:51:08.413466+0000","last_fullsized":"2024-05-09T10:51:08.413466+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:13:12.976022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.033614+0000","last_change":"2024-05-09T10:51:06.411247+0000","last_active":"2024-05-09T10:51:09.033614+0000","last_peered":"2024-05-09T10:51:09.033614+0000","last_clean":"2024-05-09T10:51:09.033614+0000","last_became_active":"2024-05-09T10:51:06.411001+0000","last_became_peered":"2024-05-09T10:51:06.411001+0000","last_unstale":"2024-05-09T10:51:09.033614+0000","last_undegraded":"2024-05-09T10:51:09.033614+0000","last_fullsized":"2024-05-09T10:51:09.033614+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:35:03.482963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.413098+0000","last_change":"2024-05-09T10:51:07.411513+0000","last_active":"2024-05-09T10:51:08.413098+0000","last_peered":"2024-05-09T10:51:08.413098+0000","last_clean":"2024-05-09T10:51:08.413098+0000","last_became_active":"2024-05-09T10:51:07.411247+0000","last_became_peered":"2024-05-09T10:51:07.411247+0000","last_unstale":"2024-05-09T10:51:08.413098+0000","last_undegraded":"2024-05-09T10:51:08.413098+0000","last_fullsized":"2024-05-09T10:51:08.413098+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:22.397249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.033867+0000","last_change":"2024-05-09T10:51:06.405035+0000","last_active":"2024-05-09T10:51:09.033867+0000","last_peered":"2024-05-09T10:51:09.033867+0000","last_clean":"2024-05-09T10:51:09.033867+0000","last_became_active":"2024-05-09T10:51:06.404712+0000","last_became_peered":"2024-05-09T10:51:06.404712+0000","last_unstale":"2024-05-09T10:51:09.033867+0000","last_undegraded":"2024-05-09T10:51:09.033867+0000","last_fullsized":"2024-05-09T10:51:09.033867+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:52:10.065444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.412943+0000","last_change":"2024-05-09T10:51:07.418253+0000","last_active":"2024-05-09T10:51:08.412943+0000","last_peered":"2024-05-09T10:51:08.412943+0000","last_clean":"2024-05-09T10:51:08.412943+0000","last_became_active":"2024-05-09T10:51:07.417960+0000","last_became_peered":"2024-05-09T10:51:07.417960+0000","last_unstale":"2024-05-09T10:51:08.412943+0000","last_undegraded":"2024-05-09T10:51:08.412943+0000","last_fullsized":"2024-05-09T10:51:08.412943+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:04.708175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.411719+0000","last_change":"2024-05-09T10:51:06.396633+0000","last_active":"2024-05-09T10:51:08.411719+0000","last_peered":"2024-05-09T10:51:08.411719+0000","last_clean":"2024-05-09T10:51:08.411719+0000","last_became_active":"2024-05-09T10:51:06.396277+0000","last_became_peered":"2024-05-09T10:51:06.396277+0000","last_unstale":"2024-05-09T10:51:08.411719+0000","last_undegraded":"2024-05-09T10:51:08.411719+0000","last_fullsized":"2024-05-09T10:51:08.411719+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:48:18.069829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.413004+0000","last_change":"2024-05-09T10:51:07.410669+0000","last_active":"2024-05-09T10:51:08.413004+0000","last_peered":"2024-05-09T10:51:08.413004+0000","last_clean":"2024-05-09T10:51:08.413004+0000","last_became_active":"2024-05-09T10:51:07.410390+0000","last_became_peered":"2024-05-09T10:51:07.410390+0000","last_unstale":"2024-05-09T10:51:08.413004+0000","last_undegraded":"2024-05-09T10:51:08.413004+0000","last_fullsized":"2024-05-09T10:51:08.413004+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:48:19.073828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.032844+0000","last_change":"2024-05-09T10:51:06.405148+0000","last_active":"2024-05-09T10:51:09.032844+0000","last_peered":"2024-05-09T10:51:09.032844+0000","last_clean":"2024-05-09T10:51:09.032844+0000","last_became_active":"2024-05-09T10:51:06.403901+0000","last_became_peered":"2024-05-09T10:51:06.403901+0000","last_unstale":"2024-05-09T10:51:09.032844+0000","last_undegraded":"2024-05-09T10:51:09.032844+0000","last_fullsized":"2024-05-09T10:51:09.032844+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:39:03.704175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.412534+0000","last_change":"2024-05-09T10:51:07.415295+0000","last_active":"2024-05-09T10:51:08.412534+0000","last_peered":"2024-05-09T10:51:08.412534+0000","last_clean":"2024-05-09T10:51:08.412534+0000","last_became_active":"2024-05-09T10:51:07.414992+0000","last_became_peered":"2024-05-09T10:51:07.414992+0000","last_unstale":"2024-05-09T10:51:08.412534+0000","last_undegraded":"2024-05-09T10:51:08.412534+0000","last_fullsized":"2024-05-09T10:51:08.412534+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:35:04.486962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.033349+0000","last_change":"2024-05-09T10:51:06.404951+0000","last_active":"2024-05-09T10:51:09.033349+0000","last_peered":"2024-05-09T10:51:09.033349+0000","last_clean":"2024-05-09T10:51:09.033349+0000","last_became_active":"2024-05-09T10:51:06.404574+0000","last_became_peered":"2024-05-09T10:51:06.404574+0000","last_unstale":"2024-05-09T10:51:09.033349+0000","last_undegraded":"2024-05-09T10:51:09.033349+0000","last_fullsized":"2024-05-09T10:51:09.033349+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:29.685531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.032994+0000","last_change":"2024-05-09T10:51:07.414739+0000","last_active":"2024-05-09T10:51:09.032994+0000","last_peered":"2024-05-09T10:51:09.032994+0000","last_clean":"2024-05-09T10:51:09.032994+0000","last_became_active":"2024-05-09T10:51:07.414049+0000","last_became_peered":"2024-05-09T10:51:07.414049+0000","last_unstale":"2024-05-09T10:51:09.032994+0000","last_undegraded":"2024-05-09T10:51:09.032994+0000","last_fullsized":"2024-05-09T10:51:09.032994+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:14:32.486025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.033519+0000","last_change":"2024-05-09T10:51:06.404809+0000","last_active":"2024-05-09T10:51:09.033519+0000","last_peered":"2024-05-09T10:51:09.033519+0000","last_clean":"2024-05-09T10:51:09.033519+0000","last_became_active":"2024-05-09T10:51:06.404529+0000","last_became_peered":"2024-05-09T10:51:06.404529+0000","last_unstale":"2024-05-09T10:51:09.033519+0000","last_undegraded":"2024-05-09T10:51:09.033519+0000","last_fullsized":"2024-05-09T10:51:09.033519+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:48:48.946936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.412577+0000","last_change":"2024-05-09T10:51:07.414278+0000","last_active":"2024-05-09T10:51:08.412577+0000","last_peered":"2024-05-09T10:51:08.412577+0000","last_clean":"2024-05-09T10:51:08.412577+0000","last_became_active":"2024-05-09T10:51:07.414025+0000","last_became_peered":"2024-05-09T10:51:07.414025+0000","last_unstale":"2024-05-09T10:51:08.412577+0000","last_undegraded":"2024-05-09T10:51:08.412577+0000","last_fullsized":"2024-05-09T10:51:08.412577+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:47:22.611813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.413635+0000","last_change":"2024-05-09T10:51:06.398765+0000","last_active":"2024-05-09T10:51:08.413635+0000","last_peered":"2024-05-09T10:51:08.413635+0000","last_clean":"2024-05-09T10:51:08.413635+0000","last_became_active":"2024-05-09T10:51:06.397480+0000","last_became_peered":"2024-05-09T10:51:06.397480+0000","last_unstale":"2024-05-09T10:51:08.413635+0000","last_undegraded":"2024-05-09T10:51:08.413635+0000","last_fullsized":"2024-05-09T10:51:08.413635+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:45:51.854599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.034546+0000","last_change":"2024-05-09T10:51:07.416793+0000","last_active":"2024-05-09T10:51:09.034546+0000","last_peered":"2024-05-09T10:51:09.034546+0000","last_clean":"2024-05-09T10:51:09.034546+0000","last_became_active":"2024-05-09T10:51:07.416307+0000","last_became_peered":"2024-05-09T10:51:07.416307+0000","last_unstale":"2024-05-09T10:51:09.034546+0000","last_undegraded":"2024-05-09T10:51:09.034546+0000","last_fullsized":"2024-05-09T10:51:09.034546+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:13:39.637223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"31'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.411937+0000","last_change":"2024-05-09T10:51:06.399613+0000","last_active":"2024-05-09T10:51:08.411937+0000","last_peered":"2024-05-09T10:51:08.411937+0000","last_clean":"2024-05-09T10:51:08.411937+0000","last_became_active":"2024-05-09T10:51:06.399333+0000","last_became_peered":"2024-05-09T10:51:06.399333+0000","last_unstale":"2024-05-09T10:51:08.411937+0000","last_undegraded":"2024-05-09T10:51:08.411937+0000","last_fullsized":"2024-05-09T10:51:08.411937+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:30:44.917770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.411583+0000","last_change":"2024-05-09T10:51:07.414339+0000","last_active":"2024-05-09T10:51:08.411583+0000","last_peered":"2024-05-09T10:51:08.411583+0000","last_clean":"2024-05-09T10:51:08.411583+0000","last_became_active":"2024-05-09T10:51:07.414069+0000","last_became_peered":"2024-05-09T10:51:07.414069+0000","last_unstale":"2024-05-09T10:51:08.411583+0000","last_undegraded":"2024-05-09T10:51:08.411583+0000","last_fullsized":"2024-05-09T10:51:08.411583+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:32:51.515139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.413563+0000","last_change":"2024-05-09T10:51:06.396469+0000","last_active":"2024-05-09T10:51:08.413563+0000","last_peered":"2024-05-09T10:51:08.413563+0000","last_clean":"2024-05-09T10:51:08.413563+0000","last_became_active":"2024-05-09T10:51:06.396180+0000","last_became_peered":"2024-05-09T10:51:06.396180+0000","last_unstale":"2024-05-09T10:51:08.413563+0000","last_undegraded":"2024-05-09T10:51:08.413563+0000","last_fullsized":"2024-05-09T10:51:08.413563+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:48:18.069829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.032748+0000","last_change":"2024-05-09T10:51:07.414878+0000","last_active":"2024-05-09T10:51:09.032748+0000","last_peered":"2024-05-09T10:51:09.032748+0000","last_clean":"2024-05-09T10:51:09.032748+0000","last_became_active":"2024-05-09T10:51:07.414537+0000","last_became_peered":"2024-05-09T10:51:07.414537+0000","last_unstale":"2024-05-09T10:51:09.032748+0000","last_undegraded":"2024-05-09T10:51:09.032748+0000","last_fullsized":"2024-05-09T10:51:09.032748+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:47:52.418635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.411820+0000","last_change":"2024-05-09T10:51:06.394559+0000","last_active":"2024-05-09T10:51:08.411820+0000","last_peered":"2024-05-09T10:51:08.411820+0000","last_clean":"2024-05-09T10:51:08.411820+0000","last_became_active":"2024-05-09T10:51:06.394288+0000","last_became_peered":"2024-05-09T10:51:06.394288+0000","last_unstale":"2024-05-09T10:51:08.411820+0000","last_undegraded":"2024-05-09T10:51:08.411820+0000","last_fullsized":"2024-05-09T10:51:08.411820+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:48:21.393250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.411697+0000","last_change":"2024-05-09T10:51:07.416799+0000","last_active":"2024-05-09T10:51:08.411697+0000","last_peered":"2024-05-09T10:51:08.411697+0000","last_clean":"2024-05-09T10:51:08.411697+0000","last_became_active":"2024-05-09T10:51:07.416529+0000","last_became_peered":"2024-05-09T10:51:07.416529+0000","last_unstale":"2024-05-09T10:51:08.411697+0000","last_undegraded":"2024-05-09T10:51:08.411697+0000","last_fullsized":"2024-05-09T10:51:08.411697+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:24:07.909942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:08.413683+0000","last_change":"2024-05-09T10:51:06.400034+0000","last_active":"2024-05-09T10:51:08.413683+0000","last_peered":"2024-05-09T10:51:08.413683+0000","last_clean":"2024-05-09T10:51:08.413683+0000","last_became_active":"2024-05-09T10:51:06.397993+0000","last_became_peered":"2024-05-09T10:51:06.397993+0000","last_unstale":"2024-05-09T10:51:08.413683+0000","last_undegraded":"2024-05-09T10:51:08.413683+0000","last_fullsized":"2024-05-09T10:51:08.413683+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:30:44.917770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.031428+0000","last_change":"2024-05-09T10:51:07.420717+0000","last_active":"2024-05-09T10:51:09.031428+0000","last_peered":"2024-05-09T10:51:09.031428+0000","last_clean":"2024-05-09T10:51:09.031428+0000","last_became_active":"2024-05-09T10:51:07.420365+0000","last_became_peered":"2024-05-09T10:51:07.420365+0000","last_unstale":"2024-05-09T10:51:09.031428+0000","last_undegraded":"2024-05-09T10:51:09.031428+0000","last_fullsized":"2024-05-09T10:51:09.031428+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:14:32.486025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.033472+0000","last_change":"2024-05-09T10:51:06.400625+0000","last_active":"2024-05-09T10:51:09.033472+0000","last_peered":"2024-05-09T10:51:09.033472+0000","last_clean":"2024-05-09T10:51:09.033472+0000","last_became_active":"2024-05-09T10:51:06.400119+0000","last_became_peered":"2024-05-09T10:51:06.400119+0000","last_unstale":"2024-05-09T10:51:09.033472+0000","last_undegraded":"2024-05-09T10:51:09.033472+0000","last_fullsized":"2024-05-09T10:51:09.033472+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:05:03.035522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.033452+0000","last_change":"2024-05-09T10:51:07.417841+0000","last_active":"2024-05-09T10:51:09.033452+0000","last_peered":"2024-05-09T10:51:09.033452+0000","last_clean":"2024-05-09T10:51:09.033452+0000","last_became_active":"2024-05-09T10:51:07.417569+0000","last_became_peered":"2024-05-09T10:51:07.417569+0000","last_unstale":"2024-05-09T10:51:09.033452+0000","last_undegraded":"2024-05-09T10:51:09.033452+0000","last_fullsized":"2024-05-09T10:51:09.033452+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:32:51.515139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.031612+0000","last_change":"2024-05-09T10:51:06.406673+0000","last_active":"2024-05-09T10:51:09.031612+0000","last_peered":"2024-05-09T10:51:09.031612+0000","last_clean":"2024-05-09T10:51:09.031612+0000","last_became_active":"2024-05-09T10:51:06.406391+0000","last_became_peered":"2024-05-09T10:51:06.406391+0000","last_unstale":"2024-05-09T10:51:09.031612+0000","last_undegraded":"2024-05-09T10:51:09.031612+0000","last_fullsized":"2024-05-09T10:51:09.031612+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:32:50.511140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.033053+0000","last_change":"2024-05-09T10:51:07.411787+0000","last_active":"2024-05-09T10:51:09.033053+0000","last_peered":"2024-05-09T10:51:09.033053+0000","last_clean":"2024-05-09T10:51:09.033053+0000","last_became_active":"2024-05-09T10:51:07.411510+0000","last_became_peered":"2024-05-09T10:51:07.411510+0000","last_unstale":"2024-05-09T10:51:09.033053+0000","last_undegraded":"2024-05-09T10:51:09.033053+0000","last_fullsized":"2024-05-09T10:51:09.033053+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:06.382079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:06.382079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:52:15.329752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.032255+0000","last_change":"2024-05-09T10:51:06.407600+0000","last_active":"2024-05-09T10:51:09.032255+0000","last_peered":"2024-05-09T10:51:09.032255+0000","last_clean":"2024-05-09T10:51:09.032255+0000","last_became_active":"2024-05-09T10:51:06.407354+0000","last_became_peered":"2024-05-09T10:51:06.407354+0000","last_unstale":"2024-05-09T10:51:09.032255+0000","last_undegraded":"2024-05-09T10:51:09.032255+0000","last_fullsized":"2024-05-09T10:51:09.032255+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:51:05.378079+0000","last_clean_scrub_stamp":"2024-05-09T10:51:05.378079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:05:03.035522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T10:51:09.032896+0000","last_change":"2024-05-09T10:50:08.410364+0000","last_active":"2024-05-09T10:51:09.032896+0000","last_peered":"2024-05-09T10:51:09.032896+0000","last_clean":"2024-05-09T10:51:09.032896+0000","last_became_active":"2024-05-09T10:50:08.409457+0000","last_became_peered":"2024-05-09T10:50:08.409457+0000","last_unstale":"2024-05-09T10:51:09.032896+0000","last_undegraded":"2024-05-09T10:51:09.032896+0000","last_fullsized":"2024-05-09T10:51:09.032896+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_clean_scrub_stamp":"2024-05-09T10:50:07.396510+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:03.566698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":946176,"data_stored":724584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":134742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":134798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":134230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":135760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":352256,"data_stored":134196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":135760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":724494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T10:51:15.054 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-09T10:51:15.054 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-09T10:51:15.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.202+0000 7f6a12cbc640 1 Processor -- start 2024-05-09T10:51:15.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.202+0000 7f6a12cbc640 1 -- start start 2024-05-09T10:51:15.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.203+0000 7f6a12cbc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6a0c15dff0 0x7f6a0c0cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:15.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.203+0000 7f6a12cbc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a0c0d1b10 0x7f6a0c0d21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:15.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.203+0000 7f6a12cbc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6a0c0cf740 0x7f6a0c0d2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:15.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.203+0000 7f6a12cbc640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a0c0b3250 con 0x7f6a0c0cf740 2024-05-09T10:51:15.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.203+0000 7f6a12cbc640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a0c0b3530 con 0x7f6a0c15dff0 2024-05-09T10:51:15.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.203+0000 7f6a12cbc640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c0b3810 con 0x7f6a0c0d1b10 2024-05-09T10:51:15.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.203+0000 7f6a11259640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6a0c0cf740 0x7f6a0c0d2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:15.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.203+0000 7f6a11259640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6a0c0cf740 0x7f6a0c0d2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57476/0 (socket says 172.21.15.98:57476) 2024-05-09T10:51:15.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.203+0000 7f6a11259640 1 -- 172.21.15.98:0/2580426771 learned_addr learned my addr 172.21.15.98:0/2580426771 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:15.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.203+0000 7f6a11259640 1 -- 172.21.15.98:0/2580426771 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a0c0d1b10 msgr2=0x7f6a0c0d21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:15.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.203+0000 7f6a0bfff640 1 --2- 172.21.15.98:0/2580426771 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a0c0d1b10 0x7f6a0c0d21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:15.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.203+0000 7f6a10a58640 1 --2- 172.21.15.98:0/2580426771 >> v2:172.21.15.105:3300/0 conn(0x7f6a0c15dff0 0x7f6a0c0cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:15.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.203+0000 7f6a11259640 1 --2- 172.21.15.98:0/2580426771 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a0c0d1b10 0x7f6a0c0d21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.203+0000 7f6a11259640 1 -- 172.21.15.98:0/2580426771 >> v2:172.21.15.105:3300/0 conn(0x7f6a0c15dff0 msgr2=0x7f6a0c0cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:15.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.203+0000 7f6a11259640 1 --2- 172.21.15.98:0/2580426771 >> v2:172.21.15.105:3300/0 conn(0x7f6a0c15dff0 0x7f6a0c0cf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.203+0000 7f6a11259640 1 -- 172.21.15.98:0/2580426771 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0c0b3af0 con 0x7f6a0c0cf740 2024-05-09T10:51:15.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.204+0000 7f6a11259640 1 --2- 172.21.15.98:0/2580426771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a0c0cf740 0x7f6a0c0d2700 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c08cad0 tx=0x7f6a0402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2d927fe32642b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:15.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.204+0000 7f6a0b7fe640 1 -- 172.21.15.98:0/2580426771 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a04036070 con 0x7f6a0c0cf740 2024-05-09T10:51:15.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.204+0000 7f6a0b7fe640 1 -- 172.21.15.98:0/2580426771 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a04031600 con 0x7f6a0c0cf740 2024-05-09T10:51:15.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.204+0000 7f6a0b7fe640 1 -- 172.21.15.98:0/2580426771 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a040319d0 con 0x7f6a0c0cf740 2024-05-09T10:51:15.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.205+0000 7f6a12cbc640 1 -- 172.21.15.98:0/2580426771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a0c0cf740 msgr2=0x7f6a0c0d2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:15.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.205+0000 7f6a12cbc640 1 --2- 172.21.15.98:0/2580426771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a0c0cf740 0x7f6a0c0d2700 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c08cad0 tx=0x7f6a0402a6a0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.205+0000 7f6a12cbc640 1 -- 172.21.15.98:0/2580426771 shutdown_connections 2024-05-09T10:51:15.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.205+0000 7f6a12cbc640 1 --2- 172.21.15.98:0/2580426771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a0c0cf740 0x7f6a0c0d2700 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.205+0000 7f6a12cbc640 1 --2- 172.21.15.98:0/2580426771 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a0c0d1b10 0x7f6a0c0d21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.205+0000 7f6a12cbc640 1 --2- 172.21.15.98:0/2580426771 >> v2:172.21.15.105:3300/0 conn(0x7f6a0c15dff0 0x7f6a0c0cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.205+0000 7f6a12cbc640 1 -- 172.21.15.98:0/2580426771 >> 172.21.15.98:0/2580426771 conn(0x7f6a0c09cec0 msgr2=0x7f6a0c0d3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:15.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.205+0000 7f6a12cbc640 1 -- 172.21.15.98:0/2580426771 shutdown_connections 2024-05-09T10:51:15.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.205+0000 7f6a12cbc640 1 -- 172.21.15.98:0/2580426771 wait complete. 2024-05-09T10:51:15.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.206+0000 7f6a12cbc640 1 Processor -- start 2024-05-09T10:51:15.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.206+0000 7f6a12cbc640 1 -- start start 2024-05-09T10:51:15.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.207+0000 7f6a12cbc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a0c0cf740 0x7f6a0c0cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:15.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.207+0000 7f6a12cbc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a0c0d1b10 0x7f6a0c1591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:15.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.207+0000 7f6a12cbc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a0c15dff0 0x7f6a0c15be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:15.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.207+0000 7f6a12cbc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c0d0370 con 0x7f6a0c0cf740 2024-05-09T10:51:15.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.207+0000 7f6a12cbc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c0d01f0 con 0x7f6a0c15dff0 2024-05-09T10:51:15.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.207+0000 7f6a12cbc640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c0d04f0 con 0x7f6a0c0d1b10 2024-05-09T10:51:15.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.207+0000 7f6a10a58640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a0c0cf740 0x7f6a0c0cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:15.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.207+0000 7f6a10a58640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a0c0cf740 0x7f6a0c0cb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57478/0 (socket says 172.21.15.98:57478) 2024-05-09T10:51:15.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.207+0000 7f6a10a58640 1 -- 172.21.15.98:0/265578698 learned_addr learned my addr 172.21.15.98:0/265578698 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:15.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.207+0000 7f6a0bfff640 1 --2- 172.21.15.98:0/265578698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a0c0d1b10 0x7f6a0c1591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:15.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.207+0000 7f6a11259640 1 --2- 172.21.15.98:0/265578698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a0c15dff0 0x7f6a0c15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:15.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.207+0000 7f6a10a58640 1 -- 172.21.15.98:0/265578698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a0c0d1b10 msgr2=0x7f6a0c1591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:15.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.208+0000 7f6a10a58640 1 --2- 172.21.15.98:0/265578698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a0c0d1b10 0x7f6a0c1591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.208+0000 7f6a10a58640 1 -- 172.21.15.98:0/265578698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a0c15dff0 msgr2=0x7f6a0c15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:15.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.208+0000 7f6a10a58640 1 --2- 172.21.15.98:0/265578698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a0c15dff0 0x7f6a0c15be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.208+0000 7f6a10a58640 1 -- 172.21.15.98:0/265578698 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0c15c3b0 con 0x7f6a0c0cf740 2024-05-09T10:51:15.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.208+0000 7f6a11259640 1 --2- 172.21.15.98:0/265578698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a0c15dff0 0x7f6a0c15be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:15.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.208+0000 7f6a0bfff640 1 --2- 172.21.15.98:0/265578698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a0c0d1b10 0x7f6a0c1591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:15.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.208+0000 7f6a10a58640 1 --2- 172.21.15.98:0/265578698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a0c0cf740 0x7f6a0c0cb870 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00c820 tx=0x7f69fc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:15.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.208+0000 7f6a097fa640 1 -- 172.21.15.98:0/265578698 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc015070 con 0x7f6a0c0cf740 2024-05-09T10:51:15.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.208+0000 7f6a097fa640 1 -- 172.21.15.98:0/265578698 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69fc004020 con 0x7f6a0c0cf740 2024-05-09T10:51:15.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.208+0000 7f6a12cbc640 1 -- 172.21.15.98:0/265578698 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a0c1f3560 con 0x7f6a0c0cf740 2024-05-09T10:51:15.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.208+0000 7f6a097fa640 1 -- 172.21.15.98:0/265578698 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc0043f0 con 0x7f6a0c0cf740 2024-05-09T10:51:15.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.208+0000 7f6a12cbc640 1 -- 172.21.15.98:0/265578698 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a0c0c7840 con 0x7f6a0c0cf740 2024-05-09T10:51:15.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.209+0000 7f6a12cbc640 1 -- 172.21.15.98:0/265578698 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a0c069cb0 con 0x7f6a0c0cf740 2024-05-09T10:51:15.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.210+0000 7f6a097fa640 1 -- 172.21.15.98:0/265578698 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69fc007830 con 0x7f6a0c0cf740 2024-05-09T10:51:15.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.210+0000 7f6a097fa640 1 --2- 172.21.15.98:0/265578698 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f69f806b4d0 0x7f69f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:15.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.210+0000 7f6a097fa640 1 -- 172.21.15.98:0/265578698 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69fc089f90 con 0x7f6a0c0cf740 2024-05-09T10:51:15.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.210+0000 7f6a0bfff640 1 --2- 172.21.15.98:0/265578698 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f69f806b4d0 0x7f69f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:15.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.210+0000 7f6a0bfff640 1 --2- 172.21.15.98:0/265578698 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f69f806b4d0 0x7f69f806d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c1f7dd0 tx=0x7f6a00008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:15.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.214+0000 7f6a097fa640 1 -- 172.21.15.98:0/265578698 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f69fc058ea0 con 0x7f6a0c0cf740 2024-05-09T10:51:15.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.454+0000 7f6a12cbc640 1 -- 172.21.15.98:0/265578698 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6a0c0b13d0 con 0x7f6a0c0cf740 2024-05-09T10:51:15.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.455+0000 7f6a097fa640 1 -- 172.21.15.98:0/265578698 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+147 (secure 0 0 0) 0x7f69fc0587a0 con 0x7f6a0c0cf740 2024-05-09T10:51:15.449 INFO:teuthology.orchestra.run.smithi098.stdout:[client.0] 2024-05-09T10:51:15.449 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQAxqjxmT+bPOBAA4L/A/EK0lI7oydvPJ27lMA== 2024-05-09T10:51:15.449 INFO:teuthology.orchestra.run.smithi098.stdout: caps mds = "allow" 2024-05-09T10:51:15.449 INFO:teuthology.orchestra.run.smithi098.stdout: caps mon = "allow r" 2024-05-09T10:51:15.449 INFO:teuthology.orchestra.run.smithi098.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-09T10:51:15.449 INFO:teuthology.orchestra.run.smithi098.stderr:updated caps for client.0 2024-05-09T10:51:15.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.457+0000 7f6a12cbc640 1 -- 172.21.15.98:0/265578698 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f69f806b4d0 msgr2=0x7f69f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:15.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.457+0000 7f6a12cbc640 1 --2- 172.21.15.98:0/265578698 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f69f806b4d0 0x7f69f806d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c1f7dd0 tx=0x7f6a00008040 comp rx=0 tx=0).stop 2024-05-09T10:51:15.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.457+0000 7f6a12cbc640 1 -- 172.21.15.98:0/265578698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a0c0cf740 msgr2=0x7f6a0c0cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:15.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.457+0000 7f6a12cbc640 1 --2- 172.21.15.98:0/265578698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a0c0cf740 0x7f6a0c0cb870 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00c820 tx=0x7f69fc00cce0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.458+0000 7f6a12cbc640 1 -- 172.21.15.98:0/265578698 shutdown_connections 2024-05-09T10:51:15.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.458+0000 7f6a12cbc640 1 --2- 172.21.15.98:0/265578698 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f69f806b4d0 0x7f69f806d990 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.458+0000 7f6a12cbc640 1 --2- 172.21.15.98:0/265578698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a0c15dff0 0x7f6a0c15be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.458+0000 7f6a12cbc640 1 --2- 172.21.15.98:0/265578698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6a0c0d1b10 0x7f6a0c1591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.458+0000 7f6a12cbc640 1 --2- 172.21.15.98:0/265578698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a0c0cf740 0x7f6a0c0cb870 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.458+0000 7f6a12cbc640 1 -- 172.21.15.98:0/265578698 >> 172.21.15.98:0/265578698 conn(0x7f6a0c09cec0 msgr2=0x7f6a0c0cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:15.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.458+0000 7f6a12cbc640 1 -- 172.21.15.98:0/265578698 shutdown_connections 2024-05-09T10:51:15.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.458+0000 7f6a12cbc640 1 -- 172.21.15.98:0/265578698 wait complete. 2024-05-09T10:51:15.462 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T10:51:15.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.568+0000 7faa3fa12640 1 Processor -- start 2024-05-09T10:51:15.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.568+0000 7faa3fa12640 1 -- start start 2024-05-09T10:51:15.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.569+0000 7faa3fa12640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7faa380b0c60 0x7faa38166c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:15.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.569+0000 7faa3fa12640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa380aeda0 con 0x7faa380ae7c0 2024-05-09T10:51:15.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.569+0000 7faa3fa12640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa380af080 con 0x7faa3815f4e0 2024-05-09T10:51:15.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.569+0000 7faa3fa12640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa380af360 con 0x7faa380b0c60 2024-05-09T10:51:15.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.569+0000 7faa3cfad640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7faa380ae7c0 0x7faa380b0550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36844/0 (socket says 172.21.15.98:36844) 2024-05-09T10:51:15.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.569+0000 7faa3cfad640 1 -- 172.21.15.98:0/2287572101 learned_addr learned my addr 172.21.15.98:0/2287572101 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:15.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.569+0000 7faa3dfaf640 1 --2- 172.21.15.98:0/2287572101 >> v2:172.21.15.105:3300/0 conn(0x7faa380b0c60 0x7faa38166c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:15.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.569+0000 7faa2ffff640 1 -- 172.21.15.98:0/2287572101 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1956244223 0 0) 0x7faa380aeda0 con 0x7faa380ae7c0 2024-05-09T10:51:15.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.569+0000 7faa2ffff640 1 -- 172.21.15.98:0/2287572101 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa24003520 con 0x7faa380ae7c0 2024-05-09T10:51:15.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.569+0000 7faa2ffff640 1 -- 172.21.15.98:0/2287572101 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443988412 0 0) 0x7faa380af080 con 0x7faa3815f4e0 2024-05-09T10:51:15.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.569+0000 7faa2ffff640 1 -- 172.21.15.98:0/2287572101 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa380aeda0 con 0x7faa3815f4e0 2024-05-09T10:51:15.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.569+0000 7faa2ffff640 1 -- 172.21.15.98:0/2287572101 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2276458533 0 0) 0x7faa24003520 con 0x7faa380ae7c0 2024-05-09T10:51:15.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.569+0000 7faa2ffff640 1 -- 172.21.15.98:0/2287572101 >> v2:172.21.15.105:3300/0 conn(0x7faa380b0c60 msgr2=0x7faa38166c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:15.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.569+0000 7faa2ffff640 1 --2- 172.21.15.98:0/2287572101 >> v2:172.21.15.105:3300/0 conn(0x7faa380b0c60 0x7faa38166c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.569+0000 7faa2ffff640 1 -- 172.21.15.98:0/2287572101 >> v1:172.21.15.105:6789/0 conn(0x7faa3815f4e0 legacy=0x7faa380afe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:15.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.569+0000 7faa2ffff640 1 -- 172.21.15.98:0/2287572101 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa380af640 con 0x7faa380ae7c0 2024-05-09T10:51:15.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.569+0000 7faa2ffff640 1 -- 172.21.15.98:0/2287572101 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7faa28001670 con 0x7faa380ae7c0 2024-05-09T10:51:15.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.570+0000 7faa2ffff640 1 -- 172.21.15.98:0/2287572101 >> v1:172.21.15.98:6789/0 conn(0x7faa380ae7c0 legacy=0x7faa380b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:15.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.570+0000 7faa2ffff640 1 --2- 172.21.15.98:0/2287572101 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa24003b40 0x7faa24003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:15.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.570+0000 7faa2ffff640 1 --2- 172.21.15.98:0/2287572101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa24004800 0x7faa24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:15.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.570+0000 7faa2ffff640 1 --2- 172.21.15.98:0/2287572101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa240051c0 0x7faa2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:15.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.570+0000 7faa2ffff640 1 -- 172.21.15.98:0/2287572101 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa380af360 con 0x7faa240051c0 2024-05-09T10:51:15.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.570+0000 7faa2ffff640 1 -- 172.21.15.98:0/2287572101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2400c0f0 con 0x7faa24004800 2024-05-09T10:51:15.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.570+0000 7faa2ffff640 1 -- 172.21.15.98:0/2287572101 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2400c2f0 con 0x7faa24003b40 2024-05-09T10:51:15.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.570+0000 7faa3dfaf640 1 --2- 172.21.15.98:0/2287572101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa240051c0 0x7faa2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:15.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.570+0000 7faa3cfad640 1 --2- 172.21.15.98:0/2287572101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa24004800 0x7faa24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:15.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.570+0000 7faa3d7ae640 1 --2- 172.21.15.98:0/2287572101 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa24003b40 0x7faa24003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:15.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.570+0000 7faa3dfaf640 1 -- 172.21.15.98:0/2287572101 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa24003b40 msgr2=0x7faa24003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:15.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.570+0000 7faa3dfaf640 1 --2- 172.21.15.98:0/2287572101 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa24003b40 0x7faa24003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.570+0000 7faa3dfaf640 1 -- 172.21.15.98:0/2287572101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa24004800 msgr2=0x7faa24004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:15.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.570+0000 7faa3dfaf640 1 --2- 172.21.15.98:0/2287572101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa24004800 0x7faa24004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.570+0000 7faa3dfaf640 1 -- 172.21.15.98:0/2287572101 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa24003520 con 0x7faa240051c0 2024-05-09T10:51:15.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.571+0000 7faa3dfaf640 1 -- 172.21.15.98:0/2287572101 reap_dead start 2024-05-09T10:51:15.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.571+0000 7faa3dfaf640 1 --2- 172.21.15.98:0/2287572101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa240051c0 0x7faa2400b9c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7faa30009fc0 tx=0x7faa24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638ab3f8c031ddae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:15.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.571+0000 7faa2ffff640 1 -- 172.21.15.98:0/2287572101 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa30035020 con 0x7faa240051c0 2024-05-09T10:51:15.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.571+0000 7faa2ffff640 1 -- 172.21.15.98:0/2287572101 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa30003bd0 con 0x7faa240051c0 2024-05-09T10:51:15.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.571+0000 7faa2ffff640 1 -- 172.21.15.98:0/2287572101 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa30009c10 con 0x7faa240051c0 2024-05-09T10:51:15.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.571+0000 7faa3fa12640 1 -- 172.21.15.98:0/2287572101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa240051c0 msgr2=0x7faa2400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:15.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.571+0000 7faa3fa12640 1 --2- 172.21.15.98:0/2287572101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa240051c0 0x7faa2400b9c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7faa30009fc0 tx=0x7faa24005830 comp rx=0 tx=0).stop 2024-05-09T10:51:15.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.571+0000 7faa3fa12640 1 -- 172.21.15.98:0/2287572101 shutdown_connections 2024-05-09T10:51:15.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.571+0000 7faa3fa12640 1 --2- 172.21.15.98:0/2287572101 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa240051c0 0x7faa2400b9c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.571+0000 7faa3fa12640 1 -- 172.21.15.98:0/2287572101 >> 172.21.15.98:0/2287572101 conn(0x7faa380b34e0 msgr2=0x7faa380b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:15.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.571+0000 7faa3fa12640 1 -- 172.21.15.98:0/2287572101 shutdown_connections 2024-05-09T10:51:15.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.571+0000 7faa3fa12640 1 -- 172.21.15.98:0/2287572101 wait complete. 2024-05-09T10:51:15.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.573+0000 7faa3fa12640 1 Processor -- start 2024-05-09T10:51:15.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.573+0000 7faa3fa12640 1 -- start start 2024-05-09T10:51:15.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.573+0000 7faa3fa12640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa240051c0 0x7faa381536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:15.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.573+0000 7faa3fa12640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa38154df0 0x7faa380cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:15.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.573+0000 7faa3fa12640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa380c7550 0x7faa380c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:15.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.573+0000 7faa3fa12640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa380cbcf0 con 0x7faa240051c0 2024-05-09T10:51:15.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.573+0000 7faa3fa12640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa380b1070 con 0x7faa38154df0 2024-05-09T10:51:15.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.573+0000 7faa3fa12640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3815f8c0 con 0x7faa380c7550 2024-05-09T10:51:15.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.573+0000 7faa3d7ae640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa240051c0 0x7faa381536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:15.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.573+0000 7faa3d7ae640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa240051c0 0x7faa381536c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57504/0 (socket says 172.21.15.98:57504) 2024-05-09T10:51:15.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.573+0000 7faa3d7ae640 1 -- 172.21.15.98:0/2477461959 learned_addr learned my addr 172.21.15.98:0/2477461959 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:15.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.574+0000 7faa3d7ae640 1 -- 172.21.15.98:0/2477461959 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa380c7550 msgr2=0x7faa380c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:15.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.574+0000 7faa3d7ae640 1 --2- 172.21.15.98:0/2477461959 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa380c7550 0x7faa380c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.574+0000 7faa3d7ae640 1 -- 172.21.15.98:0/2477461959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa38154df0 msgr2=0x7faa380cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:15.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.574+0000 7faa3cfad640 1 --2- 172.21.15.98:0/2477461959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa38154df0 0x7faa380cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:15.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.574+0000 7faa3d7ae640 1 --2- 172.21.15.98:0/2477461959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa38154df0 0x7faa380cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.574+0000 7faa3d7ae640 1 -- 172.21.15.98:0/2477461959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa38153d50 con 0x7faa240051c0 2024-05-09T10:51:15.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.574+0000 7faa3d7ae640 1 --2- 172.21.15.98:0/2477461959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa240051c0 0x7faa381536c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7faa3400f7d0 tx=0x7faa3400fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:15.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.574+0000 7faa2dffb640 1 -- 172.21.15.98:0/2477461959 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3401a070 con 0x7faa240051c0 2024-05-09T10:51:15.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.574+0000 7faa2dffb640 1 -- 172.21.15.98:0/2477461959 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3400fe80 con 0x7faa240051c0 2024-05-09T10:51:15.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.574+0000 7faa3fa12640 1 -- 172.21.15.98:0/2477461959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa38154030 con 0x7faa240051c0 2024-05-09T10:51:15.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.574+0000 7faa2dffb640 1 -- 172.21.15.98:0/2477461959 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa34007050 con 0x7faa240051c0 2024-05-09T10:51:15.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.574+0000 7faa3fa12640 1 -- 172.21.15.98:0/2477461959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa381545f0 con 0x7faa240051c0 2024-05-09T10:51:15.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.575+0000 7faa3fa12640 1 -- 172.21.15.98:0/2477461959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa381551d0 con 0x7faa240051c0 2024-05-09T10:51:15.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.576+0000 7faa2dffb640 1 -- 172.21.15.98:0/2477461959 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa3401d070 con 0x7faa240051c0 2024-05-09T10:51:15.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.576+0000 7faa2dffb640 1 --2- 172.21.15.98:0/2477461959 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7faa1c06b4d0 0x7faa1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:15.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.576+0000 7faa2dffb640 1 -- 172.21.15.98:0/2477461959 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa34089990 con 0x7faa240051c0 2024-05-09T10:51:15.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.576+0000 7faa3cfad640 1 --2- 172.21.15.98:0/2477461959 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7faa1c06b4d0 0x7faa1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:15.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.577+0000 7faa3cfad640 1 --2- 172.21.15.98:0/2477461959 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7faa1c06b4d0 0x7faa1c06d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faa38155a70 tx=0x7faa28009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:15.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.580+0000 7faa2dffb640 1 -- 172.21.15.98:0/2477461959 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faa34058810 con 0x7faa240051c0 2024-05-09T10:51:15.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.784+0000 7faa3fa12640 1 -- 172.21.15.98:0/2477461959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa38074480 con 0x7faa240051c0 2024-05-09T10:51:15.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.785+0000 7faa2dffb640 1 -- 172.21.15.98:0/2477461959 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4941 (secure 0 0 0) 0x7faa34058110 con 0x7faa240051c0 2024-05-09T10:51:15.778 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:15.778 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2332373437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2332373437},{"type":"v1","addr":"172.21.15.105:6834","nonce":2332373437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4849,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/4092257628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":4092257628},{"type":"v1","addr":"172.21.15.98:6836","nonce":4092257628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4855,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3957569958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3957569958},{"type":"v1","addr":"172.21.15.105:6835","nonce":3957569958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T10:51:06.745057+0000","modified":"2024-05-09T10:51:08.738161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5127":{"gid":5127,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/164638690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":164638690},{"type":"v1","addr":"172.21.15.98:6837","nonce":164638690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-09T10:51:15.779 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 13 2024-05-09T10:51:15.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.787+0000 7faa3fa12640 1 -- 172.21.15.98:0/2477461959 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7faa1c06b4d0 msgr2=0x7faa1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:15.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.787+0000 7faa3fa12640 1 --2- 172.21.15.98:0/2477461959 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7faa1c06b4d0 0x7faa1c06d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faa38155a70 tx=0x7faa28009040 comp rx=0 tx=0).stop 2024-05-09T10:51:15.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.787+0000 7faa3fa12640 1 -- 172.21.15.98:0/2477461959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa240051c0 msgr2=0x7faa381536c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:15.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.787+0000 7faa3fa12640 1 --2- 172.21.15.98:0/2477461959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa240051c0 0x7faa381536c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7faa3400f7d0 tx=0x7faa3400fc90 comp rx=0 tx=0).stop 2024-05-09T10:51:15.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.787+0000 7faa3fa12640 1 -- 172.21.15.98:0/2477461959 shutdown_connections 2024-05-09T10:51:15.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.787+0000 7faa3fa12640 1 --2- 172.21.15.98:0/2477461959 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7faa1c06b4d0 0x7faa1c06d990 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.787+0000 7faa3fa12640 1 --2- 172.21.15.98:0/2477461959 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faa380c7550 0x7faa380c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.787+0000 7faa3fa12640 1 --2- 172.21.15.98:0/2477461959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faa38154df0 0x7faa380cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.787+0000 7faa3fa12640 1 --2- 172.21.15.98:0/2477461959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa240051c0 0x7faa381536c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:15.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.787+0000 7faa3fa12640 1 -- 172.21.15.98:0/2477461959 >> 172.21.15.98:0/2477461959 conn(0x7faa380b34e0 msgr2=0x7faa38152c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:15.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.787+0000 7faa3fa12640 1 -- 172.21.15.98:0/2477461959 shutdown_connections 2024-05-09T10:51:15.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:15.787+0000 7faa3fa12640 1 -- 172.21.15.98:0/2477461959 wait complete. 2024-05-09T10:51:16.194 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T10:51:06.745057+0000', 'modified': '2024-05-09T10:51:08.738161+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5127}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5127': {'gid': 5127, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6837/164638690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6835', 'nonce': 164638690}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 164638690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-09T10:51:16.194 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T10:51:16.194 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-09T10:51:16.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.304+0000 7efeb804a640 1 Processor -- start 2024-05-09T10:51:16.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.304+0000 7efeb804a640 1 -- start start 2024-05-09T10:51:16.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.305+0000 7efeb804a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efeb015e0c0 0x7efeb006bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.305+0000 7efeb804a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb00d0c70 con 0x7efeb006cac0 2024-05-09T10:51:16.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.305+0000 7efeb804a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb00d0f50 con 0x7efeb015dce0 2024-05-09T10:51:16.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.305+0000 7efeb804a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00b4350 con 0x7efeb015e0c0 2024-05-09T10:51:16.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.305+0000 7efeb55e5640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7efeb006cac0 0x7efeb00b1ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36846/0 (socket says 172.21.15.98:36846) 2024-05-09T10:51:16.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.305+0000 7efeb55e5640 1 -- 172.21.15.98:0/751358309 learned_addr learned my addr 172.21.15.98:0/751358309 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:16.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb65e7640 1 --2- 172.21.15.98:0/751358309 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efeb015e0c0 0x7efeb006bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb4de4640 1 -- 172.21.15.98:0/751358309 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 879110037 0 0) 0x7efeb00d0c70 con 0x7efeb006cac0 2024-05-09T10:51:16.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb4de4640 1 -- 172.21.15.98:0/751358309 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe9c003520 con 0x7efeb006cac0 2024-05-09T10:51:16.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb4de4640 1 -- 172.21.15.98:0/751358309 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 419636984 0 0) 0x7efe9c003520 con 0x7efeb006cac0 2024-05-09T10:51:16.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb4de4640 1 -- 172.21.15.98:0/751358309 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efeb015e0c0 msgr2=0x7efeb006bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:16.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb4de4640 1 --2- 172.21.15.98:0/751358309 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efeb015e0c0 0x7efeb006bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb4de4640 1 -- 172.21.15.98:0/751358309 >> v1:172.21.15.105:6789/0 conn(0x7efeb015dce0 legacy=0x7efeb006c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:16.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb4de4640 1 -- 172.21.15.98:0/751358309 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb00b0fd0 con 0x7efeb006cac0 2024-05-09T10:51:16.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb4de4640 1 -- 172.21.15.98:0/751358309 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7efea0001670 con 0x7efeb006cac0 2024-05-09T10:51:16.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb4de4640 1 -- 172.21.15.98:0/751358309 >> v1:172.21.15.98:6789/0 conn(0x7efeb006cac0 legacy=0x7efeb00b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:16.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb4de4640 1 --2- 172.21.15.98:0/751358309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe9c0039e0 0x7efe9c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb4de4640 1 --2- 172.21.15.98:0/751358309 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efeb015e0c0 0x7efe9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb4de4640 1 --2- 172.21.15.98:0/751358309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe9c004d40 0x7efe9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb4de4640 1 -- 172.21.15.98:0/751358309 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c00bb00 con 0x7efe9c0039e0 2024-05-09T10:51:16.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb4de4640 1 -- 172.21.15.98:0/751358309 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c00bc80 con 0x7efe9c004d40 2024-05-09T10:51:16.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb4de4640 1 -- 172.21.15.98:0/751358309 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c00be80 con 0x7efeb015e0c0 2024-05-09T10:51:16.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb5de6640 1 --2- 172.21.15.98:0/751358309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe9c0039e0 0x7efe9c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb55e5640 1 --2- 172.21.15.98:0/751358309 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efeb015e0c0 0x7efe9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb65e7640 1 --2- 172.21.15.98:0/751358309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe9c004d40 0x7efe9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb5de6640 1 -- 172.21.15.98:0/751358309 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efeb015e0c0 msgr2=0x7efe9c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:16.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb5de6640 1 --2- 172.21.15.98:0/751358309 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efeb015e0c0 0x7efe9c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb5de6640 1 -- 172.21.15.98:0/751358309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe9c004d40 msgr2=0x7efe9c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:16.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb5de6640 1 --2- 172.21.15.98:0/751358309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe9c004d40 0x7efe9c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.306+0000 7efeb5de6640 1 -- 172.21.15.98:0/751358309 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9c003520 con 0x7efe9c0039e0 2024-05-09T10:51:16.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.307+0000 7efeb5de6640 1 --2- 172.21.15.98:0/751358309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe9c0039e0 0x7efe9c003e00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efeac001e20 tx=0x7efe9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc46c75f24880d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:16.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.307+0000 7efeb4de4640 1 -- 172.21.15.98:0/751358309 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeac001a70 con 0x7efe9c0039e0 2024-05-09T10:51:16.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.307+0000 7efeb4de4640 1 -- 172.21.15.98:0/751358309 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeac02ae40 con 0x7efe9c0039e0 2024-05-09T10:51:16.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.307+0000 7efeb4de4640 1 -- 172.21.15.98:0/751358309 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeac003bb0 con 0x7efe9c0039e0 2024-05-09T10:51:16.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.308+0000 7efeb804a640 1 -- 172.21.15.98:0/751358309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe9c0039e0 msgr2=0x7efe9c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:16.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.308+0000 7efeb804a640 1 --2- 172.21.15.98:0/751358309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe9c0039e0 0x7efe9c003e00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efeac001e20 tx=0x7efe9c005830 comp rx=0 tx=0).stop 2024-05-09T10:51:16.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.308+0000 7efeb65e7640 1 -- 172.21.15.98:0/751358309 reap_dead start 2024-05-09T10:51:16.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.308+0000 7efeb804a640 1 -- 172.21.15.98:0/751358309 shutdown_connections 2024-05-09T10:51:16.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.308+0000 7efeb804a640 1 -- 172.21.15.98:0/751358309 >> 172.21.15.98:0/751358309 conn(0x7efeb00b34e0 msgr2=0x7efeb00b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:16.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.308+0000 7efeb804a640 1 -- 172.21.15.98:0/751358309 shutdown_connections 2024-05-09T10:51:16.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.308+0000 7efeb804a640 1 -- 172.21.15.98:0/751358309 wait complete. 2024-05-09T10:51:16.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.309+0000 7efeb804a640 1 Processor -- start 2024-05-09T10:51:16.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.309+0000 7efeb804a640 1 -- start start 2024-05-09T10:51:16.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.310+0000 7efeb804a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe9c0039e0 0x7efeb01470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.310+0000 7efeb804a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efeb00c70e0 0x7efeb0156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.310+0000 7efeb804a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efeb015b6c0 0x7efeb015bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.310+0000 7efeb804a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00d0700 con 0x7efe9c0039e0 2024-05-09T10:51:16.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.310+0000 7efeb804a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00b1590 con 0x7efeb015b6c0 2024-05-09T10:51:16.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.310+0000 7efeb804a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00d0880 con 0x7efeb00c70e0 2024-05-09T10:51:16.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.310+0000 7efeb5de6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe9c0039e0 0x7efeb01470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.310+0000 7efeb5de6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe9c0039e0 0x7efeb01470d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57530/0 (socket says 172.21.15.98:57530) 2024-05-09T10:51:16.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.310+0000 7efeb5de6640 1 -- 172.21.15.98:0/1131883942 learned_addr learned my addr 172.21.15.98:0/1131883942 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:16.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.310+0000 7efeb55e5640 1 --2- 172.21.15.98:0/1131883942 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efeb00c70e0 0x7efeb0156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.310+0000 7efeb65e7640 1 --2- 172.21.15.98:0/1131883942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efeb015b6c0 0x7efeb015bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.310+0000 7efeb5de6640 1 -- 172.21.15.98:0/1131883942 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efeb00c70e0 msgr2=0x7efeb0156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:16.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.310+0000 7efeb5de6640 1 --2- 172.21.15.98:0/1131883942 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efeb00c70e0 0x7efeb0156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.310+0000 7efeb5de6640 1 -- 172.21.15.98:0/1131883942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efeb015b6c0 msgr2=0x7efeb015bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:16.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.310+0000 7efeb5de6640 1 --2- 172.21.15.98:0/1131883942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efeb015b6c0 0x7efeb015bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.310+0000 7efeb5de6640 1 -- 172.21.15.98:0/1131883942 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb01574a0 con 0x7efe9c0039e0 2024-05-09T10:51:16.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.310+0000 7efeb5de6640 1 --2- 172.21.15.98:0/1131883942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe9c0039e0 0x7efeb01470d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7efeac001f40 tx=0x7efeac0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:16.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.311+0000 7efea67fc640 1 -- 172.21.15.98:0/1131883942 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeac037040 con 0x7efe9c0039e0 2024-05-09T10:51:16.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.311+0000 7efea67fc640 1 -- 172.21.15.98:0/1131883942 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeac041070 con 0x7efe9c0039e0 2024-05-09T10:51:16.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.311+0000 7efeb804a640 1 -- 172.21.15.98:0/1131883942 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb00c7560 con 0x7efe9c0039e0 2024-05-09T10:51:16.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.311+0000 7efea67fc640 1 -- 172.21.15.98:0/1131883942 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeac0052f0 con 0x7efe9c0039e0 2024-05-09T10:51:16.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.311+0000 7efeb804a640 1 -- 172.21.15.98:0/1131883942 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb00c7af0 con 0x7efe9c0039e0 2024-05-09T10:51:16.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.312+0000 7efea67fc640 1 -- 172.21.15.98:0/1131883942 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efeac005490 con 0x7efe9c0039e0 2024-05-09T10:51:16.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.313+0000 7efea67fc640 1 --2- 172.21.15.98:0/1131883942 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7efe9406b4d0 0x7efe9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.313+0000 7efeb55e5640 1 --2- 172.21.15.98:0/1131883942 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7efe9406b4d0 0x7efe9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.313+0000 7efea67fc640 1 -- 172.21.15.98:0/1131883942 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efeac0aa310 con 0x7efe9c0039e0 2024-05-09T10:51:16.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.313+0000 7efeb55e5640 1 --2- 172.21.15.98:0/1131883942 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7efe9406b4d0 0x7efe9406d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efeb00cf3f0 tx=0x7efea00078f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:16.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.314+0000 7efebae26740 1 Processor -- start 2024-05-09T10:51:16.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.314+0000 7efebae26740 1 -- start start 2024-05-09T10:51:16.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.315+0000 7efebae26740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d234d50230 0x55d234d50610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.315+0000 7efebae26740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d6dad0 0x55d234d6e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.315+0000 7efebae26740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55d234d25f10 0x55d234d6c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.315+0000 7efebae26740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d234c88a10 con 0x55d234d6dad0 2024-05-09T10:51:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.315+0000 7efebae26740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d234c31b30 con 0x55d234d50230 2024-05-09T10:51:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.315+0000 7efebae26740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d234c84c50 con 0x55d234d25f10 2024-05-09T10:51:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.315+0000 7efeb55e5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d6dad0 0x55d234d6e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.315+0000 7efeb55e5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d6dad0 0x55d234d6e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57546/0 (socket says 172.21.15.98:57546) 2024-05-09T10:51:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.315+0000 7efeb55e5640 1 -- 172.21.15.98:0/4220812441 learned_addr learned my addr 172.21.15.98:0/4220812441 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.315+0000 7efeb65e7640 1 --2- 172.21.15.98:0/4220812441 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55d234d25f10 0x55d234d6c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.315+0000 7efeb5de6640 1 --2- 172.21.15.98:0/4220812441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d234d50230 0x55d234d50610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.315+0000 7efeb55e5640 1 -- 172.21.15.98:0/4220812441 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55d234d25f10 msgr2=0x55d234d6c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.315+0000 7efeb55e5640 1 --2- 172.21.15.98:0/4220812441 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55d234d25f10 0x55d234d6c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.315+0000 7efeb55e5640 1 -- 172.21.15.98:0/4220812441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d234d50230 msgr2=0x55d234d50610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.315+0000 7efeb55e5640 1 --2- 172.21.15.98:0/4220812441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d234d50230 0x55d234d50610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.315+0000 7efeb55e5640 1 -- 172.21.15.98:0/4220812441 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d234c46e10 con 0x55d234d6dad0 2024-05-09T10:51:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.315+0000 7efeb5de6640 1 --2- 172.21.15.98:0/4220812441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d234d50230 0x55d234d50610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.315+0000 7efeb65e7640 1 --2- 172.21.15.98:0/4220812441 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55d234d25f10 0x55d234d6c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.315+0000 7efeb55e5640 1 --2- 172.21.15.98:0/4220812441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d6dad0 0x55d234d6e020 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efea0001820 tx=0x7efea000c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c065808b8e21e0aa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.315+0000 7efe937fe640 1 -- 172.21.15.98:0/4220812441 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0028070 con 0x55d234d6dad0 2024-05-09T10:51:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.315+0000 7efe937fe640 1 -- 172.21.15.98:0/4220812441 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea0015bd0 con 0x55d234d6dad0 2024-05-09T10:51:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.315+0000 7efe937fe640 1 -- 172.21.15.98:0/4220812441 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea00213c0 con 0x55d234d6dad0 2024-05-09T10:51:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.316+0000 7efebae26740 1 -- 172.21.15.98:0/4220812441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d6dad0 msgr2=0x55d234d6e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.316+0000 7efebae26740 1 --2- 172.21.15.98:0/4220812441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d6dad0 0x55d234d6e020 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efea0001820 tx=0x7efea000c000 comp rx=0 tx=0).stop 2024-05-09T10:51:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.316+0000 7efebae26740 1 -- 172.21.15.98:0/4220812441 shutdown_connections 2024-05-09T10:51:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.316+0000 7efebae26740 1 --2- 172.21.15.98:0/4220812441 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55d234d25f10 0x55d234d6c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.316+0000 7efebae26740 1 --2- 172.21.15.98:0/4220812441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d6dad0 0x55d234d6e020 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.316+0000 7efebae26740 1 --2- 172.21.15.98:0/4220812441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d234d50230 0x55d234d50610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.316+0000 7efebae26740 1 -- 172.21.15.98:0/4220812441 >> 172.21.15.98:0/4220812441 conn(0x55d234d22840 msgr2=0x55d234d6b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.316+0000 7efebae26740 1 -- 172.21.15.98:0/4220812441 shutdown_connections 2024-05-09T10:51:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.316+0000 7efebae26740 1 -- 172.21.15.98:0/4220812441 wait complete. 2024-05-09T10:51:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.317+0000 7efebae26740 1 -- start start 2024-05-09T10:51:16.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.317+0000 7efebae26740 1 Processor -- start 2024-05-09T10:51:16.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.317+0000 7efebae26740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d50230 0x55d234d4c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.317+0000 7efebae26740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55d234d6dad0 0x55d234d61bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efebae26740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d234d4e7b0 0x55d234d4ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efebae26740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d234d26cb0 con 0x55d234d50230 2024-05-09T10:51:16.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efebae26740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d234c450c0 con 0x55d234d4e7b0 2024-05-09T10:51:16.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efebae26740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d234d20770 con 0x55d234d6dad0 2024-05-09T10:51:16.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efebae26740 10 client.0 _ll_register_callbacks cb 0x55d234c38ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T10:51:16.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efeb5de6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d50230 0x55d234d4c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efeb5de6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d50230 0x55d234d4c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57552/0 (socket says 172.21.15.98:57552) 2024-05-09T10:51:16.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efeb5de6640 1 -- 172.21.15.98:0/252055355 learned_addr learned my addr 172.21.15.98:0/252055355 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:16.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efeb55e5640 1 --2- 172.21.15.98:0/252055355 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55d234d6dad0 0x55d234d61bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efeb65e7640 1 --2- 172.21.15.98:0/252055355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d234d4e7b0 0x55d234d4ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efeb5de6640 1 -- 172.21.15.98:0/252055355 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55d234d6dad0 msgr2=0x55d234d61bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:16.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efeb5de6640 1 --2- 172.21.15.98:0/252055355 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55d234d6dad0 0x55d234d61bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efeb5de6640 1 -- 172.21.15.98:0/252055355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d234d4e7b0 msgr2=0x55d234d4ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:16.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efeb5de6640 1 --2- 172.21.15.98:0/252055355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d234d4e7b0 0x55d234d4ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efeb5de6640 1 -- 172.21.15.98:0/252055355 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d23499d2d0 con 0x55d234d50230 2024-05-09T10:51:16.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efeb65e7640 1 --2- 172.21.15.98:0/252055355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d234d4e7b0 0x55d234d4ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:16.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efeb5de6640 1 --2- 172.21.15.98:0/252055355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d50230 0x55d234d4c480 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efeac074ef0 tx=0x7efeac078a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a985b73a6d82f0f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:16.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efe90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-05-09T10:51:16.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efe90ff9640 1 -- 172.21.15.98:0/252055355 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeac076d90 con 0x55d234d50230 2024-05-09T10:51:16.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efe90ff9640 1 -- 172.21.15.98:0/252055355 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efeac001a70 con 0x55d234d50230 2024-05-09T10:51:16.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efebae26740 1 -- 172.21.15.98:0/252055355 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efea0015bd0 con 0x55d234d50230 2024-05-09T10:51:16.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efe90ff9640 1 -- 172.21.15.98:0/252055355 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7efeac077ee0 con 0x55d234d50230 2024-05-09T10:51:16.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efebae26740 10 client.5331 fetch_fsmap learned FSMap version 13 2024-05-09T10:51:16.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efebae26740 1 -- 172.21.15.98:0/252055355 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x55d234a676b0 con 0x55d234d50230 2024-05-09T10:51:16.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.318+0000 7efe90ff9640 1 -- 172.21.15.98:0/252055355 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea0015bd0 con 0x55d234d50230 2024-05-09T10:51:16.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.319+0000 7efe90ff9640 1 -- 172.21.15.98:0/252055355 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 13) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efeac078080 con 0x55d234d50230 2024-05-09T10:51:16.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.319+0000 7efebae26740 10 client.5331 fetch_fsmap finished waiting for FSMap version 13 2024-05-09T10:51:16.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.319+0000 7efebae26740 10 client.5331 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T10:51:16.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.319+0000 7efebae26740 20 client.5331 populate_metadata read hostname 'smithi098' 2024-05-09T10:51:16.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.319+0000 7efebae26740 1 --2- 172.21.15.98:0/252055355 >> [v2:172.21.15.98:6835/164638690,v1:172.21.15.98:6837/164638690] conn(0x55d234d513c0 0x55d234d517c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.319+0000 7efebae26740 4 client.5331 mds_command: new command op to 5127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T10:51:16.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.319+0000 7efebae26740 1 -- 172.21.15.98:0/252055355 --> [v2:172.21.15.98:6835/164638690,v1:172.21.15.98:6837/164638690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d234ce7920 con 0x55d234d513c0 2024-05-09T10:51:16.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.319+0000 7efeb55e5640 1 --2- 172.21.15.98:0/252055355 >> [v2:172.21.15.98:6835/164638690,v1:172.21.15.98:6837/164638690] conn(0x55d234d513c0 0x55d234d517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.319+0000 7efe90ff9640 10 client.5331 ms_handle_connect on v2:172.21.15.98:6835/164638690 2024-05-09T10:51:16.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.319+0000 7efeb55e5640 1 --2- 172.21.15.98:0/252055355 >> [v2:172.21.15.98:6835/164638690,v1:172.21.15.98:6837/164638690] conn(0x55d234d513c0 0x55d234d517c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45199adb93a6636d server_cookie=86b126ed96231edb in_seq=0 out_seq=0 2024-05-09T10:51:16.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.320+0000 7efe90ff9640 1 -- 172.21.15.98:0/252055355 <== mds.0 v2:172.21.15.98:6835/164638690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d234ce7920 con 0x55d234d513c0 2024-05-09T10:51:16.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.320+0000 7efe90ff9640 10 client.5331 handle_command_reply: tid=0 2024-05-09T10:51:16.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.320+0000 7efebae26740 1 client.5331 shutdown 2024-05-09T10:51:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.320+0000 7efebae26740 1 -- 172.21.15.98:0/252055355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d50230 msgr2=0x55d234d4c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.320+0000 7efebae26740 1 --2- 172.21.15.98:0/252055355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d50230 0x55d234d4c480 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efeac074ef0 tx=0x7efeac078a90 comp rx=0 tx=0).stop 2024-05-09T10:51:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.320+0000 7efebae26740 1 -- 172.21.15.98:0/252055355 shutdown_connections 2024-05-09T10:51:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.320+0000 7efebae26740 1 --2- 172.21.15.98:0/252055355 >> [v2:172.21.15.98:6835/164638690,v1:172.21.15.98:6837/164638690] conn(0x55d234d513c0 0x55d234d517c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.320+0000 7efebae26740 1 --2- 172.21.15.98:0/252055355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d234d4e7b0 0x55d234d4ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.320+0000 7efebae26740 1 --2- 172.21.15.98:0/252055355 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55d234d6dad0 0x55d234d61bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.320+0000 7efebae26740 1 --2- 172.21.15.98:0/252055355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d50230 0x55d234d4c480 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.320+0000 7efe90ff9640 0 client.5331 ms_handle_reset on v2:172.21.15.98:6835/164638690 2024-05-09T10:51:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.320+0000 7efebae26740 1 -- 172.21.15.98:0/252055355 >> 172.21.15.98:0/252055355 conn(0x55d234d25f10 msgr2=0x55d234d6e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.320+0000 7efebae26740 1 -- 172.21.15.98:0/252055355 shutdown_connections 2024-05-09T10:51:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.320+0000 7efebae26740 1 -- 172.21.15.98:0/252055355 wait complete. 2024-05-09T10:51:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.320+0000 7efebae26740 20 client.5331 trim_cache size 0 max 16384 2024-05-09T10:51:16.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.336+0000 7efebae26740 1 Processor -- start 2024-05-09T10:51:16.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.336+0000 7efebae26740 1 -- start start 2024-05-09T10:51:16.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.336+0000 7efebae26740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d4e7b0 0x55d234d4f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.336+0000 7efebae26740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55d234d50230 0x55d234d6a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.336+0000 7efebae26740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d234d6dad0 0x55d234d517a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.336+0000 7efebae26740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb0200570 con 0x55d234d4e7b0 2024-05-09T10:51:16.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.336+0000 7efebae26740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb01ff850 con 0x55d234d6dad0 2024-05-09T10:51:16.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.336+0000 7efebae26740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb01455c0 con 0x55d234d50230 2024-05-09T10:51:16.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.336+0000 7efeb5de6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d4e7b0 0x55d234d4f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.336+0000 7efeb5de6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d4e7b0 0x55d234d4f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57554/0 (socket says 172.21.15.98:57554) 2024-05-09T10:51:16.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.336+0000 7efeb5de6640 1 -- 172.21.15.98:0/2487888496 learned_addr learned my addr 172.21.15.98:0/2487888496 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:16.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.337+0000 7efeb5de6640 1 -- 172.21.15.98:0/2487888496 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55d234d50230 msgr2=0x55d234d6a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:16.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.337+0000 7efeb5de6640 1 --2- 172.21.15.98:0/2487888496 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55d234d50230 0x55d234d6a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.337+0000 7efeb5de6640 1 -- 172.21.15.98:0/2487888496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d234d6dad0 msgr2=0x55d234d517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:16.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.337+0000 7efeb5de6640 1 --2- 172.21.15.98:0/2487888496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d234d6dad0 0x55d234d517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.337+0000 7efeb5de6640 1 -- 172.21.15.98:0/2487888496 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d234d4f290 con 0x55d234d4e7b0 2024-05-09T10:51:16.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.337+0000 7efeb5de6640 1 --2- 172.21.15.98:0/2487888496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d4e7b0 0x55d234d4f790 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7efeac076d40 tx=0x7efeac07a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30705e2c80acb359 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:16.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.337+0000 7efe92ffd640 1 -- 172.21.15.98:0/2487888496 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeac0783f0 con 0x55d234d4e7b0 2024-05-09T10:51:16.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.337+0000 7efe92ffd640 1 -- 172.21.15.98:0/2487888496 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeac076d90 con 0x55d234d4e7b0 2024-05-09T10:51:16.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.337+0000 7efe92ffd640 1 -- 172.21.15.98:0/2487888496 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeac078080 con 0x55d234d4e7b0 2024-05-09T10:51:16.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.337+0000 7efebae26740 1 -- 172.21.15.98:0/2487888496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d4e7b0 msgr2=0x55d234d4f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:16.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.337+0000 7efebae26740 1 --2- 172.21.15.98:0/2487888496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d4e7b0 0x55d234d4f790 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7efeac076d40 tx=0x7efeac07a000 comp rx=0 tx=0).stop 2024-05-09T10:51:16.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.338+0000 7efebae26740 1 -- 172.21.15.98:0/2487888496 shutdown_connections 2024-05-09T10:51:16.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.338+0000 7efebae26740 1 --2- 172.21.15.98:0/2487888496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d234d6dad0 0x55d234d517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.338+0000 7efebae26740 1 --2- 172.21.15.98:0/2487888496 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55d234d50230 0x55d234d6a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.338+0000 7efebae26740 1 --2- 172.21.15.98:0/2487888496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d4e7b0 0x55d234d4f790 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.338+0000 7efebae26740 1 -- 172.21.15.98:0/2487888496 >> 172.21.15.98:0/2487888496 conn(0x55d234d25f10 msgr2=0x55d234d4a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:16.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.338+0000 7efebae26740 1 -- 172.21.15.98:0/2487888496 shutdown_connections 2024-05-09T10:51:16.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.338+0000 7efebae26740 1 -- 172.21.15.98:0/2487888496 wait complete. 2024-05-09T10:51:16.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.338+0000 7efebae26740 1 -- start start 2024-05-09T10:51:16.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.338+0000 7efebae26740 1 Processor -- start 2024-05-09T10:51:16.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.339+0000 7efebae26740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d4e7b0 0x55d234dd5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.339+0000 7efebae26740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55d234d50230 0x55d234d6e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.339+0000 7efebae26740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d234d6dad0 0x55d234d6ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.339+0000 7efebae26740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d234d52aa0 con 0x55d234d4e7b0 2024-05-09T10:51:16.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.339+0000 7efebae26740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d234d26cb0 con 0x55d234d6dad0 2024-05-09T10:51:16.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.339+0000 7efebae26740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d234d532c0 con 0x55d234d50230 2024-05-09T10:51:16.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.339+0000 7efebae26740 10 client.0 _ll_register_callbacks cb 0x55d234d20c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T10:51:16.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.339+0000 7efeb5de6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d4e7b0 0x55d234dd5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.339+0000 7efeb5de6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d4e7b0 0x55d234dd5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57558/0 (socket says 172.21.15.98:57558) 2024-05-09T10:51:16.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.339+0000 7efeb5de6640 1 -- 172.21.15.98:0/2349746526 learned_addr learned my addr 172.21.15.98:0/2349746526 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:16.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.339+0000 7efeb55e5640 1 --2- 172.21.15.98:0/2349746526 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55d234d50230 0x55d234d6e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.339+0000 7efeb65e7640 1 --2- 172.21.15.98:0/2349746526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d234d6dad0 0x55d234d6ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.339+0000 7efeb5de6640 1 -- 172.21.15.98:0/2349746526 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55d234d50230 msgr2=0x55d234d6e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:16.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.339+0000 7efeb5de6640 1 --2- 172.21.15.98:0/2349746526 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55d234d50230 0x55d234d6e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.339+0000 7efeb5de6640 1 -- 172.21.15.98:0/2349746526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d234d6dad0 msgr2=0x55d234d6ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:16.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.339+0000 7efeb5de6640 1 --2- 172.21.15.98:0/2349746526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d234d6dad0 0x55d234d6ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.339+0000 7efeb5de6640 1 -- 172.21.15.98:0/2349746526 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d234d4c3b0 con 0x55d234d4e7b0 2024-05-09T10:51:16.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.339+0000 7efeb65e7640 1 --2- 172.21.15.98:0/2349746526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d234d6dad0 0x55d234d6ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:16.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.340+0000 7efeb5de6640 1 --2- 172.21.15.98:0/2349746526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d4e7b0 0x55d234dd5a50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efeac075410 tx=0x7efeac0a8710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a87f3133cd27b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:16.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.340+0000 7efe90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-05-09T10:51:16.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.340+0000 7efe90ff9640 1 -- 172.21.15.98:0/2349746526 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeac0783f0 con 0x55d234d4e7b0 2024-05-09T10:51:16.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.340+0000 7efe90ff9640 1 -- 172.21.15.98:0/2349746526 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efeac0749f0 con 0x55d234d4e7b0 2024-05-09T10:51:16.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.340+0000 7efebae26740 1 -- 172.21.15.98:0/2349746526 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efeac076d90 con 0x55d234d4e7b0 2024-05-09T10:51:16.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.340+0000 7efe90ff9640 1 -- 172.21.15.98:0/2349746526 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7efeac077950 con 0x55d234d4e7b0 2024-05-09T10:51:16.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.340+0000 7efebae26740 10 client.5337 fetch_fsmap learned FSMap version 13 2024-05-09T10:51:16.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.340+0000 7efebae26740 1 -- 172.21.15.98:0/2349746526 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x55d234d4c3b0 con 0x55d234d4e7b0 2024-05-09T10:51:16.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.340+0000 7efe90ff9640 1 -- 172.21.15.98:0/2349746526 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeac076d90 con 0x55d234d4e7b0 2024-05-09T10:51:16.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.340+0000 7efe90ff9640 1 -- 172.21.15.98:0/2349746526 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 13) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efeac07dce0 con 0x55d234d4e7b0 2024-05-09T10:51:16.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.340+0000 7efebae26740 10 client.5337 fetch_fsmap finished waiting for FSMap version 13 2024-05-09T10:51:16.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.340+0000 7efebae26740 10 client.5337 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T10:51:16.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.340+0000 7efebae26740 20 client.5337 populate_metadata read hostname 'smithi098' 2024-05-09T10:51:16.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.340+0000 7efebae26740 1 --2- 172.21.15.98:0/2349746526 >> [v2:172.21.15.98:6835/164638690,v1:172.21.15.98:6837/164638690] conn(0x55d234d84ab0 0x55d234d84e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.340+0000 7efebae26740 4 client.5337 mds_command: new command op to 5127 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T10:51:16.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.340+0000 7efebae26740 1 -- 172.21.15.98:0/2349746526 --> [v2:172.21.15.98:6835/164638690,v1:172.21.15.98:6837/164638690] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d234c95f50 con 0x55d234d84ab0 2024-05-09T10:51:16.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.340+0000 7efeb55e5640 1 --2- 172.21.15.98:0/2349746526 >> [v2:172.21.15.98:6835/164638690,v1:172.21.15.98:6837/164638690] conn(0x55d234d84ab0 0x55d234d84e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.341+0000 7efe90ff9640 10 client.5337 ms_handle_connect on v2:172.21.15.98:6835/164638690 2024-05-09T10:51:16.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.341+0000 7efeb55e5640 1 --2- 172.21.15.98:0/2349746526 >> [v2:172.21.15.98:6835/164638690,v1:172.21.15.98:6837/164638690] conn(0x55d234d84ab0 0x55d234d84e90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff4df3cb304d7357 server_cookie=8e6112a1d49aa66b in_seq=0 out_seq=0 2024-05-09T10:51:16.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.341+0000 7efe90ff9640 1 -- 172.21.15.98:0/2349746526 <== mds.0 v2:172.21.15.98:6835/164638690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55d234c95f50 con 0x55d234d84ab0 2024-05-09T10:51:16.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.341+0000 7efe90ff9640 10 client.5337 handle_command_reply: tid=0 2024-05-09T10:51:16.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.341+0000 7efebae26740 1 client.5337 shutdown 2024-05-09T10:51:16.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.341+0000 7efebae26740 1 -- 172.21.15.98:0/2349746526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d4e7b0 msgr2=0x55d234dd5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:16.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.341+0000 7efebae26740 1 --2- 172.21.15.98:0/2349746526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d4e7b0 0x55d234dd5a50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efeac075410 tx=0x7efeac0a8710 comp rx=0 tx=0).stop 2024-05-09T10:51:16.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.341+0000 7efebae26740 1 -- 172.21.15.98:0/2349746526 shutdown_connections 2024-05-09T10:51:16.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.341+0000 7efebae26740 1 --2- 172.21.15.98:0/2349746526 >> [v2:172.21.15.98:6835/164638690,v1:172.21.15.98:6837/164638690] conn(0x55d234d84ab0 0x55d234d84e90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.341+0000 7efebae26740 1 --2- 172.21.15.98:0/2349746526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d234d6dad0 0x55d234d6ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.341+0000 7efebae26740 1 --2- 172.21.15.98:0/2349746526 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55d234d50230 0x55d234d6e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.341+0000 7efe90ff9640 0 client.5337 ms_handle_reset on v2:172.21.15.98:6835/164638690 2024-05-09T10:51:16.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.341+0000 7efebae26740 1 --2- 172.21.15.98:0/2349746526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d234d4e7b0 0x55d234dd5a50 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.341+0000 7efebae26740 1 -- 172.21.15.98:0/2349746526 >> 172.21.15.98:0/2349746526 conn(0x55d234d25f10 msgr2=0x55d234d517a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:16.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.341+0000 7efebae26740 1 -- 172.21.15.98:0/2349746526 shutdown_connections 2024-05-09T10:51:16.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.341+0000 7efebae26740 1 -- 172.21.15.98:0/2349746526 wait complete. 2024-05-09T10:51:16.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.341+0000 7efebae26740 20 client.5337 trim_cache size 0 max 16384 2024-05-09T10:51:16.341 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-05-09T10:51:16.341 INFO:teuthology.orchestra.run.smithi098.stdout: "cluster_fsid": "3f705980-2043-4fe5-8639-5415c9d3c944", 2024-05-09T10:51:16.341 INFO:teuthology.orchestra.run.smithi098.stdout: "whoami": 0, 2024-05-09T10:51:16.341 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 5127, 2024-05-09T10:51:16.342 INFO:teuthology.orchestra.run.smithi098.stdout: "want_state": "up:active", 2024-05-09T10:51:16.342 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "up:active", 2024-05-09T10:51:16.342 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs", 2024-05-09T10:51:16.342 INFO:teuthology.orchestra.run.smithi098.stdout: "rank_uptime": 8.9490982169999995, 2024-05-09T10:51:16.342 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsmap_epoch": 13, 2024-05-09T10:51:16.342 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch": 31, 2024-05-09T10:51:16.342 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch_barrier": 31, 2024-05-09T10:51:16.342 INFO:teuthology.orchestra.run.smithi098.stdout: "uptime": 15.82502678 2024-05-09T10:51:16.342 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-05-09T10:51:16.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.342+0000 7efeb804a640 1 -- 172.21.15.98:0/1131883942 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7efe9406b4d0 msgr2=0x7efe9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:16.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.342+0000 7efeb804a640 1 --2- 172.21.15.98:0/1131883942 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7efe9406b4d0 0x7efe9406d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efeb00cf3f0 tx=0x7efea00078f0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.342+0000 7efeb804a640 1 -- 172.21.15.98:0/1131883942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe9c0039e0 msgr2=0x7efeb01470d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:16.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.342+0000 7efeb804a640 1 --2- 172.21.15.98:0/1131883942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe9c0039e0 0x7efeb01470d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7efeac001f40 tx=0x7efeac0014a0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.342+0000 7efeb804a640 1 -- 172.21.15.98:0/1131883942 shutdown_connections 2024-05-09T10:51:16.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.342+0000 7efeb804a640 1 --2- 172.21.15.98:0/1131883942 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7efe9406b4d0 0x7efe9406d990 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.342+0000 7efeb804a640 1 --2- 172.21.15.98:0/1131883942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efeb015b6c0 0x7efeb015bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.342+0000 7efeb804a640 1 --2- 172.21.15.98:0/1131883942 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efeb00c70e0 0x7efeb0156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.342+0000 7efeb804a640 1 --2- 172.21.15.98:0/1131883942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe9c0039e0 0x7efeb01470d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.342+0000 7efeb804a640 1 -- 172.21.15.98:0/1131883942 >> 172.21.15.98:0/1131883942 conn(0x7efeb00b34e0 msgr2=0x7efeb00ae830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:16.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.342+0000 7efeb804a640 1 -- 172.21.15.98:0/1131883942 shutdown_connections 2024-05-09T10:51:16.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.342+0000 7efeb804a640 1 -- 172.21.15.98:0/1131883942 wait complete. 2024-05-09T10:51:16.346 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T10:51:16.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.457+0000 7f87051b0640 1 Processor -- start 2024-05-09T10:51:16.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.457+0000 7f87051b0640 1 -- start start 2024-05-09T10:51:16.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.458+0000 7f87051b0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8700143740 0x7f87000720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.458+0000 7f87051b0640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f87001443e0 0x7f870015e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.458+0000 7f87051b0640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8700143b20 con 0x7f8700143d70 2024-05-09T10:51:16.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.458+0000 7f87051b0640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f870015f460 con 0x7f87001443e0 2024-05-09T10:51:16.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.458+0000 7f87051b0640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87000b28c0 con 0x7f8700143740 2024-05-09T10:51:16.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.458+0000 7f86f7fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f87001443e0 0x7f870015e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.458+0000 7f86ff59e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f8700143d70 0x7f870015ec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36848/0 (socket says 172.21.15.98:36848) 2024-05-09T10:51:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.458+0000 7f86ff59e640 1 -- 172.21.15.98:0/1408474261 learned_addr learned my addr 172.21.15.98:0/1408474261 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.458+0000 7f86fe59c640 1 -- 172.21.15.98:0/1408474261 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3128221193 0 0) 0x7f8700143b20 con 0x7f8700143d70 2024-05-09T10:51:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.458+0000 7f86fe59c640 1 -- 172.21.15.98:0/1408474261 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86ec003520 con 0x7f8700143d70 2024-05-09T10:51:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.458+0000 7f86fed9d640 1 --2- 172.21.15.98:0/1408474261 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8700143740 0x7f87000720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.458+0000 7f86f7fff640 1 -- 172.21.15.98:0/1408474261 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8700143740 msgr2=0x7f87000720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.458+0000 7f86f7fff640 1 --2- 172.21.15.98:0/1408474261 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8700143740 0x7f87000720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.458+0000 7f86f7fff640 1 -- 172.21.15.98:0/1408474261 >> v1:172.21.15.98:6789/0 conn(0x7f8700143d70 legacy=0x7f870015ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.458+0000 7f86f7fff640 1 -- 172.21.15.98:0/1408474261 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87000b2ba0 con 0x7f87001443e0 2024-05-09T10:51:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.458+0000 7f86fed9d640 1 --2- 172.21.15.98:0/1408474261 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8700143740 0x7f87000720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.459+0000 7f86f7fff640 1 --2- 172.21.15.98:0/1408474261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87001443e0 0x7f870015e6e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f86e80097a0 tx=0x7f86e802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4e180924e3c51 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.459+0000 7f86fe59c640 1 -- 172.21.15.98:0/1408474261 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8037070 con 0x7f87001443e0 2024-05-09T10:51:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.459+0000 7f86fe59c640 1 -- 172.21.15.98:0/1408474261 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e802ae50 con 0x7f87001443e0 2024-05-09T10:51:16.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.459+0000 7f86fe59c640 1 -- 172.21.15.98:0/1408474261 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8032660 con 0x7f87001443e0 2024-05-09T10:51:16.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.459+0000 7f87051b0640 1 -- 172.21.15.98:0/1408474261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87001443e0 msgr2=0x7f870015e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:16.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.459+0000 7f87051b0640 1 --2- 172.21.15.98:0/1408474261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87001443e0 0x7f870015e6e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f86e80097a0 tx=0x7f86e802a100 comp rx=0 tx=0).stop 2024-05-09T10:51:16.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.459+0000 7f87051b0640 1 -- 172.21.15.98:0/1408474261 shutdown_connections 2024-05-09T10:51:16.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.460+0000 7f87051b0640 1 --2- 172.21.15.98:0/1408474261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87001443e0 0x7f870015e6e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.460+0000 7f87051b0640 1 --2- 172.21.15.98:0/1408474261 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8700143740 0x7f87000720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.460+0000 7f87051b0640 1 -- 172.21.15.98:0/1408474261 >> 172.21.15.98:0/1408474261 conn(0x7f870006abe0 msgr2=0x7f870006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:16.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.460+0000 7f87051b0640 1 -- 172.21.15.98:0/1408474261 shutdown_connections 2024-05-09T10:51:16.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.460+0000 7f87051b0640 1 -- 172.21.15.98:0/1408474261 wait complete. 2024-05-09T10:51:16.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.461+0000 7f87051b0640 1 Processor -- start 2024-05-09T10:51:16.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.461+0000 7f87051b0640 1 -- start start 2024-05-09T10:51:16.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.462+0000 7f87051b0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8700143740 0x7f870015cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.462+0000 7f87051b0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8700143d70 0x7f87000cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.462+0000 7f87051b0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87001443e0 0x7f870013f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.462+0000 7f87051b0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87000cc5b0 con 0x7f8700143d70 2024-05-09T10:51:16.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.462+0000 7f87051b0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87000cc430 con 0x7f8700143740 2024-05-09T10:51:16.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.462+0000 7f87051b0640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87000cc730 con 0x7f87001443e0 2024-05-09T10:51:16.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.462+0000 7f86f7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8700143d70 0x7f87000cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.462+0000 7f86f7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8700143d70 0x7f87000cfdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57576/0 (socket says 172.21.15.98:57576) 2024-05-09T10:51:16.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.462+0000 7f86f7fff640 1 -- 172.21.15.98:0/542635608 learned_addr learned my addr 172.21.15.98:0/542635608 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:16.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.462+0000 7f86ff59e640 1 --2- 172.21.15.98:0/542635608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87001443e0 0x7f870013f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.462+0000 7f86fed9d640 1 --2- 172.21.15.98:0/542635608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8700143740 0x7f870015cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.462+0000 7f86f7fff640 1 -- 172.21.15.98:0/542635608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87001443e0 msgr2=0x7f870013f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:16.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.462+0000 7f86f7fff640 1 --2- 172.21.15.98:0/542635608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87001443e0 0x7f870013f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.462+0000 7f86f7fff640 1 -- 172.21.15.98:0/542635608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8700143740 msgr2=0x7f870015cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:16.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.462+0000 7f86f7fff640 1 --2- 172.21.15.98:0/542635608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8700143740 0x7f870015cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.462+0000 7f86f7fff640 1 -- 172.21.15.98:0/542635608 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870015d330 con 0x7f8700143d70 2024-05-09T10:51:16.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.462+0000 7f86fed9d640 1 --2- 172.21.15.98:0/542635608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8700143740 0x7f870015cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:16.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.462+0000 7f86f7fff640 1 --2- 172.21.15.98:0/542635608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8700143d70 0x7f87000cfdf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f86e8035040 tx=0x7f86e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:16.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.462+0000 7f86f77fe640 1 -- 172.21.15.98:0/542635608 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8041070 con 0x7f8700143d70 2024-05-09T10:51:16.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.462+0000 7f86f77fe640 1 -- 172.21.15.98:0/542635608 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e803c460 con 0x7f8700143d70 2024-05-09T10:51:16.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.462+0000 7f87051b0640 1 -- 172.21.15.98:0/542635608 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f870015d610 con 0x7f8700143d70 2024-05-09T10:51:16.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.462+0000 7f86f77fe640 1 -- 172.21.15.98:0/542635608 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8037040 con 0x7f8700143d70 2024-05-09T10:51:16.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.462+0000 7f87051b0640 1 -- 172.21.15.98:0/542635608 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f870015dbd0 con 0x7f8700143d70 2024-05-09T10:51:16.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.463+0000 7f87051b0640 1 -- 172.21.15.98:0/542635608 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f870015df10 con 0x7f8700143d70 2024-05-09T10:51:16.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.464+0000 7f86f77fe640 1 -- 172.21.15.98:0/542635608 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f86e803a070 con 0x7f8700143d70 2024-05-09T10:51:16.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.464+0000 7f86f77fe640 1 --2- 172.21.15.98:0/542635608 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f86e006b4d0 0x7f86e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:16.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.464+0000 7f86f77fe640 1 -- 172.21.15.98:0/542635608 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86e80aa430 con 0x7f8700143d70 2024-05-09T10:51:16.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.464+0000 7f86fed9d640 1 --2- 172.21.15.98:0/542635608 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f86e006b4d0 0x7f86e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:16.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.465+0000 7f86fed9d640 1 --2- 172.21.15.98:0/542635608 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f86e006b4d0 0x7f86e006d990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f86f000aa30 tx=0x7f86f0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:16.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.468+0000 7f86f77fe640 1 -- 172.21.15.98:0/542635608 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f870015df10 con 0x7f8700143d70 2024-05-09T10:51:16.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.678+0000 7f87051b0640 1 -- 172.21.15.98:0/542635608 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8700069cb0 con 0x7f8700143d70 2024-05-09T10:51:16.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.678+0000 7f86f77fe640 1 -- 172.21.15.98:0/542635608 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+73 (secure 0 0 0) 0x7f86e8078cf0 con 0x7f8700143d70 2024-05-09T10:51:16.672 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T10:51:16.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.680+0000 7f87051b0640 1 -- 172.21.15.98:0/542635608 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f86e006b4d0 msgr2=0x7f86e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:16.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.680+0000 7f87051b0640 1 --2- 172.21.15.98:0/542635608 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f86e006b4d0 0x7f86e006d990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f86f000aa30 tx=0x7f86f0008040 comp rx=0 tx=0).stop 2024-05-09T10:51:16.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.680+0000 7f87051b0640 1 -- 172.21.15.98:0/542635608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8700143d70 msgr2=0x7f87000cfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:16.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.680+0000 7f87051b0640 1 --2- 172.21.15.98:0/542635608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8700143d70 0x7f87000cfdf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f86e8035040 tx=0x7f86e80046c0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.680+0000 7f87051b0640 1 -- 172.21.15.98:0/542635608 shutdown_connections 2024-05-09T10:51:16.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.680+0000 7f87051b0640 1 --2- 172.21.15.98:0/542635608 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f86e006b4d0 0x7f86e006d990 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.680+0000 7f87051b0640 1 --2- 172.21.15.98:0/542635608 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f87001443e0 0x7f870013f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.680+0000 7f87051b0640 1 --2- 172.21.15.98:0/542635608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8700143d70 0x7f87000cfdf0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.680+0000 7f87051b0640 1 --2- 172.21.15.98:0/542635608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8700143740 0x7f870015cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:16.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.680+0000 7f87051b0640 1 -- 172.21.15.98:0/542635608 >> 172.21.15.98:0/542635608 conn(0x7f870006abe0 msgr2=0x7f870006c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:16.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.680+0000 7f87051b0640 1 -- 172.21.15.98:0/542635608 shutdown_connections 2024-05-09T10:51:16.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:16.680+0000 7f87051b0640 1 -- 172.21.15.98:0/542635608 wait complete. 2024-05-09T10:51:16.685 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-09T10:51:16.685 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T10:51:16.685 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2024-05-09T10:51:16.685 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T10:51:16.685 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T10:51:16.685 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-09T10:51:16.685 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-09T10:51:16.685 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T10:51:16.685 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T10:51:16.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:16.685 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2024-05-09T10:51:16.701 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T10:51:16.702 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T10:51:16.702 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T10:51:16.702 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:51:16.702 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-05-09T10:51:16.702 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:51:16.702 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T10:51:16.702 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2024-05-09T10:51:16.702 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2024-05-09T10:51:16.702 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T10:51:16.702 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2024-05-09T10:51:16.702 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2024-05-09T10:51:16.702 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T10:51:16.703 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2024-05-09T10:51:16.703 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T10:51:16.703 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2024-05-09T10:51:16.703 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-09T10:51:16.703 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 42734sec preferred_lft 42734sec 2024-05-09T10:51:16.703 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link 2024-05-09T10:51:16.703 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:51:16.703 INFO:teuthology.orchestra.run.smithi098.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T10:51:16.703 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether ba:0c:be:30:aa:9c brd ff:ff:ff:ff:ff:ff 2024-05-09T10:51:16.703 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T10:51:16.703 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:51:16.703 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::f0ac:f3ff:fe82:9c89/64 scope link 2024-05-09T10:51:16.703 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:51:16.703 INFO:teuthology.orchestra.run.smithi098.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T10:51:16.703 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether ba:0c:be:30:aa:9c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T10:51:16.704 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::b80c:beff:fe30:aa9c/64 scope link 2024-05-09T10:51:16.704 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:51:16.704 INFO:teuthology.orchestra.run.smithi098.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T10:51:16.704 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether ea:ab:05:05:6c:c4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T10:51:16.704 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::e8ab:5ff:fe05:6cc4/64 scope link 2024-05-09T10:51:16.704 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:51:16.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:16.704 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T10:51:16.765 INFO:teuthology.orchestra.run.smithi098.stdout:1 2024-05-09T10:51:16.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:16.767 DEBUG:teuthology.orchestra.run.smithi098:> route 2024-05-09T10:51:16.823 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2024-05-09T10:51:16.824 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T10:51:16.824 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T10:51:16.824 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T10:51:16.824 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T10:51:16.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:16.824 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec bash -c ' 2024-05-09T10:51:16.824 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-05-09T10:51:16.824 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T10:51:16.824 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T10:51:16.824 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T10:51:16.824 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-05-09T10:51:16.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:16.910 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-05-09T10:51:16.926 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-09T10:51:16.926 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T10:51:16.926 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T10:51:16.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:16.926 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.0 up 2024-05-09T10:51:16.992 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T10:51:16.992 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T10:51:17.047 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T10:51:17.048 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T10:51:17.048 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T10:51:17.103 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-09T10:51:17.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:17.103 DEBUG:teuthology.orchestra.run.smithi098:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-09T10:51:17.186 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-09T10:51:17.186 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-09T10:51:17.186 INFO:teuthology.orchestra.run.smithi098.stdout:invalid new device string format 2024-05-09T10:51:17.194 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-09T10:51:17.194 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-09T10:51:17.195 INFO:teuthology.orchestra.run.smithi098.stdout:invalid new device string format 2024-05-09T10:51:17.195 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.105:3300,172.21.15.105:3301" 2024-05-09T10:51:17.195 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-09T10:51:17.203 INFO:teuthology.orchestra.run.smithi098.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-09T10:51:17.203 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-09T10:51:17.203 INFO:teuthology.orchestra.run.smithi098.stdout:invalid new device string format 2024-05-09T10:51:17.203 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.105:3300,172.21.15.105:3301" 2024-05-09T10:51:17.203 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-09T10:51:17.203 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: resolved to: "172.21.15.98:6789,172.21.15.105:6789,172.21.15.105:6790" 2024-05-09T10:51:17.203 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.98:6789,172.21.15.105:6789,172.21.15.105:6790:/ 2024-05-09T10:51:17.203 INFO:teuthology.orchestra.run.smithi098.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=3f705980-2043-4fe5-8639-5415c9d3c944" will pass to kernel 2024-05-09T10:51:17.210 INFO:teuthology.orchestra.run.smithi098.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-09T10:51:17.210 INFO:teuthology.orchestra.run.smithi098.stdout: You just mounted a file system that supports labels which does not 2024-05-09T10:51:17.210 INFO:teuthology.orchestra.run.smithi098.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-09T10:51:17.210 INFO:teuthology.orchestra.run.smithi098.stdout: applications will generate AVC messages and not be allowed access to 2024-05-09T10:51:17.210 INFO:teuthology.orchestra.run.smithi098.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-09T10:51:17.210 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-09T10:51:17.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:17.211 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T10:51:17.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:17.277 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T10:51:17.337 INFO:teuthology.orchestra.run.smithi098.stdout:client5349 2024-05-09T10:51:17.337 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:51:17.337 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:51:17.394 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T10:51:17.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.541+0000 7fcd3841b640 1 Processor -- start 2024-05-09T10:51:17.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.541+0000 7fcd3841b640 1 -- start start 2024-05-09T10:51:17.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.542+0000 7fcd3841b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcd3015dce0 0x7fcd3006bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:17.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.542+0000 7fcd3841b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd3015e0c0 0x7fcd3006c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:17.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.542+0000 7fcd3841b640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd300d0c70 con 0x7fcd300d0700 2024-05-09T10:51:17.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.542+0000 7fcd3841b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd300b4350 con 0x7fcd3015dce0 2024-05-09T10:51:17.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.542+0000 7fcd3841b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300d0f50 con 0x7fcd3015e0c0 2024-05-09T10:51:17.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.542+0000 7fcd369b8640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fcd300d0700 0x7fcd30161a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36856/0 (socket says 172.21.15.98:36856) 2024-05-09T10:51:17.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.542+0000 7fcd369b8640 1 -- 172.21.15.98:0/3085581929 learned_addr learned my addr 172.21.15.98:0/3085581929 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:17.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.542+0000 7fcd359b6640 1 --2- 172.21.15.98:0/3085581929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd3015e0c0 0x7fcd3006c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:17.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.543+0000 7fcd361b7640 1 --2- 172.21.15.98:0/3085581929 >> v2:172.21.15.105:3300/0 conn(0x7fcd3015dce0 0x7fcd3006bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:17.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.543+0000 7fcd351b5640 1 -- 172.21.15.98:0/3085581929 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371764452 0 0) 0x7fcd300d0c70 con 0x7fcd300d0700 2024-05-09T10:51:17.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.543+0000 7fcd351b5640 1 -- 172.21.15.98:0/3085581929 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd1c003520 con 0x7fcd300d0700 2024-05-09T10:51:17.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.543+0000 7fcd359b6640 1 -- 172.21.15.98:0/3085581929 >> v2:172.21.15.105:3300/0 conn(0x7fcd3015dce0 msgr2=0x7fcd3006bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:17.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.543+0000 7fcd359b6640 1 --2- 172.21.15.98:0/3085581929 >> v2:172.21.15.105:3300/0 conn(0x7fcd3015dce0 0x7fcd3006bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:17.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.543+0000 7fcd359b6640 1 -- 172.21.15.98:0/3085581929 >> v1:172.21.15.98:6789/0 conn(0x7fcd300d0700 legacy=0x7fcd30161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:17.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.543+0000 7fcd351b5640 1 -- 172.21.15.98:0/3085581929 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1995285889 0 0) 0x7fcd1c003520 con 0x7fcd300d0700 2024-05-09T10:51:17.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.543+0000 7fcd359b6640 1 -- 172.21.15.98:0/3085581929 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd300b0fd0 con 0x7fcd3015e0c0 2024-05-09T10:51:17.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.543+0000 7fcd361b7640 1 --2- 172.21.15.98:0/3085581929 >> v2:172.21.15.105:3300/0 conn(0x7fcd3015dce0 0x7fcd3006bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:51:17.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.544+0000 7fcd359b6640 1 --2- 172.21.15.98:0/3085581929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd3015e0c0 0x7fcd3006c3b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcd200097a0 tx=0x7fcd2002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99a7db80cfdadf72 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:17.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.544+0000 7fcd351b5640 1 -- 172.21.15.98:0/3085581929 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20037070 con 0x7fcd3015e0c0 2024-05-09T10:51:17.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.544+0000 7fcd351b5640 1 -- 172.21.15.98:0/3085581929 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2002ac30 con 0x7fcd3015e0c0 2024-05-09T10:51:17.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.544+0000 7fcd351b5640 1 -- 172.21.15.98:0/3085581929 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd280019f0 con 0x7fcd3015e0c0 2024-05-09T10:51:17.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.544+0000 7fcd3841b640 1 -- 172.21.15.98:0/3085581929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd3015e0c0 msgr2=0x7fcd3006c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:17.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.544+0000 7fcd3841b640 1 --2- 172.21.15.98:0/3085581929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd3015e0c0 0x7fcd3006c3b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcd200097a0 tx=0x7fcd2002a1a0 comp rx=0 tx=0).stop 2024-05-09T10:51:17.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.544+0000 7fcd3841b640 1 -- 172.21.15.98:0/3085581929 shutdown_connections 2024-05-09T10:51:17.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.544+0000 7fcd3841b640 1 --2- 172.21.15.98:0/3085581929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd3015e0c0 0x7fcd3006c3b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:17.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.544+0000 7fcd3841b640 1 --2- 172.21.15.98:0/3085581929 >> v2:172.21.15.105:3300/0 conn(0x7fcd3015dce0 0x7fcd3006bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:17.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.544+0000 7fcd3841b640 1 -- 172.21.15.98:0/3085581929 >> 172.21.15.98:0/3085581929 conn(0x7fcd300b34e0 msgr2=0x7fcd300b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:17.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.545+0000 7fcd3841b640 1 -- 172.21.15.98:0/3085581929 shutdown_connections 2024-05-09T10:51:17.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.545+0000 7fcd3841b640 1 -- 172.21.15.98:0/3085581929 wait complete. 2024-05-09T10:51:17.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.546+0000 7fcd3841b640 1 Processor -- start 2024-05-09T10:51:17.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.546+0000 7fcd3841b640 1 -- start start 2024-05-09T10:51:17.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.546+0000 7fcd3841b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd300d0700 0x7fcd3014b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:17.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.546+0000 7fcd3841b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd3015dce0 0x7fcd300b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:17.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.546+0000 7fcd3841b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd3015e0c0 0x7fcd300b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:17.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.546+0000 7fcd3841b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300b2ac0 con 0x7fcd3015dce0 2024-05-09T10:51:17.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.546+0000 7fcd3841b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300b1590 con 0x7fcd300d0700 2024-05-09T10:51:17.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.546+0000 7fcd3841b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300b2c40 con 0x7fcd3015e0c0 2024-05-09T10:51:17.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.546+0000 7fcd359b6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd3015dce0 0x7fcd300b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:17.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.546+0000 7fcd359b6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd3015dce0 0x7fcd300b03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57588/0 (socket says 172.21.15.98:57588) 2024-05-09T10:51:17.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.546+0000 7fcd359b6640 1 -- 172.21.15.98:0/2860405026 learned_addr learned my addr 172.21.15.98:0/2860405026 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:17.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.547+0000 7fcd369b8640 1 --2- 172.21.15.98:0/2860405026 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd3015e0c0 0x7fcd300b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:17.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.547+0000 7fcd361b7640 1 --2- 172.21.15.98:0/2860405026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd300d0700 0x7fcd3014b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:17.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.547+0000 7fcd359b6640 1 -- 172.21.15.98:0/2860405026 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd3015e0c0 msgr2=0x7fcd300b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:17.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.547+0000 7fcd359b6640 1 --2- 172.21.15.98:0/2860405026 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd3015e0c0 0x7fcd300b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:17.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.547+0000 7fcd359b6640 1 -- 172.21.15.98:0/2860405026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd300d0700 msgr2=0x7fcd3014b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:17.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.547+0000 7fcd359b6640 1 --2- 172.21.15.98:0/2860405026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd300d0700 0x7fcd3014b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:17.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.547+0000 7fcd359b6640 1 -- 172.21.15.98:0/2860405026 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd300c73f0 con 0x7fcd3015dce0 2024-05-09T10:51:17.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.547+0000 7fcd361b7640 1 --2- 172.21.15.98:0/2860405026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd300d0700 0x7fcd3014b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:17.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.547+0000 7fcd359b6640 1 --2- 172.21.15.98:0/2860405026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd3015dce0 0x7fcd300b03c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcd200098c0 tx=0x7fcd200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:17.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.547+0000 7fcd26ffd640 1 -- 172.21.15.98:0/2860405026 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20037050 con 0x7fcd3015dce0 2024-05-09T10:51:17.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.547+0000 7fcd26ffd640 1 -- 172.21.15.98:0/2860405026 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd20043070 con 0x7fcd3015dce0 2024-05-09T10:51:17.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.547+0000 7fcd3841b640 1 -- 172.21.15.98:0/2860405026 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd300c76d0 con 0x7fcd3015dce0 2024-05-09T10:51:17.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.547+0000 7fcd26ffd640 1 -- 172.21.15.98:0/2860405026 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20006e10 con 0x7fcd3015dce0 2024-05-09T10:51:17.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.547+0000 7fcd3841b640 1 -- 172.21.15.98:0/2860405026 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd300c7c90 con 0x7fcd3015dce0 2024-05-09T10:51:17.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.548+0000 7fcd3841b640 1 -- 172.21.15.98:0/2860405026 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd30069cb0 con 0x7fcd3015dce0 2024-05-09T10:51:17.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.549+0000 7fcd26ffd640 1 -- 172.21.15.98:0/2860405026 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd2003e670 con 0x7fcd3015dce0 2024-05-09T10:51:17.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.549+0000 7fcd26ffd640 1 --2- 172.21.15.98:0/2860405026 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fcd1406b500 0x7fcd1406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:17.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.549+0000 7fcd361b7640 1 --2- 172.21.15.98:0/2860405026 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fcd1406b500 0x7fcd1406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:17.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.549+0000 7fcd26ffd640 1 -- 172.21.15.98:0/2860405026 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd200aaad0 con 0x7fcd3015dce0 2024-05-09T10:51:17.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.550+0000 7fcd361b7640 1 --2- 172.21.15.98:0/2860405026 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fcd1406b500 0x7fcd1406d9c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009ac0 tx=0x7fcd2c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:17.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.553+0000 7fcd26ffd640 1 -- 172.21.15.98:0/2860405026 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd20079b00 con 0x7fcd3015dce0 2024-05-09T10:51:17.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.760+0000 7fcd3841b640 1 -- 172.21.15.98:0/2860405026 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcd300d0c70 con 0x7fcd3015dce0 2024-05-09T10:51:17.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.760+0000 7fcd26ffd640 1 -- 172.21.15.98:0/2860405026 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcd20079400 con 0x7fcd3015dce0 2024-05-09T10:51:17.753 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T10:51:17.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.762+0000 7fcd3841b640 1 -- 172.21.15.98:0/2860405026 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fcd1406b500 msgr2=0x7fcd1406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:17.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.762+0000 7fcd3841b640 1 --2- 172.21.15.98:0/2860405026 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fcd1406b500 0x7fcd1406d9c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009ac0 tx=0x7fcd2c008040 comp rx=0 tx=0).stop 2024-05-09T10:51:17.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.762+0000 7fcd3841b640 1 -- 172.21.15.98:0/2860405026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd3015dce0 msgr2=0x7fcd300b03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:17.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.762+0000 7fcd3841b640 1 --2- 172.21.15.98:0/2860405026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd3015dce0 0x7fcd300b03c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcd200098c0 tx=0x7fcd200046c0 comp rx=0 tx=0).stop 2024-05-09T10:51:17.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.762+0000 7fcd3841b640 1 -- 172.21.15.98:0/2860405026 shutdown_connections 2024-05-09T10:51:17.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.762+0000 7fcd3841b640 1 --2- 172.21.15.98:0/2860405026 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fcd1406b500 0x7fcd1406d9c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:17.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.762+0000 7fcd3841b640 1 --2- 172.21.15.98:0/2860405026 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd3015e0c0 0x7fcd300b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:17.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.762+0000 7fcd3841b640 1 --2- 172.21.15.98:0/2860405026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd3015dce0 0x7fcd300b03c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:17.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.762+0000 7fcd3841b640 1 --2- 172.21.15.98:0/2860405026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd300d0700 0x7fcd3014b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:17.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.762+0000 7fcd3841b640 1 -- 172.21.15.98:0/2860405026 >> 172.21.15.98:0/2860405026 conn(0x7fcd300b34e0 msgr2=0x7fcd300b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:17.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.762+0000 7fcd3841b640 1 -- 172.21.15.98:0/2860405026 shutdown_connections 2024-05-09T10:51:17.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:17.762+0000 7fcd3841b640 1 -- 172.21.15.98:0/2860405026 wait complete. 2024-05-09T10:51:17.766 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-09T10:51:17.766 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T10:51:17.766 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2024-05-09T10:51:17.766 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-09T10:51:17.766 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-09T10:51:17.766 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-09T10:51:17.766 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-09T10:51:17.766 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T10:51:17.767 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-09T10:51:17.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:17.767 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2024-05-09T10:51:17.788 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T10:51:17.788 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T10:51:17.788 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T10:51:17.788 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:51:17.788 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-05-09T10:51:17.788 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:51:17.788 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T10:51:17.788 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2024-05-09T10:51:17.788 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2024-05-09T10:51:17.788 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T10:51:17.788 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2024-05-09T10:51:17.788 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2024-05-09T10:51:17.789 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T10:51:17.789 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2024-05-09T10:51:17.789 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T10:51:17.789 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2024-05-09T10:51:17.789 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-09T10:51:17.789 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42728sec preferred_lft 42728sec 2024-05-09T10:51:17.789 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2024-05-09T10:51:17.789 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:51:17.789 INFO:teuthology.orchestra.run.smithi105.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T10:51:17.789 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 76:1d:6d:2c:57:3f brd ff:ff:ff:ff:ff:ff 2024-05-09T10:51:17.789 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T10:51:17.789 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:51:17.789 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::6ccf:48ff:fe53:46aa/64 scope link 2024-05-09T10:51:17.789 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:51:17.789 INFO:teuthology.orchestra.run.smithi105.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T10:51:17.790 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 76:1d:6d:2c:57:3f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T10:51:17.790 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::741d:6dff:fe2c:573f/64 scope link 2024-05-09T10:51:17.790 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:51:17.790 INFO:teuthology.orchestra.run.smithi105.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T10:51:17.790 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether a2:e5:cb:6d:9f:65 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-09T10:51:17.790 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::a0e5:cbff:fe6d:9f65/64 scope link 2024-05-09T10:51:17.790 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-05-09T10:51:17.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:17.790 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T10:51:17.856 INFO:teuthology.orchestra.run.smithi105.stdout:1 2024-05-09T10:51:17.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:17.858 DEBUG:teuthology.orchestra.run.smithi105:> route 2024-05-09T10:51:17.917 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2024-05-09T10:51:17.917 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T10:51:17.917 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T10:51:17.917 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T10:51:17.917 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T10:51:17.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:17.918 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec bash -c ' 2024-05-09T10:51:17.918 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-05-09T10:51:17.918 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T10:51:17.918 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T10:51:17.918 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T10:51:17.918 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-05-09T10:51:18.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:18.011 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-05-09T10:51:18.028 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-09T10:51:18.028 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-09T10:51:18.028 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-09T10:51:18.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:18.028 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2024-05-09T10:51:18.093 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T10:51:18.094 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-09T10:51:18.150 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-09T10:51:18.150 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T10:51:18.151 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-09T10:51:18.210 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-09T10:51:18.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:18.210 DEBUG:teuthology.orchestra.run.smithi105:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-09T10:51:18.293 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-09T10:51:18.294 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-09T10:51:18.294 INFO:teuthology.orchestra.run.smithi105.stdout:invalid new device string format 2024-05-09T10:51:18.302 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-09T10:51:18.302 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-09T10:51:18.302 INFO:teuthology.orchestra.run.smithi105.stdout:invalid new device string format 2024-05-09T10:51:18.302 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.105:3300,172.21.15.105:3301" 2024-05-09T10:51:18.302 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-09T10:51:18.307 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-09T10:51:18.307 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-09T10:51:18.307 INFO:teuthology.orchestra.run.smithi105.stdout:invalid new device string format 2024-05-09T10:51:18.307 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: resolved to: "172.21.15.98:3300,172.21.15.105:3300,172.21.15.105:3301" 2024-05-09T10:51:18.307 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-09T10:51:18.307 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: resolved to: "172.21.15.98:6789,172.21.15.105:6789,172.21.15.105:6790" 2024-05-09T10:51:18.307 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.98:6789,172.21.15.105:6789,172.21.15.105:6790:/ 2024-05-09T10:51:18.307 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=3f705980-2043-4fe5-8639-5415c9d3c944" will pass to kernel 2024-05-09T10:51:18.308 INFO:teuthology.orchestra.run.smithi105.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-09T10:51:18.308 INFO:teuthology.orchestra.run.smithi105.stdout: You just mounted a file system that supports labels which does not 2024-05-09T10:51:18.308 INFO:teuthology.orchestra.run.smithi105.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-09T10:51:18.308 INFO:teuthology.orchestra.run.smithi105.stdout: applications will generate AVC messages and not be allowed access to 2024-05-09T10:51:18.308 INFO:teuthology.orchestra.run.smithi105.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-09T10:51:18.310 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-09T10:51:18.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:18.310 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-09T10:51:18.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:18.375 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T10:51:18.432 INFO:teuthology.orchestra.run.smithi105.stdout:client5017 2024-05-09T10:51:18.433 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:51:18.433 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:51:18.489 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-05-09T10:51:18.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.611+0000 7f2d1357e640 1 Processor -- start 2024-05-09T10:51:18.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.611+0000 7f2d1357e640 1 -- start start 2024-05-09T10:51:18.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.612+0000 7f2d1357e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2d0c0cedf0 0x7f2d0c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:18.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.612+0000 7f2d1357e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d0c0cd660 con 0x7f2d0c15a4e0 2024-05-09T10:51:18.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.612+0000 7f2d1357e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d0c0cd920 con 0x7f2d0c15a100 2024-05-09T10:51:18.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.612+0000 7f2d1357e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d0c0cdc00 con 0x7f2d0c0cedf0 2024-05-09T10:51:18.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.612+0000 7f2d11b1b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2d0c0cedf0 0x7f2d0c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:18.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.612+0000 7f2d11b1b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2d0c0cedf0 0x7f2d0c06d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57590/0 (socket says 172.21.15.98:57590) 2024-05-09T10:51:18.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.612+0000 7f2d11b1b640 1 -- 172.21.15.98:0/3639584469 learned_addr learned my addr 172.21.15.98:0/3639584469 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:18.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.612+0000 7f2d10b19640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2d0c15a4e0 0x7f2d0c0ce6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36866/0 (socket says 172.21.15.98:36866) 2024-05-09T10:51:18.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.613+0000 7f2d11b1b640 1 -- 172.21.15.98:0/3639584469 >> v1:172.21.15.105:6789/0 conn(0x7f2d0c15a100 legacy=0x7f2d0c0c4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:18.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.613+0000 7f2d11b1b640 1 -- 172.21.15.98:0/3639584469 >> v1:172.21.15.98:6789/0 conn(0x7f2d0c15a4e0 legacy=0x7f2d0c0ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:18.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.613+0000 7f2d03fff640 1 -- 172.21.15.98:0/3639584469 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2477734323 0 0) 0x7f2d0c0cd660 con 0x7f2d0c15a4e0 2024-05-09T10:51:18.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.613+0000 7f2d11b1b640 1 -- 172.21.15.98:0/3639584469 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c0cdee0 con 0x7f2d0c0cedf0 2024-05-09T10:51:18.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.613+0000 7f2d11b1b640 1 --2- 172.21.15.98:0/3639584469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d0c0cedf0 0x7f2d0c06d3c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009e50 tx=0x7f2d0402a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e227a0cd26c11c5f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:18.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.613+0000 7f2d03fff640 1 -- 172.21.15.98:0/3639584469 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d04037070 con 0x7f2d0c0cedf0 2024-05-09T10:51:18.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.614+0000 7f2d03fff640 1 -- 172.21.15.98:0/3639584469 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d04032630 con 0x7f2d0c0cedf0 2024-05-09T10:51:18.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.614+0000 7f2d03fff640 1 -- 172.21.15.98:0/3639584469 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0402e050 con 0x7f2d0c0cedf0 2024-05-09T10:51:18.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.614+0000 7f2d1357e640 1 -- 172.21.15.98:0/3639584469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d0c0cedf0 msgr2=0x7f2d0c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:18.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.614+0000 7f2d1357e640 1 --2- 172.21.15.98:0/3639584469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d0c0cedf0 0x7f2d0c06d3c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009e50 tx=0x7f2d0402a840 comp rx=0 tx=0).stop 2024-05-09T10:51:18.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.614+0000 7f2d1357e640 1 -- 172.21.15.98:0/3639584469 shutdown_connections 2024-05-09T10:51:18.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.614+0000 7f2d1357e640 1 --2- 172.21.15.98:0/3639584469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d0c0cedf0 0x7f2d0c06d3c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:18.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.614+0000 7f2d1357e640 1 -- 172.21.15.98:0/3639584469 >> 172.21.15.98:0/3639584469 conn(0x7f2d0c0cf760 msgr2=0x7f2d0c0cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:18.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.614+0000 7f2d1357e640 1 -- 172.21.15.98:0/3639584469 shutdown_connections 2024-05-09T10:51:18.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.614+0000 7f2d1357e640 1 -- 172.21.15.98:0/3639584469 wait complete. 2024-05-09T10:51:18.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.615+0000 7f2d1357e640 1 Processor -- start 2024-05-09T10:51:18.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.615+0000 7f2d1357e640 1 -- start start 2024-05-09T10:51:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.616+0000 7f2d1357e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d0c0cedf0 0x7f2d0c1f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.616+0000 7f2d1357e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d0c15a100 0x7f2d0c0ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.616+0000 7f2d1357e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d0c15a4e0 0x7f2d0c1f1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.616+0000 7f2d1357e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c0c4d30 con 0x7f2d0c15a4e0 2024-05-09T10:51:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.616+0000 7f2d1357e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c06e460 con 0x7f2d0c0cedf0 2024-05-09T10:51:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.616+0000 7f2d1357e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c1f7b10 con 0x7f2d0c15a100 2024-05-09T10:51:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.616+0000 7f2d11b1b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d0c15a4e0 0x7f2d0c1f1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.616+0000 7f2d11b1b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d0c15a4e0 0x7f2d0c1f1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57606/0 (socket says 172.21.15.98:57606) 2024-05-09T10:51:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.616+0000 7f2d11b1b640 1 -- 172.21.15.98:0/2240094698 learned_addr learned my addr 172.21.15.98:0/2240094698 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.616+0000 7f2d1131a640 1 --2- 172.21.15.98:0/2240094698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d0c0cedf0 0x7f2d0c1f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:18.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.616+0000 7f2d10b19640 1 --2- 172.21.15.98:0/2240094698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d0c15a100 0x7f2d0c0ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:18.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.617+0000 7f2d11b1b640 1 -- 172.21.15.98:0/2240094698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d0c15a100 msgr2=0x7f2d0c0ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:18.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.617+0000 7f2d11b1b640 1 --2- 172.21.15.98:0/2240094698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d0c15a100 0x7f2d0c0ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:18.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.617+0000 7f2d11b1b640 1 -- 172.21.15.98:0/2240094698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d0c0cedf0 msgr2=0x7f2d0c1f4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:18.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.617+0000 7f2d11b1b640 1 --2- 172.21.15.98:0/2240094698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d0c0cedf0 0x7f2d0c1f4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:18.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.617+0000 7f2d11b1b640 1 -- 172.21.15.98:0/2240094698 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c1f7df0 con 0x7f2d0c15a4e0 2024-05-09T10:51:18.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.617+0000 7f2d10b19640 1 --2- 172.21.15.98:0/2240094698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d0c15a100 0x7f2d0c0ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:18.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.617+0000 7f2d11b1b640 1 --2- 172.21.15.98:0/2240094698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d0c15a4e0 0x7f2d0c1f1140 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009f70 tx=0x7f2d04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:18.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.617+0000 7f2d01ffb640 1 -- 172.21.15.98:0/2240094698 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d04037050 con 0x7f2d0c15a4e0 2024-05-09T10:51:18.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.617+0000 7f2d01ffb640 1 -- 172.21.15.98:0/2240094698 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d04043070 con 0x7f2d0c15a4e0 2024-05-09T10:51:18.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.617+0000 7f2d1357e640 1 -- 172.21.15.98:0/2240094698 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0c1f80d0 con 0x7f2d0c15a4e0 2024-05-09T10:51:18.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.617+0000 7f2d01ffb640 1 -- 172.21.15.98:0/2240094698 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d04009510 con 0x7f2d0c15a4e0 2024-05-09T10:51:18.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.617+0000 7f2d1357e640 1 -- 172.21.15.98:0/2240094698 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0c1f7770 con 0x7f2d0c15a4e0 2024-05-09T10:51:18.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.618+0000 7f2d1357e640 1 -- 172.21.15.98:0/2240094698 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0c072c80 con 0x7f2d0c15a4e0 2024-05-09T10:51:18.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.619+0000 7f2d01ffb640 1 -- 172.21.15.98:0/2240094698 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d04007b50 con 0x7f2d0c15a4e0 2024-05-09T10:51:18.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.619+0000 7f2d01ffb640 1 --2- 172.21.15.98:0/2240094698 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2cf006b4d0 0x7f2cf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:18.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.619+0000 7f2d01ffb640 1 -- 172.21.15.98:0/2240094698 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d040aad10 con 0x7f2d0c15a4e0 2024-05-09T10:51:18.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.619+0000 7f2d1131a640 1 --2- 172.21.15.98:0/2240094698 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2cf006b4d0 0x7f2cf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:18.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.620+0000 7f2d1131a640 1 --2- 172.21.15.98:0/2240094698 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2cf006b4d0 0x7f2cf006d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2d08000f60 tx=0x7f2d08009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:18.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.623+0000 7f2d01ffb640 1 -- 172.21.15.98:0/2240094698 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d04079cb0 con 0x7f2d0c15a4e0 2024-05-09T10:51:18.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:18.828+0000 7f2d1357e640 1 -- 172.21.15.98:0/2240094698 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2d0c0b5760 con 0x7f2d0c15a4e0 2024-05-09T10:51:19.383 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T10:51:19.385 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T10:51:19.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.440+0000 7f2d01ffb640 1 -- 172.21.15.98:0/2240094698 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v14) v1 ==== 136+0+0 (secure 0 0 0) 0x7f2d040795b0 con 0x7f2d0c15a4e0 2024-05-09T10:51:19.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.442+0000 7f2d1357e640 1 -- 172.21.15.98:0/2240094698 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2cf006b4d0 msgr2=0x7f2cf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:19.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.442+0000 7f2d1357e640 1 --2- 172.21.15.98:0/2240094698 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2cf006b4d0 0x7f2cf006d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2d08000f60 tx=0x7f2d08009040 comp rx=0 tx=0).stop 2024-05-09T10:51:19.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.442+0000 7f2d1357e640 1 -- 172.21.15.98:0/2240094698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d0c15a4e0 msgr2=0x7f2d0c1f1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:19.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.442+0000 7f2d1357e640 1 --2- 172.21.15.98:0/2240094698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d0c15a4e0 0x7f2d0c1f1140 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009f70 tx=0x7f2d04004a40 comp rx=0 tx=0).stop 2024-05-09T10:51:19.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.442+0000 7f2d1357e640 1 -- 172.21.15.98:0/2240094698 shutdown_connections 2024-05-09T10:51:19.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.442+0000 7f2d1357e640 1 --2- 172.21.15.98:0/2240094698 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2cf006b4d0 0x7f2cf006d990 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:19.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.442+0000 7f2d1357e640 1 --2- 172.21.15.98:0/2240094698 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2d0c15a4e0 0x7f2d0c1f1140 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:19.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.442+0000 7f2d1357e640 1 --2- 172.21.15.98:0/2240094698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2d0c15a100 0x7f2d0c0ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:19.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.442+0000 7f2d1357e640 1 --2- 172.21.15.98:0/2240094698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d0c0cedf0 0x7f2d0c1f4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:19.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.442+0000 7f2d1357e640 1 -- 172.21.15.98:0/2240094698 >> 172.21.15.98:0/2240094698 conn(0x7f2d0c0cf760 msgr2=0x7f2d0c0c5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:19.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.442+0000 7f2d1357e640 1 -- 172.21.15.98:0/2240094698 shutdown_connections 2024-05-09T10:51:19.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.442+0000 7f2d1357e640 1 -- 172.21.15.98:0/2240094698 wait complete. 2024-05-09T10:51:19.446 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-05-09T10:51:19.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.554+0000 7fa04fb9e640 1 Processor -- start 2024-05-09T10:51:19.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.554+0000 7fa04fb9e640 1 -- start start 2024-05-09T10:51:19.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.555+0000 7fa04fb9e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa04815de40 0x7fa0480d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:19.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.555+0000 7fa04fb9e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0480cf250 0x7fa0480cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:19.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.555+0000 7fa04fb9e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa0480d0470 0x7fa0480d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:19.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.555+0000 7fa04fb9e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0480d1790 con 0x7fa04815de40 2024-05-09T10:51:19.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.555+0000 7fa04fb9e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa04815e220 con 0x7fa0480d0470 2024-05-09T10:51:19.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.555+0000 7fa04fb9e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0480d19e0 con 0x7fa0480cf250 2024-05-09T10:51:19.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.555+0000 7fa04d93a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa04815de40 0x7fa0480d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:19.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.555+0000 7fa04d93a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa04815de40 0x7fa0480d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57620/0 (socket says 172.21.15.98:57620) 2024-05-09T10:51:19.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.555+0000 7fa04d93a640 1 -- 172.21.15.98:0/4016290040 learned_addr learned my addr 172.21.15.98:0/4016290040 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:19.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.556+0000 7fa04e13b640 1 --2- 172.21.15.98:0/4016290040 >> v2:172.21.15.105:3300/0 conn(0x7fa0480d0470 0x7fa0480d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:19.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.556+0000 7fa04d93a640 1 -- 172.21.15.98:0/4016290040 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0480cf250 msgr2=0x7fa0480cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:19.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.556+0000 7fa04d93a640 1 --2- 172.21.15.98:0/4016290040 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0480cf250 0x7fa0480cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:19.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.556+0000 7fa04d93a640 1 -- 172.21.15.98:0/4016290040 >> v2:172.21.15.105:3300/0 conn(0x7fa0480d0470 msgr2=0x7fa0480d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:19.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.556+0000 7fa04d93a640 1 --2- 172.21.15.98:0/4016290040 >> v2:172.21.15.105:3300/0 conn(0x7fa0480d0470 0x7fa0480d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:19.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.556+0000 7fa04d93a640 1 -- 172.21.15.98:0/4016290040 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0480d1cc0 con 0x7fa04815de40 2024-05-09T10:51:19.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.556+0000 7fa04e13b640 1 --2- 172.21.15.98:0/4016290040 >> v2:172.21.15.105:3300/0 conn(0x7fa0480d0470 0x7fa0480d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:19.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.556+0000 7fa04d93a640 1 --2- 172.21.15.98:0/4016290040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa04815de40 0x7fa0480d27a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa048072ed0 tx=0x7fa03802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20581a5ceef598b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:19.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.556+0000 7fa04c938640 1 -- 172.21.15.98:0/4016290040 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa038036070 con 0x7fa04815de40 2024-05-09T10:51:19.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.557+0000 7fa04c938640 1 -- 172.21.15.98:0/4016290040 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa038031600 con 0x7fa04815de40 2024-05-09T10:51:19.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.557+0000 7fa04c938640 1 -- 172.21.15.98:0/4016290040 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0380319d0 con 0x7fa04815de40 2024-05-09T10:51:19.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.557+0000 7fa04fb9e640 1 -- 172.21.15.98:0/4016290040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa04815de40 msgr2=0x7fa0480d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:19.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.557+0000 7fa04fb9e640 1 --2- 172.21.15.98:0/4016290040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa04815de40 0x7fa0480d27a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa048072ed0 tx=0x7fa03802a6a0 comp rx=0 tx=0).stop 2024-05-09T10:51:19.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.557+0000 7fa04fb9e640 1 -- 172.21.15.98:0/4016290040 shutdown_connections 2024-05-09T10:51:19.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.557+0000 7fa04fb9e640 1 --2- 172.21.15.98:0/4016290040 >> v2:172.21.15.105:3300/0 conn(0x7fa0480d0470 0x7fa0480d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:19.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.557+0000 7fa04fb9e640 1 --2- 172.21.15.98:0/4016290040 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0480cf250 0x7fa0480cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:19.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.557+0000 7fa04fb9e640 1 --2- 172.21.15.98:0/4016290040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa04815de40 0x7fa0480d27a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:19.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.557+0000 7fa04fb9e640 1 -- 172.21.15.98:0/4016290040 >> 172.21.15.98:0/4016290040 conn(0x7fa04809ce90 msgr2=0x7fa0480d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:19.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.557+0000 7fa04fb9e640 1 -- 172.21.15.98:0/4016290040 shutdown_connections 2024-05-09T10:51:19.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.557+0000 7fa04fb9e640 1 -- 172.21.15.98:0/4016290040 wait complete. 2024-05-09T10:51:19.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.558+0000 7fa04fb9e640 1 Processor -- start 2024-05-09T10:51:19.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.558+0000 7fa04fb9e640 1 -- start start 2024-05-09T10:51:19.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.559+0000 7fa04fb9e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0480cf250 0x7fa04815cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:19.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.559+0000 7fa04fb9e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0480d0470 0x7fa0481f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:19.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.559+0000 7fa04fb9e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa04815de40 0x7fa04806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:19.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.559+0000 7fa04fb9e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0480d4680 con 0x7fa0480d0470 2024-05-09T10:51:19.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.559+0000 7fa04fb9e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0480d2280 con 0x7fa04815de40 2024-05-09T10:51:19.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.559+0000 7fa04fb9e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0480d4800 con 0x7fa0480cf250 2024-05-09T10:51:19.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.559+0000 7fa04d139640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0480d0470 0x7fa0481f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:19.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.559+0000 7fa04d139640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0480d0470 0x7fa0481f3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57622/0 (socket says 172.21.15.98:57622) 2024-05-09T10:51:19.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.559+0000 7fa04d139640 1 -- 172.21.15.98:0/2999453308 learned_addr learned my addr 172.21.15.98:0/2999453308 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:19.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.559+0000 7fa04e13b640 1 --2- 172.21.15.98:0/2999453308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa04815de40 0x7fa04806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:19.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.559+0000 7fa04d93a640 1 --2- 172.21.15.98:0/2999453308 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0480cf250 0x7fa04815cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:19.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.560+0000 7fa04d139640 1 -- 172.21.15.98:0/2999453308 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0480cf250 msgr2=0x7fa04815cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:19.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.560+0000 7fa04d139640 1 --2- 172.21.15.98:0/2999453308 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0480cf250 0x7fa04815cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:19.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.560+0000 7fa04d139640 1 -- 172.21.15.98:0/2999453308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa04815de40 msgr2=0x7fa04806c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:19.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.560+0000 7fa04d139640 1 --2- 172.21.15.98:0/2999453308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa04815de40 0x7fa04806c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:19.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.560+0000 7fa04d139640 1 -- 172.21.15.98:0/2999453308 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0480a0720 con 0x7fa0480d0470 2024-05-09T10:51:19.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.560+0000 7fa04e13b640 1 --2- 172.21.15.98:0/2999453308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa04815de40 0x7fa04806c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:19.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.560+0000 7fa04d93a640 1 --2- 172.21.15.98:0/2999453308 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0480cf250 0x7fa04815cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:19.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.560+0000 7fa04d139640 1 --2- 172.21.15.98:0/2999453308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0480d0470 0x7fa0481f3e30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00b410 tx=0x7fa03c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:19.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.560+0000 7fa0467fc640 1 -- 172.21.15.98:0/2999453308 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c015020 con 0x7fa0480d0470 2024-05-09T10:51:19.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.560+0000 7fa0467fc640 1 -- 172.21.15.98:0/2999453308 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03c011850 con 0x7fa0480d0470 2024-05-09T10:51:19.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.560+0000 7fa04fb9e640 1 -- 172.21.15.98:0/2999453308 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0480a1780 con 0x7fa0480d0470 2024-05-09T10:51:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.560+0000 7fa0467fc640 1 -- 172.21.15.98:0/2999453308 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c011c40 con 0x7fa0480d0470 2024-05-09T10:51:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.560+0000 7fa04fb9e640 1 -- 172.21.15.98:0/2999453308 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0480a0f80 con 0x7fa0480d0470 2024-05-09T10:51:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.561+0000 7fa04fb9e640 1 -- 172.21.15.98:0/2999453308 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa048069cb0 con 0x7fa0480d0470 2024-05-09T10:51:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.561+0000 7fa0467fc640 1 -- 172.21.15.98:0/2999453308 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa03c018020 con 0x7fa0480d0470 2024-05-09T10:51:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.562+0000 7fa0467fc640 1 --2- 172.21.15.98:0/2999453308 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa03406b500 0x7fa03406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.562+0000 7fa0467fc640 1 -- 172.21.15.98:0/2999453308 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa03c08a010 con 0x7fa0480d0470 2024-05-09T10:51:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.562+0000 7fa04d93a640 1 --2- 172.21.15.98:0/2999453308 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa03406b500 0x7fa03406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.562+0000 7fa04d93a640 1 --2- 172.21.15.98:0/2999453308 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa03406b500 0x7fa03406d9c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa038033000 tx=0x7fa038009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:19.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.566+0000 7fa0467fc640 1 -- 172.21.15.98:0/2999453308 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa03c01f020 con 0x7fa0480d0470 2024-05-09T10:51:19.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.759+0000 7fa04fb9e640 1 -- 172.21.15.98:0/2999453308 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0480a0a70 con 0x7fa03406b500 2024-05-09T10:51:19.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.760+0000 7fa0467fc640 1 -- 172.21.15.98:0/2999453308 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa0480a0a70 con 0x7fa03406b500 2024-05-09T10:51:19.753 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-05-09T10:51:19.753 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:19.753 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cephfs" 2024-05-09T10:51:19.753 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:19.754 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-05-09T10:51:19.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.762+0000 7fa04fb9e640 1 -- 172.21.15.98:0/2999453308 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa03406b500 msgr2=0x7fa03406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:19.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.762+0000 7fa04fb9e640 1 --2- 172.21.15.98:0/2999453308 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa03406b500 0x7fa03406d9c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa038033000 tx=0x7fa038009000 comp rx=0 tx=0).stop 2024-05-09T10:51:19.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.762+0000 7fa04fb9e640 1 -- 172.21.15.98:0/2999453308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0480d0470 msgr2=0x7fa0481f3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:19.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.762+0000 7fa04fb9e640 1 --2- 172.21.15.98:0/2999453308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0480d0470 0x7fa0481f3e30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00b410 tx=0x7fa03c00b8d0 comp rx=0 tx=0).stop 2024-05-09T10:51:19.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.762+0000 7fa04fb9e640 1 -- 172.21.15.98:0/2999453308 shutdown_connections 2024-05-09T10:51:19.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.762+0000 7fa04fb9e640 1 --2- 172.21.15.98:0/2999453308 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa03406b500 0x7fa03406d9c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:19.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.762+0000 7fa04fb9e640 1 --2- 172.21.15.98:0/2999453308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa04815de40 0x7fa04806c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:19.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.762+0000 7fa04fb9e640 1 --2- 172.21.15.98:0/2999453308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0480d0470 0x7fa0481f3e30 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:19.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.762+0000 7fa04fb9e640 1 --2- 172.21.15.98:0/2999453308 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0480cf250 0x7fa04815cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:19.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.762+0000 7fa04fb9e640 1 -- 172.21.15.98:0/2999453308 >> 172.21.15.98:0/2999453308 conn(0x7fa04809ce90 msgr2=0x7fa0480b7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:19.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.762+0000 7fa04fb9e640 1 -- 172.21.15.98:0/2999453308 shutdown_connections 2024-05-09T10:51:19.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.762+0000 7fa04fb9e640 1 -- 172.21.15.98:0/2999453308 wait complete. 2024-05-09T10:51:19.766 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-09T10:51:19.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.876+0000 7f05c5790640 1 Processor -- start 2024-05-09T10:51:19.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.876+0000 7f05c5790640 1 -- start start 2024-05-09T10:51:19.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.877+0000 7f05c5790640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f05c0161450 0x7f05c00d4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:19.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.877+0000 7f05c5790640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c0161830 con 0x7f05c00d2820 2024-05-09T10:51:19.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.877+0000 7f05c5790640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c00cf2f0 con 0x7f05c00b2a30 2024-05-09T10:51:19.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.877+0000 7f05c5790640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05c00b0760 con 0x7f05c0161450 2024-05-09T10:51:19.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.877+0000 7f05beffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f05c0161450 0x7f05c00d4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:19.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.877+0000 7f05bf7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f05c00d2820 0x7f05c00b0ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36872/0 (socket says 172.21.15.98:36872) 2024-05-09T10:51:19.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.877+0000 7f05bf7fe640 1 -- 172.21.15.98:0/3892101892 learned_addr learned my addr 172.21.15.98:0/3892101892 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:19.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.877+0000 7f05beffd640 1 -- 172.21.15.98:0/3892101892 >> v1:172.21.15.105:6789/0 conn(0x7f05c00b2a30 legacy=0x7f05c00d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:19.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.877+0000 7f05beffd640 1 -- 172.21.15.98:0/3892101892 >> v1:172.21.15.98:6789/0 conn(0x7f05c00d2820 legacy=0x7f05c00b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:19.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.877+0000 7f05beffd640 1 -- 172.21.15.98:0/3892101892 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c00cf5d0 con 0x7f05c0161450 2024-05-09T10:51:19.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.878+0000 7f05beffd640 1 --2- 172.21.15.98:0/3892101892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05c0161450 0x7f05c00d4650 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f05b00097d0 tx=0x7f05b002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa1c55c6cc1c456 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:19.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.878+0000 7f05be7fc640 1 -- 172.21.15.98:0/3892101892 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0037070 con 0x7f05c0161450 2024-05-09T10:51:19.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.878+0000 7f05be7fc640 1 -- 172.21.15.98:0/3892101892 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b0032440 con 0x7f05c0161450 2024-05-09T10:51:19.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.878+0000 7f05be7fc640 1 -- 172.21.15.98:0/3892101892 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0032810 con 0x7f05c0161450 2024-05-09T10:51:19.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.879+0000 7f05c5790640 1 -- 172.21.15.98:0/3892101892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05c0161450 msgr2=0x7f05c00d4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:19.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.879+0000 7f05c5790640 1 --2- 172.21.15.98:0/3892101892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05c0161450 0x7f05c00d4650 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f05b00097d0 tx=0x7f05b002a1c0 comp rx=0 tx=0).stop 2024-05-09T10:51:19.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.879+0000 7f05c5790640 1 -- 172.21.15.98:0/3892101892 shutdown_connections 2024-05-09T10:51:19.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.879+0000 7f05c5790640 1 --2- 172.21.15.98:0/3892101892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05c0161450 0x7f05c00d4650 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:19.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.879+0000 7f05c5790640 1 -- 172.21.15.98:0/3892101892 >> 172.21.15.98:0/3892101892 conn(0x7f05c00d3b30 msgr2=0x7f05c00d3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:19.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.879+0000 7f05c5790640 1 -- 172.21.15.98:0/3892101892 shutdown_connections 2024-05-09T10:51:19.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.879+0000 7f05c5790640 1 -- 172.21.15.98:0/3892101892 wait complete. 2024-05-09T10:51:19.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.880+0000 7f05c5790640 1 Processor -- start 2024-05-09T10:51:19.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.880+0000 7f05c5790640 1 -- start start 2024-05-09T10:51:19.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.881+0000 7f05c5790640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05c00b2a30 0x7f05c00cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:19.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.881+0000 7f05c5790640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05c00d2820 0x7f05c0156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:19.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.881+0000 7f05c5790640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05c0161450 0x7f05c00cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:19.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.881+0000 7f05c5790640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c00b15e0 con 0x7f05c00b2a30 2024-05-09T10:51:19.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.881+0000 7f05c5790640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c00b09b0 con 0x7f05c0161450 2024-05-09T10:51:19.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.881+0000 7f05c5790640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c00b1760 con 0x7f05c00d2820 2024-05-09T10:51:19.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.881+0000 7f05beffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05c00b2a30 0x7f05c00cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:19.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.881+0000 7f05beffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05c00b2a30 0x7f05c00cd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57634/0 (socket says 172.21.15.98:57634) 2024-05-09T10:51:19.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.881+0000 7f05beffd640 1 -- 172.21.15.98:0/3932732254 learned_addr learned my addr 172.21.15.98:0/3932732254 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:19.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.881+0000 7f05beffd640 1 -- 172.21.15.98:0/3932732254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05c00d2820 msgr2=0x7f05c0156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:19.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.881+0000 7f05beffd640 1 --2- 172.21.15.98:0/3932732254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05c00d2820 0x7f05c0156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:19.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.881+0000 7f05beffd640 1 -- 172.21.15.98:0/3932732254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05c0161450 msgr2=0x7f05c00cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:19.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.881+0000 7f05beffd640 1 --2- 172.21.15.98:0/3932732254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05c0161450 0x7f05c00cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:19.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.881+0000 7f05beffd640 1 -- 172.21.15.98:0/3932732254 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c00cde20 con 0x7f05c00b2a30 2024-05-09T10:51:19.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.881+0000 7f05beffd640 1 --2- 172.21.15.98:0/3932732254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05c00b2a30 0x7f05c00cd8e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f05b00098f0 tx=0x7f05b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:19.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.881+0000 7f05b7fff640 1 -- 172.21.15.98:0/3932732254 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0037040 con 0x7f05c00b2a30 2024-05-09T10:51:19.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.882+0000 7f05b7fff640 1 -- 172.21.15.98:0/3932732254 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b0042070 con 0x7f05c00b2a30 2024-05-09T10:51:19.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.882+0000 7f05c5790640 1 -- 172.21.15.98:0/3932732254 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c01f7ed0 con 0x7f05c00b2a30 2024-05-09T10:51:19.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.882+0000 7f05b7fff640 1 -- 172.21.15.98:0/3932732254 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0030a70 con 0x7f05c00b2a30 2024-05-09T10:51:19.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.882+0000 7f05c5790640 1 -- 172.21.15.98:0/3932732254 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c01f8490 con 0x7f05c00b2a30 2024-05-09T10:51:19.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.883+0000 7f05c5790640 1 -- 172.21.15.98:0/3932732254 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05c00a1ae0 con 0x7f05c00b2a30 2024-05-09T10:51:19.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.883+0000 7f05b7fff640 1 -- 172.21.15.98:0/3932732254 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f05b0030c10 con 0x7f05c00b2a30 2024-05-09T10:51:19.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.884+0000 7f05b7fff640 1 --2- 172.21.15.98:0/3932732254 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f05a006b4d0 0x7f05a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:19.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.884+0000 7f05b67fc640 1 --2- 172.21.15.98:0/3932732254 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f05a006b4d0 0x7f05a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:19.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.884+0000 7f05b7fff640 1 -- 172.21.15.98:0/3932732254 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05b00aabe0 con 0x7f05c00b2a30 2024-05-09T10:51:19.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.884+0000 7f05b67fc640 1 --2- 172.21.15.98:0/3932732254 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f05a006b4d0 0x7f05a006d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f05a8000f60 tx=0x7f05a80079d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:19.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:19.888+0000 7f05b7fff640 1 -- 172.21.15.98:0/3932732254 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f05b0079af0 con 0x7f05c00b2a30 2024-05-09T10:51:20.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.063+0000 7f05c5790640 1 -- 172.21.15.98:0/3932732254 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f05c00b27f0 con 0x7f05c00b2a30 2024-05-09T10:51:20.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.064+0000 7f05b7fff640 1 -- 172.21.15.98:0/3932732254 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f05b00793f0 con 0x7f05c00b2a30 2024-05-09T10:51:20.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.065+0000 7f05c5790640 1 -- 172.21.15.98:0/3932732254 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f05a006b4d0 msgr2=0x7f05a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:20.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.065+0000 7f05c5790640 1 --2- 172.21.15.98:0/3932732254 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f05a006b4d0 0x7f05a006d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f05a8000f60 tx=0x7f05a80079d0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.065+0000 7f05c5790640 1 -- 172.21.15.98:0/3932732254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05c00b2a30 msgr2=0x7f05c00cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:20.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.065+0000 7f05c5790640 1 --2- 172.21.15.98:0/3932732254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05c00b2a30 0x7f05c00cd8e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f05b00098f0 tx=0x7f05b00046c0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.066+0000 7f05c5790640 1 -- 172.21.15.98:0/3932732254 shutdown_connections 2024-05-09T10:51:20.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.066+0000 7f05c5790640 1 --2- 172.21.15.98:0/3932732254 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f05a006b4d0 0x7f05a006d990 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.066+0000 7f05c5790640 1 --2- 172.21.15.98:0/3932732254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05c0161450 0x7f05c00cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.066+0000 7f05c5790640 1 --2- 172.21.15.98:0/3932732254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05c00d2820 0x7f05c0156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.066+0000 7f05c5790640 1 --2- 172.21.15.98:0/3932732254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05c00b2a30 0x7f05c00cd8e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.066+0000 7f05c5790640 1 -- 172.21.15.98:0/3932732254 >> 172.21.15.98:0/3932732254 conn(0x7f05c00d3b30 msgr2=0x7f05c00d40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:20.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.066+0000 7f05c5790640 1 -- 172.21.15.98:0/3932732254 shutdown_connections 2024-05-09T10:51:20.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.066+0000 7f05c5790640 1 -- 172.21.15.98:0/3932732254 wait complete. 2024-05-09T10:51:20.070 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp59180 2024-05-09T10:51:20.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.180+0000 7f5af759e640 1 Processor -- start 2024-05-09T10:51:20.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.180+0000 7f5af759e640 1 -- start start 2024-05-09T10:51:20.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.181+0000 7f5af759e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5af815dd60 0x7f5af806b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:20.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.181+0000 7f5af759e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5af80d3950 0x7f5af80d3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:20.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.181+0000 7f5af759e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af80cf400 con 0x7f5af80ce5d0 2024-05-09T10:51:20.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.181+0000 7f5af759e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af80cf9c0 con 0x7f5af815dd60 2024-05-09T10:51:20.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.181+0000 7f5af759e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af80d42d0 con 0x7f5af80d3950 2024-05-09T10:51:20.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.181+0000 7f5af659c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5af815dd60 0x7f5af806b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:20.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.181+0000 7f5af6d9d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f5af80ce5d0 0x7f5af80ce9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36876/0 (socket says 172.21.15.98:36876) 2024-05-09T10:51:20.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.181+0000 7f5af6d9d640 1 -- 172.21.15.98:0/1867630340 learned_addr learned my addr 172.21.15.98:0/1867630340 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:20.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.181+0000 7f5aedd9b640 1 --2- 172.21.15.98:0/1867630340 >> v2:172.21.15.105:3300/0 conn(0x7f5af80d3950 0x7f5af80d3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:20.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.181+0000 7f5af659c640 1 -- 172.21.15.98:0/1867630340 >> v2:172.21.15.105:3300/0 conn(0x7f5af80d3950 msgr2=0x7f5af80d3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:20.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.181+0000 7f5af659c640 1 --2- 172.21.15.98:0/1867630340 >> v2:172.21.15.105:3300/0 conn(0x7f5af80d3950 0x7f5af80d3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.181+0000 7f5af659c640 1 -- 172.21.15.98:0/1867630340 >> v1:172.21.15.98:6789/0 conn(0x7f5af80ce5d0 legacy=0x7f5af80ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:20.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.181+0000 7f5af5d9b640 1 -- 172.21.15.98:0/1867630340 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3114200344 0 0) 0x7f5af80cf400 con 0x7f5af80ce5d0 2024-05-09T10:51:20.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.181+0000 7f5af659c640 1 -- 172.21.15.98:0/1867630340 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af80cf6e0 con 0x7f5af815dd60 2024-05-09T10:51:20.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.182+0000 7f5af659c640 1 --2- 172.21.15.98:0/1867630340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5af815dd60 0x7f5af806b690 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5af808d9a0 tx=0x7f5ae802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d160984773dc026d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:20.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.182+0000 7f5af5d9b640 1 -- 172.21.15.98:0/1867630340 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8037070 con 0x7f5af815dd60 2024-05-09T10:51:20.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.182+0000 7f5af5d9b640 1 -- 172.21.15.98:0/1867630340 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae8032440 con 0x7f5af815dd60 2024-05-09T10:51:20.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.182+0000 7f5af5d9b640 1 -- 172.21.15.98:0/1867630340 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8032810 con 0x7f5af815dd60 2024-05-09T10:51:20.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.183+0000 7f5af759e640 1 -- 172.21.15.98:0/1867630340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5af815dd60 msgr2=0x7f5af806b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:20.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.183+0000 7f5af759e640 1 --2- 172.21.15.98:0/1867630340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5af815dd60 0x7f5af806b690 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5af808d9a0 tx=0x7f5ae802a1a0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.183+0000 7f5af759e640 1 -- 172.21.15.98:0/1867630340 shutdown_connections 2024-05-09T10:51:20.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.183+0000 7f5af759e640 1 --2- 172.21.15.98:0/1867630340 >> v2:172.21.15.105:3300/0 conn(0x7f5af80d3950 0x7f5af80d3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.183+0000 7f5af759e640 1 --2- 172.21.15.98:0/1867630340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5af815dd60 0x7f5af806b690 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.183+0000 7f5af759e640 1 -- 172.21.15.98:0/1867630340 >> 172.21.15.98:0/1867630340 conn(0x7f5af808d3f0 msgr2=0x7f5af809cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:20.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.183+0000 7f5af759e640 1 -- 172.21.15.98:0/1867630340 shutdown_connections 2024-05-09T10:51:20.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.183+0000 7f5af759e640 1 -- 172.21.15.98:0/1867630340 wait complete. 2024-05-09T10:51:20.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.184+0000 7f5af759e640 1 Processor -- start 2024-05-09T10:51:20.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.184+0000 7f5af759e640 1 -- start start 2024-05-09T10:51:20.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.185+0000 7f5af759e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5af80ce5d0 0x7f5af80c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:20.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.185+0000 7f5af759e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5af80d3950 0x7f5af80d0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:20.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.185+0000 7f5af759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5af815dd60 0x7f5af80d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:20.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.185+0000 7f5af759e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af806bbd0 con 0x7f5af80d3950 2024-05-09T10:51:20.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.185+0000 7f5af759e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af80d19a0 con 0x7f5af815dd60 2024-05-09T10:51:20.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.185+0000 7f5af759e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af80cf0f0 con 0x7f5af80ce5d0 2024-05-09T10:51:20.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.185+0000 7f5aedd9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5af80d3950 0x7f5af80d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:20.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.185+0000 7f5aedd9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5af80d3950 0x7f5af80d0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57644/0 (socket says 172.21.15.98:57644) 2024-05-09T10:51:20.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.185+0000 7f5aedd9b640 1 -- 172.21.15.98:0/2664739945 learned_addr learned my addr 172.21.15.98:0/2664739945 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:20.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.185+0000 7f5af6d9d640 1 --2- 172.21.15.98:0/2664739945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5af815dd60 0x7f5af80d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:20.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.185+0000 7f5af659c640 1 --2- 172.21.15.98:0/2664739945 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5af80ce5d0 0x7f5af80c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:20.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.185+0000 7f5aedd9b640 1 -- 172.21.15.98:0/2664739945 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5af80ce5d0 msgr2=0x7f5af80c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:20.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.185+0000 7f5aedd9b640 1 --2- 172.21.15.98:0/2664739945 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5af80ce5d0 0x7f5af80c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.185+0000 7f5aedd9b640 1 -- 172.21.15.98:0/2664739945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5af815dd60 msgr2=0x7f5af80d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:20.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.185+0000 7f5aedd9b640 1 --2- 172.21.15.98:0/2664739945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5af815dd60 0x7f5af80d0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.185+0000 7f5aedd9b640 1 -- 172.21.15.98:0/2664739945 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af80ca1d0 con 0x7f5af80d3950 2024-05-09T10:51:20.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.185+0000 7f5af6d9d640 1 --2- 172.21.15.98:0/2664739945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5af815dd60 0x7f5af80d0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:20.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.186+0000 7f5aedd9b640 1 --2- 172.21.15.98:0/2664739945 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5af80d3950 0x7f5af80d0890 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000d800 tx=0x7f5ae000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:20.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.186+0000 7f5aef7fe640 1 -- 172.21.15.98:0/2664739945 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae000b6c0 con 0x7f5af80d3950 2024-05-09T10:51:20.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.186+0000 7f5aef7fe640 1 -- 172.21.15.98:0/2664739945 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae0019070 con 0x7f5af80d3950 2024-05-09T10:51:20.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.186+0000 7f5af759e640 1 -- 172.21.15.98:0/2664739945 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af81f7000 con 0x7f5af80d3950 2024-05-09T10:51:20.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.186+0000 7f5aef7fe640 1 -- 172.21.15.98:0/2664739945 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae000bac0 con 0x7f5af80d3950 2024-05-09T10:51:20.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.186+0000 7f5af759e640 1 -- 172.21.15.98:0/2664739945 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af81f75c0 con 0x7f5af80d3950 2024-05-09T10:51:20.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.187+0000 7f5af759e640 1 -- 172.21.15.98:0/2664739945 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af81f7c80 con 0x7f5af80d3950 2024-05-09T10:51:20.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.187+0000 7f5aef7fe640 1 -- 172.21.15.98:0/2664739945 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ae000bc60 con 0x7f5af80d3950 2024-05-09T10:51:20.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.188+0000 7f5aef7fe640 1 --2- 172.21.15.98:0/2664739945 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5ad806b500 0x7f5ad806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:20.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.188+0000 7f5aef7fe640 1 -- 172.21.15.98:0/2664739945 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ae008a6e0 con 0x7f5af80d3950 2024-05-09T10:51:20.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.188+0000 7f5af659c640 1 --2- 172.21.15.98:0/2664739945 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5ad806b500 0x7f5ad806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:20.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.188+0000 7f5af659c640 1 --2- 172.21.15.98:0/2664739945 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5ad806b500 0x7f5ad806d9c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8034000 tx=0x7f5ae802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:20.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.192+0000 7f5aef7fe640 1 -- 172.21.15.98:0/2664739945 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5af81f7c80 con 0x7f5af80d3950 2024-05-09T10:51:20.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.385+0000 7f5af759e640 1 -- 172.21.15.98:0/2664739945 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp59180", "target": ["mon-mgr", ""]}) v1 -- 0x7f5af81f6db0 con 0x7f5ad806b500 2024-05-09T10:51:20.382 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:51:20.389+0000 7fb2501d2640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T10:51:20.382 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:51:20.389+0000 7fb2501d2640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T10:51:20.382 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:51:20.389+0000 7fb2501d2640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T10:51:20.382 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:51:20.389+0000 7fb2501d2640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T10:51:20.383 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:51:20.389+0000 7fb2501d2640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T10:51:20.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.402+0000 7f5aef7fe640 1 -- 172.21.15.98:0/2664739945 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5af81f6db0 con 0x7f5ad806b500 2024-05-09T10:51:20.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.404+0000 7f5af759e640 1 -- 172.21.15.98:0/2664739945 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5ad806b500 msgr2=0x7f5ad806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:20.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.404+0000 7f5af759e640 1 --2- 172.21.15.98:0/2664739945 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5ad806b500 0x7f5ad806d9c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8034000 tx=0x7f5ae802e000 comp rx=0 tx=0).stop 2024-05-09T10:51:20.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.404+0000 7f5af759e640 1 -- 172.21.15.98:0/2664739945 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5af80d3950 msgr2=0x7f5af80d0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:20.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.404+0000 7f5af759e640 1 --2- 172.21.15.98:0/2664739945 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5af80d3950 0x7f5af80d0890 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000d800 tx=0x7f5ae000dcc0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.404+0000 7f5af759e640 1 -- 172.21.15.98:0/2664739945 shutdown_connections 2024-05-09T10:51:20.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.404+0000 7f5af759e640 1 --2- 172.21.15.98:0/2664739945 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f5ad806b500 0x7f5ad806d9c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.404+0000 7f5af759e640 1 --2- 172.21.15.98:0/2664739945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5af815dd60 0x7f5af80d0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.404+0000 7f5af759e640 1 --2- 172.21.15.98:0/2664739945 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5af80d3950 0x7f5af80d0890 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.404+0000 7f5af759e640 1 --2- 172.21.15.98:0/2664739945 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5af80ce5d0 0x7f5af80c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.404+0000 7f5af759e640 1 -- 172.21.15.98:0/2664739945 >> 172.21.15.98:0/2664739945 conn(0x7f5af808d3f0 msgr2=0x7f5af80b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:20.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.404+0000 7f5af759e640 1 -- 172.21.15.98:0/2664739945 shutdown_connections 2024-05-09T10:51:20.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.404+0000 7f5af759e640 1 -- 172.21.15.98:0/2664739945 wait complete. 2024-05-09T10:51:20.408 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol17470 --group_name subvol_grp59180 2024-05-09T10:51:20.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.520+0000 7fccd7fa5640 1 Processor -- start 2024-05-09T10:51:20.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.520+0000 7fccd7fa5640 1 -- start start 2024-05-09T10:51:20.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.521+0000 7fccd7fa5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fccd0161700 0x7fccd00b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:20.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.521+0000 7fccd7fa5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fccd00d29a0 0x7fccd00b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:20.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.521+0000 7fccd7fa5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd00b2c80 con 0x7fccd00b3130 2024-05-09T10:51:20.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.521+0000 7fccd7fa5640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd00b2ef0 con 0x7fccd00d29a0 2024-05-09T10:51:20.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.521+0000 7fccd7fa5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd00d4ca0 con 0x7fccd0161700 2024-05-09T10:51:20.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.521+0000 7fccd5540640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fccd00d29a0 0x7fccd00b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:20.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.521+0000 7fccd5540640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fccd00d29a0 0x7fccd00b3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57674/0 (socket says 172.21.15.98:57674) 2024-05-09T10:51:20.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.521+0000 7fccd5540640 1 -- 172.21.15.98:0/1877416362 learned_addr learned my addr 172.21.15.98:0/1877416362 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:20.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.521+0000 7fccd5d41640 1 --2- 172.21.15.98:0/1877416362 >> v2:172.21.15.105:3300/0 conn(0x7fccd0161700 0x7fccd00b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:20.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.521+0000 7fccd5540640 1 -- 172.21.15.98:0/1877416362 >> v2:172.21.15.105:3300/0 conn(0x7fccd0161700 msgr2=0x7fccd00b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:20.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.521+0000 7fccd5540640 1 --2- 172.21.15.98:0/1877416362 >> v2:172.21.15.105:3300/0 conn(0x7fccd0161700 0x7fccd00b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.521+0000 7fccd5540640 1 -- 172.21.15.98:0/1877416362 >> v1:172.21.15.98:6789/0 conn(0x7fccd00b3130 legacy=0x7fccd0166e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:20.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.521+0000 7fccd5540640 1 -- 172.21.15.98:0/1877416362 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd00d49c0 con 0x7fccd00d29a0 2024-05-09T10:51:20.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.521+0000 7fccd5d41640 1 --2- 172.21.15.98:0/1877416362 >> v2:172.21.15.105:3300/0 conn(0x7fccd0161700 0x7fccd00b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:20.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.522+0000 7fccd5540640 1 --2- 172.21.15.98:0/1877416362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fccd00d29a0 0x7fccd00b3bc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fccd008ca40 tx=0x7fcccc02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b6cf975f13f52c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:20.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.522+0000 7fccd4d3f640 1 -- 172.21.15.98:0/1877416362 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc037070 con 0x7fccd00d29a0 2024-05-09T10:51:20.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.522+0000 7fccd4d3f640 1 -- 172.21.15.98:0/1877416362 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcccc032440 con 0x7fccd00d29a0 2024-05-09T10:51:20.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.522+0000 7fccd4d3f640 1 -- 172.21.15.98:0/1877416362 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc032810 con 0x7fccd00d29a0 2024-05-09T10:51:20.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.523+0000 7fccd7fa5640 1 -- 172.21.15.98:0/1877416362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fccd00d29a0 msgr2=0x7fccd00b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:20.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.523+0000 7fccd7fa5640 1 --2- 172.21.15.98:0/1877416362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fccd00d29a0 0x7fccd00b3bc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fccd008ca40 tx=0x7fcccc02a150 comp rx=0 tx=0).stop 2024-05-09T10:51:20.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.523+0000 7fccd7fa5640 1 -- 172.21.15.98:0/1877416362 shutdown_connections 2024-05-09T10:51:20.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.523+0000 7fccd7fa5640 1 --2- 172.21.15.98:0/1877416362 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fccd00d29a0 0x7fccd00b3bc0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.523+0000 7fccd7fa5640 1 --2- 172.21.15.98:0/1877416362 >> v2:172.21.15.105:3300/0 conn(0x7fccd0161700 0x7fccd00b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.523+0000 7fccd7fa5640 1 -- 172.21.15.98:0/1877416362 >> 172.21.15.98:0/1877416362 conn(0x7fccd00d3d70 msgr2=0x7fccd00d4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:20.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.523+0000 7fccd7fa5640 1 -- 172.21.15.98:0/1877416362 shutdown_connections 2024-05-09T10:51:20.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.523+0000 7fccd7fa5640 1 -- 172.21.15.98:0/1877416362 wait complete. 2024-05-09T10:51:20.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.524+0000 7fccd7fa5640 1 Processor -- start 2024-05-09T10:51:20.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.524+0000 7fccd7fa5640 1 -- start start 2024-05-09T10:51:20.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.525+0000 7fccd7fa5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccd00b3130 0x7fccd01f7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:20.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.525+0000 7fccd7fa5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fccd00d29a0 0x7fccd00cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:20.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.525+0000 7fccd7fa5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccd0161700 0x7fccd00cee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:20.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.525+0000 7fccd7fa5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd0161ae0 con 0x7fccd00d29a0 2024-05-09T10:51:20.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.525+0000 7fccd7fa5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd00b29b0 con 0x7fccd0161700 2024-05-09T10:51:20.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.525+0000 7fccd7fa5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd00b4bc0 con 0x7fccd00b3130 2024-05-09T10:51:20.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.525+0000 7fccd5540640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fccd00d29a0 0x7fccd00cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.525+0000 7fccd5540640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fccd00d29a0 0x7fccd00cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57684/0 (socket says 172.21.15.98:57684) 2024-05-09T10:51:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.525+0000 7fccd5540640 1 -- 172.21.15.98:0/1988085336 learned_addr learned my addr 172.21.15.98:0/1988085336 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.525+0000 7fccd5540640 1 -- 172.21.15.98:0/1988085336 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccd00b3130 msgr2=0x7fccd01f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.525+0000 7fccd5540640 1 --2- 172.21.15.98:0/1988085336 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccd00b3130 0x7fccd01f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.525+0000 7fccd5540640 1 -- 172.21.15.98:0/1988085336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccd0161700 msgr2=0x7fccd00cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.525+0000 7fccd6542640 1 --2- 172.21.15.98:0/1988085336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccd0161700 0x7fccd00cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.525+0000 7fccd5540640 1 --2- 172.21.15.98:0/1988085336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccd0161700 0x7fccd00cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.525+0000 7fccd5540640 1 -- 172.21.15.98:0/1988085336 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd01f7580 con 0x7fccd00d29a0 2024-05-09T10:51:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.525+0000 7fccd6542640 1 --2- 172.21.15.98:0/1988085336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccd0161700 0x7fccd00cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.525+0000 7fccd5540640 1 --2- 172.21.15.98:0/1988085336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fccd00d29a0 0x7fccd00cd3f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcccc000c00 tx=0x7fcccc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.525+0000 7fccc27fc640 1 -- 172.21.15.98:0/1988085336 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc037040 con 0x7fccd00d29a0 2024-05-09T10:51:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.526+0000 7fccc27fc640 1 -- 172.21.15.98:0/1988085336 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcccc042070 con 0x7fccd00d29a0 2024-05-09T10:51:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.526+0000 7fccd7fa5640 1 -- 172.21.15.98:0/1988085336 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd01f7800 con 0x7fccd00d29a0 2024-05-09T10:51:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.526+0000 7fccc27fc640 1 -- 172.21.15.98:0/1988085336 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc030bb0 con 0x7fccd00d29a0 2024-05-09T10:51:20.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.526+0000 7fccd7fa5640 1 -- 172.21.15.98:0/1988085336 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd01f7dc0 con 0x7fccd00d29a0 2024-05-09T10:51:20.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.526+0000 7fccd7fa5640 1 -- 172.21.15.98:0/1988085336 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd00a1d20 con 0x7fccd00d29a0 2024-05-09T10:51:20.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.527+0000 7fccc27fc640 1 -- 172.21.15.98:0/1988085336 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcccc047460 con 0x7fccd00d29a0 2024-05-09T10:51:20.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.527+0000 7fccc27fc640 1 --2- 172.21.15.98:0/1988085336 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fccb406b500 0x7fccb406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:20.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.528+0000 7fccc27fc640 1 -- 172.21.15.98:0/1988085336 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcccc0aad40 con 0x7fccd00d29a0 2024-05-09T10:51:20.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.528+0000 7fccd5d41640 1 --2- 172.21.15.98:0/1988085336 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fccb406b500 0x7fccb406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:20.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.528+0000 7fccd5d41640 1 --2- 172.21.15.98:0/1988085336 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fccb406b500 0x7fccb406d9c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fccc4006fd0 tx=0x7fccc4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:20.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.532+0000 7fccc27fc640 1 -- 172.21.15.98:0/1988085336 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fccd01f7580 con 0x7fccd00d29a0 2024-05-09T10:51:20.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.728+0000 7fccd7fa5640 1 -- 172.21.15.98:0/1988085336 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol17470", "group_name": "subvol_grp59180", "target": ["mon-mgr", ""]}) v1 -- 0x7fccd015e990 con 0x7fccb406b500 2024-05-09T10:51:20.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.756+0000 7fccc27fc640 1 -- 172.21.15.98:0/1988085336 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fccd015e990 con 0x7fccb406b500 2024-05-09T10:51:20.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.758+0000 7fccd7fa5640 1 -- 172.21.15.98:0/1988085336 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fccb406b500 msgr2=0x7fccb406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:20.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.758+0000 7fccd7fa5640 1 --2- 172.21.15.98:0/1988085336 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fccb406b500 0x7fccb406d9c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fccc4006fd0 tx=0x7fccc4008040 comp rx=0 tx=0).stop 2024-05-09T10:51:20.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.758+0000 7fccd7fa5640 1 -- 172.21.15.98:0/1988085336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fccd00d29a0 msgr2=0x7fccd00cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:20.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.758+0000 7fccd7fa5640 1 --2- 172.21.15.98:0/1988085336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fccd00d29a0 0x7fccd00cd3f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcccc000c00 tx=0x7fcccc0046c0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.758+0000 7fccd7fa5640 1 -- 172.21.15.98:0/1988085336 shutdown_connections 2024-05-09T10:51:20.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.758+0000 7fccd7fa5640 1 --2- 172.21.15.98:0/1988085336 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fccb406b500 0x7fccb406d9c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.758+0000 7fccd7fa5640 1 --2- 172.21.15.98:0/1988085336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccd0161700 0x7fccd00cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.758+0000 7fccd7fa5640 1 --2- 172.21.15.98:0/1988085336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fccd00d29a0 0x7fccd00cd3f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.758+0000 7fccd7fa5640 1 --2- 172.21.15.98:0/1988085336 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fccd00b3130 0x7fccd01f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.758+0000 7fccd7fa5640 1 -- 172.21.15.98:0/1988085336 >> 172.21.15.98:0/1988085336 conn(0x7fccd00d3d70 msgr2=0x7fccd00b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:20.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.758+0000 7fccd7fa5640 1 -- 172.21.15.98:0/1988085336 shutdown_connections 2024-05-09T10:51:20.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.758+0000 7fccd7fa5640 1 -- 172.21.15.98:0/1988085336 wait complete. 2024-05-09T10:51:20.762 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol28581 --group_name subvol_grp59180 2024-05-09T10:51:20.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.874+0000 7fdb8353b640 1 Processor -- start 2024-05-09T10:51:20.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.874+0000 7fdb8353b640 1 -- start start 2024-05-09T10:51:20.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.875+0000 7fdb8353b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb7c0ae370 0x7fdb7c168830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:20.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.875+0000 7fdb8353b640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb7c0afa70 con 0x7fdb7c0ad880 2024-05-09T10:51:20.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.875+0000 7fdb8353b640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb7c15e200 con 0x7fdb7c0b5080 2024-05-09T10:51:20.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.875+0000 7fdb8353b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c15e4c0 con 0x7fdb7c0ae370 2024-05-09T10:51:20.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.875+0000 7fdb812d7640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fdb7c0ad880 0x7fdb7c0b2fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36904/0 (socket says 172.21.15.98:36904) 2024-05-09T10:51:20.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.875+0000 7fdb812d7640 1 -- 172.21.15.98:0/3403358015 learned_addr learned my addr 172.21.15.98:0/3403358015 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:20.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb81ad8640 1 --2- 172.21.15.98:0/3403358015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb7c0ae370 0x7fdb7c168830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:20.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb73fff640 1 -- 172.21.15.98:0/3403358015 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2217981205 0 0) 0x7fdb78001110 con 0x7fdb7c0ad880 2024-05-09T10:51:20.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb73fff640 1 -- 172.21.15.98:0/3403358015 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb68003520 con 0x7fdb7c0ad880 2024-05-09T10:51:20.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb73fff640 1 -- 172.21.15.98:0/3403358015 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1960318815 0 0) 0x7fdb68003520 con 0x7fdb7c0ad880 2024-05-09T10:51:20.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb73fff640 1 -- 172.21.15.98:0/3403358015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb7c0ae370 msgr2=0x7fdb7c168830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:20.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb73fff640 1 --2- 172.21.15.98:0/3403358015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb7c0ae370 0x7fdb7c168830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb73fff640 1 -- 172.21.15.98:0/3403358015 >> v1:172.21.15.105:6789/0 conn(0x7fdb7c0b5080 legacy=0x7fdb7c0adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:20.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb73fff640 1 -- 172.21.15.98:0/3403358015 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c15e7a0 con 0x7fdb7c0ad880 2024-05-09T10:51:20.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb73fff640 1 -- 172.21.15.98:0/3403358015 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7fdb78001840 con 0x7fdb7c0ad880 2024-05-09T10:51:20.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb73fff640 1 -- 172.21.15.98:0/3403358015 >> v1:172.21.15.98:6789/0 conn(0x7fdb7c0ad880 legacy=0x7fdb7c0b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:20.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb73fff640 1 --2- 172.21.15.98:0/3403358015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb7c0ae370 0x7fdb680039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:20.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb73fff640 1 --2- 172.21.15.98:0/3403358015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb68003f20 0x7fdb68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:20.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb73fff640 1 --2- 172.21.15.98:0/3403358015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb68004d40 0x7fdb680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:20.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb73fff640 1 -- 172.21.15.98:0/3403358015 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c15e4c0 con 0x7fdb68003f20 2024-05-09T10:51:20.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb73fff640 1 -- 172.21.15.98:0/3403358015 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6800bb80 con 0x7fdb68004d40 2024-05-09T10:51:20.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb73fff640 1 -- 172.21.15.98:0/3403358015 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6800bd80 con 0x7fdb7c0ae370 2024-05-09T10:51:20.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb80ad6640 1 --2- 172.21.15.98:0/3403358015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb68003f20 0x7fdb68004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:20.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb812d7640 1 --2- 172.21.15.98:0/3403358015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb7c0ae370 0x7fdb680039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:20.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb81ad8640 1 --2- 172.21.15.98:0/3403358015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb68004d40 0x7fdb680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:20.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb80ad6640 1 -- 172.21.15.98:0/3403358015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb7c0ae370 msgr2=0x7fdb680039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:20.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb80ad6640 1 --2- 172.21.15.98:0/3403358015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb7c0ae370 0x7fdb680039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb80ad6640 1 -- 172.21.15.98:0/3403358015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb68004d40 msgr2=0x7fdb680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:20.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb80ad6640 1 --2- 172.21.15.98:0/3403358015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb68004d40 0x7fdb680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb80ad6640 1 -- 172.21.15.98:0/3403358015 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb68003520 con 0x7fdb68003f20 2024-05-09T10:51:20.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.876+0000 7fdb81ad8640 1 --2- 172.21.15.98:0/3403358015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb68004d40 0x7fdb680051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:20.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.877+0000 7fdb80ad6640 1 --2- 172.21.15.98:0/3403358015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb68003f20 0x7fdb68004800 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c1552f0 tx=0x7fdb68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96977d148cce0b70 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:20.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.877+0000 7fdb73fff640 1 -- 172.21.15.98:0/3403358015 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c037070 con 0x7fdb68003f20 2024-05-09T10:51:20.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.877+0000 7fdb73fff640 1 -- 172.21.15.98:0/3403358015 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb6c02ab90 con 0x7fdb68003f20 2024-05-09T10:51:20.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.877+0000 7fdb73fff640 1 -- 172.21.15.98:0/3403358015 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c001430 con 0x7fdb68003f20 2024-05-09T10:51:20.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.878+0000 7fdb8353b640 1 -- 172.21.15.98:0/3403358015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb68003f20 msgr2=0x7fdb68004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:20.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.878+0000 7fdb8353b640 1 --2- 172.21.15.98:0/3403358015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb68003f20 0x7fdb68004800 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c1552f0 tx=0x7fdb68005830 comp rx=0 tx=0).stop 2024-05-09T10:51:20.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.878+0000 7fdb81ad8640 1 -- 172.21.15.98:0/3403358015 reap_dead start 2024-05-09T10:51:20.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.878+0000 7fdb8353b640 1 -- 172.21.15.98:0/3403358015 shutdown_connections 2024-05-09T10:51:20.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.878+0000 7fdb8353b640 1 -- 172.21.15.98:0/3403358015 >> 172.21.15.98:0/3403358015 conn(0x7fdb7c06cbf0 msgr2=0x7fdb7c06d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:20.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.878+0000 7fdb8353b640 1 -- 172.21.15.98:0/3403358015 shutdown_connections 2024-05-09T10:51:20.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.878+0000 7fdb8353b640 1 -- 172.21.15.98:0/3403358015 wait complete. 2024-05-09T10:51:20.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.879+0000 7fdb8353b640 1 Processor -- start 2024-05-09T10:51:20.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.879+0000 7fdb8353b640 1 -- start start 2024-05-09T10:51:20.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.880+0000 7fdb8353b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb68003f20 0x7fdb7c0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:20.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.880+0000 7fdb8353b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb7c1f4410 0x7fdb7c14ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:20.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.880+0000 7fdb8353b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb7c1564b0 0x7fdb7c071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:20.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.880+0000 7fdb8353b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c06bbf0 con 0x7fdb68003f20 2024-05-09T10:51:20.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.880+0000 7fdb8353b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c15ed60 con 0x7fdb7c1564b0 2024-05-09T10:51:20.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.880+0000 7fdb8353b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c06bd70 con 0x7fdb7c1f4410 2024-05-09T10:51:20.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.880+0000 7fdb812d7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb68003f20 0x7fdb7c0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:20.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.880+0000 7fdb812d7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb68003f20 0x7fdb7c0cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57698/0 (socket says 172.21.15.98:57698) 2024-05-09T10:51:20.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.880+0000 7fdb812d7640 1 -- 172.21.15.98:0/4156436988 learned_addr learned my addr 172.21.15.98:0/4156436988 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:20.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.880+0000 7fdb80ad6640 1 --2- 172.21.15.98:0/4156436988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb7c1f4410 0x7fdb7c14ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:20.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.880+0000 7fdb81ad8640 1 --2- 172.21.15.98:0/4156436988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb7c1564b0 0x7fdb7c071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:20.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.880+0000 7fdb812d7640 1 -- 172.21.15.98:0/4156436988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb7c1f4410 msgr2=0x7fdb7c14ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:20.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.880+0000 7fdb812d7640 1 --2- 172.21.15.98:0/4156436988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb7c1f4410 0x7fdb7c14ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.880+0000 7fdb812d7640 1 -- 172.21.15.98:0/4156436988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb7c1564b0 msgr2=0x7fdb7c071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:20.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.880+0000 7fdb812d7640 1 --2- 172.21.15.98:0/4156436988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb7c1564b0 0x7fdb7c071ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:20.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.880+0000 7fdb812d7640 1 -- 172.21.15.98:0/4156436988 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c1f4980 con 0x7fdb68003f20 2024-05-09T10:51:20.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.880+0000 7fdb812d7640 1 --2- 172.21.15.98:0/4156436988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb68003f20 0x7fdb7c0cd8a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fdb78004ef0 tx=0x7fdb78007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:20.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.880+0000 7fdb71ffb640 1 -- 172.21.15.98:0/4156436988 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7800f750 con 0x7fdb68003f20 2024-05-09T10:51:20.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.880+0000 7fdb71ffb640 1 -- 172.21.15.98:0/4156436988 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb78018070 con 0x7fdb68003f20 2024-05-09T10:51:20.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.880+0000 7fdb8353b640 1 -- 172.21.15.98:0/4156436988 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb7c1f4c60 con 0x7fdb68003f20 2024-05-09T10:51:20.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.880+0000 7fdb71ffb640 1 -- 172.21.15.98:0/4156436988 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7800fb10 con 0x7fdb68003f20 2024-05-09T10:51:20.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.881+0000 7fdb8353b640 1 -- 172.21.15.98:0/4156436988 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb7c1f5220 con 0x7fdb68003f20 2024-05-09T10:51:20.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.881+0000 7fdb8353b640 1 -- 172.21.15.98:0/4156436988 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb7c0b3660 con 0x7fdb68003f20 2024-05-09T10:51:20.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.882+0000 7fdb71ffb640 1 -- 172.21.15.98:0/4156436988 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb7800fcb0 con 0x7fdb68003f20 2024-05-09T10:51:20.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.882+0000 7fdb71ffb640 1 --2- 172.21.15.98:0/4156436988 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fdb6006b500 0x7fdb6006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:20.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.883+0000 7fdb71ffb640 1 -- 172.21.15.98:0/4156436988 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb7808a130 con 0x7fdb68003f20 2024-05-09T10:51:20.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.883+0000 7fdb80ad6640 1 --2- 172.21.15.98:0/4156436988 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fdb6006b500 0x7fdb6006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:20.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.883+0000 7fdb80ad6640 1 --2- 172.21.15.98:0/4156436988 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fdb6006b500 0x7fdb6006d9c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c034000 tx=0x7fdb6c009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:20.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:20.887+0000 7fdb71ffb640 1 -- 172.21.15.98:0/4156436988 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdb7c0b3660 con 0x7fdb68003f20 2024-05-09T10:51:21.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.080+0000 7fdb8353b640 1 -- 172.21.15.98:0/4156436988 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol28581", "group_name": "subvol_grp59180", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb7c14f240 con 0x7fdb6006b500 2024-05-09T10:51:21.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.109+0000 7fdb71ffb640 1 -- 172.21.15.98:0/4156436988 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdb6c005020 con 0x7fdb6006b500 2024-05-09T10:51:21.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.111+0000 7fdb8353b640 1 -- 172.21.15.98:0/4156436988 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fdb6006b500 msgr2=0x7fdb6006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:21.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.111+0000 7fdb8353b640 1 --2- 172.21.15.98:0/4156436988 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fdb6006b500 0x7fdb6006d9c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c034000 tx=0x7fdb6c009030 comp rx=0 tx=0).stop 2024-05-09T10:51:21.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.111+0000 7fdb8353b640 1 -- 172.21.15.98:0/4156436988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb68003f20 msgr2=0x7fdb7c0cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:21.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.111+0000 7fdb8353b640 1 --2- 172.21.15.98:0/4156436988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb68003f20 0x7fdb7c0cd8a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fdb78004ef0 tx=0x7fdb78007430 comp rx=0 tx=0).stop 2024-05-09T10:51:21.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.111+0000 7fdb8353b640 1 -- 172.21.15.98:0/4156436988 shutdown_connections 2024-05-09T10:51:21.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.111+0000 7fdb8353b640 1 --2- 172.21.15.98:0/4156436988 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fdb6006b500 0x7fdb6006d9c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.111+0000 7fdb8353b640 1 --2- 172.21.15.98:0/4156436988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb7c1564b0 0x7fdb7c071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.111+0000 7fdb8353b640 1 --2- 172.21.15.98:0/4156436988 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb7c1f4410 0x7fdb7c14ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.111+0000 7fdb8353b640 1 --2- 172.21.15.98:0/4156436988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb68003f20 0x7fdb7c0cd8a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.111+0000 7fdb8353b640 1 -- 172.21.15.98:0/4156436988 >> 172.21.15.98:0/4156436988 conn(0x7fdb7c06cbf0 msgr2=0x7fdb7c0b0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:21.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.111+0000 7fdb8353b640 1 -- 172.21.15.98:0/4156436988 shutdown_connections 2024-05-09T10:51:21.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.111+0000 7fdb8353b640 1 -- 172.21.15.98:0/4156436988 wait complete. 2024-05-09T10:51:21.115 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol27112 --group_name subvol_grp59180 2024-05-09T10:51:21.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.226+0000 7f1bd1fa5640 1 Processor -- start 2024-05-09T10:51:21.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.226+0000 7f1bd1fa5640 1 -- start start 2024-05-09T10:51:21.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.227+0000 7f1bd1fa5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1bcc159740 0x7f1bcc0c6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:21.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.227+0000 7f1bd1fa5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1bcc0aede0 0x7f1bcc0c4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:21.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.227+0000 7f1bd1fa5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bcc06af30 con 0x7f1bcc15c9a0 2024-05-09T10:51:21.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.227+0000 7f1bd1fa5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bcc06b210 con 0x7f1bcc159740 2024-05-09T10:51:21.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.227+0000 7f1bd1fa5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc06b4f0 con 0x7f1bcc0aede0 2024-05-09T10:51:21.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.227+0000 7f1bcaffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f1bcc15c9a0 0x7f1bcc0ae6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36916/0 (socket says 172.21.15.98:36916) 2024-05-09T10:51:21.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.227+0000 7f1bcaffd640 1 -- 172.21.15.98:0/4291720999 learned_addr learned my addr 172.21.15.98:0/4291720999 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:21.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.227+0000 7f1bca7fc640 1 -- 172.21.15.98:0/4291720999 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419153729 0 0) 0x7f1bcc06af30 con 0x7f1bcc15c9a0 2024-05-09T10:51:21.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.227+0000 7f1bcb7fe640 1 --2- 172.21.15.98:0/4291720999 >> v2:172.21.15.105:3300/0 conn(0x7f1bcc159740 0x7f1bcc0c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:21.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.227+0000 7f1bcbfff640 1 --2- 172.21.15.98:0/4291720999 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1bcc0aede0 0x7f1bcc0c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:21.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.227+0000 7f1bca7fc640 1 -- 172.21.15.98:0/4291720999 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bb4003520 con 0x7f1bcc15c9a0 2024-05-09T10:51:21.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.228+0000 7f1bca7fc640 1 -- 172.21.15.98:0/4291720999 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4189034226 0 0) 0x7f1bb4003520 con 0x7f1bcc15c9a0 2024-05-09T10:51:21.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.228+0000 7f1bca7fc640 1 -- 172.21.15.98:0/4291720999 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1bcc0aede0 msgr2=0x7f1bcc0c4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:21.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.228+0000 7f1bca7fc640 1 --2- 172.21.15.98:0/4291720999 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1bcc0aede0 0x7f1bcc0c4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.228+0000 7f1bca7fc640 1 -- 172.21.15.98:0/4291720999 >> v2:172.21.15.105:3300/0 conn(0x7f1bcc159740 msgr2=0x7f1bcc0c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:21.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.228+0000 7f1bca7fc640 1 --2- 172.21.15.98:0/4291720999 >> v2:172.21.15.105:3300/0 conn(0x7f1bcc159740 0x7f1bcc0c6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.228+0000 7f1bca7fc640 1 -- 172.21.15.98:0/4291720999 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bcc15c480 con 0x7f1bcc15c9a0 2024-05-09T10:51:21.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.228+0000 7f1bca7fc640 1 -- 172.21.15.98:0/4291720999 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f1bc0001690 con 0x7f1bcc15c9a0 2024-05-09T10:51:21.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.228+0000 7f1bca7fc640 1 -- 172.21.15.98:0/4291720999 >> v1:172.21.15.98:6789/0 conn(0x7f1bcc15c9a0 legacy=0x7f1bcc0ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:21.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.228+0000 7f1bca7fc640 1 --2- 172.21.15.98:0/4291720999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1bb40039b0 0x7f1bb4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:21.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.228+0000 7f1bca7fc640 1 --2- 172.21.15.98:0/4291720999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bb4004800 0x7f1bb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:21.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.228+0000 7f1bca7fc640 1 --2- 172.21.15.98:0/4291720999 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1bcc0aede0 0x7f1bb40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:21.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.228+0000 7f1bca7fc640 1 -- 172.21.15.98:0/4291720999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc06b210 con 0x7f1bb4004800 2024-05-09T10:51:21.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.228+0000 7f1bca7fc640 1 -- 172.21.15.98:0/4291720999 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc06b4f0 con 0x7f1bb40039b0 2024-05-09T10:51:21.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.228+0000 7f1bca7fc640 1 -- 172.21.15.98:0/4291720999 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb400b9c0 con 0x7f1bcc0aede0 2024-05-09T10:51:21.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.228+0000 7f1bcaffd640 1 --2- 172.21.15.98:0/4291720999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bb4004800 0x7f1bb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:21.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.228+0000 7f1bcb7fe640 1 --2- 172.21.15.98:0/4291720999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1bb40039b0 0x7f1bb4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:21.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.228+0000 7f1bcbfff640 1 --2- 172.21.15.98:0/4291720999 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1bcc0aede0 0x7f1bb40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:21.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.228+0000 7f1bcaffd640 1 -- 172.21.15.98:0/4291720999 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1bcc0aede0 msgr2=0x7f1bb40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:21.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.228+0000 7f1bcaffd640 1 --2- 172.21.15.98:0/4291720999 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1bcc0aede0 0x7f1bb40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.228+0000 7f1bcaffd640 1 -- 172.21.15.98:0/4291720999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1bb40039b0 msgr2=0x7f1bb4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:21.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.228+0000 7f1bcaffd640 1 --2- 172.21.15.98:0/4291720999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1bb40039b0 0x7f1bb4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.228+0000 7f1bcaffd640 1 -- 172.21.15.98:0/4291720999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb4003520 con 0x7f1bb4004800 2024-05-09T10:51:21.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.229+0000 7f1bcaffd640 1 --2- 172.21.15.98:0/4291720999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bb4004800 0x7f1bb4004c80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000c210 tx=0x7f1bb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb2fa20d5bc87dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:21.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.229+0000 7f1bca7fc640 1 -- 172.21.15.98:0/4291720999 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0007d30 con 0x7f1bb4004800 2024-05-09T10:51:21.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.229+0000 7f1bca7fc640 1 -- 172.21.15.98:0/4291720999 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc00058d0 con 0x7f1bb4004800 2024-05-09T10:51:21.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.229+0000 7f1bca7fc640 1 -- 172.21.15.98:0/4291720999 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0005ca0 con 0x7f1bb4004800 2024-05-09T10:51:21.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.229+0000 7f1bd1fa5640 1 -- 172.21.15.98:0/4291720999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bb4004800 msgr2=0x7f1bb4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:21.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.229+0000 7f1bd1fa5640 1 --2- 172.21.15.98:0/4291720999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bb4004800 0x7f1bb4004c80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000c210 tx=0x7f1bb4005830 comp rx=0 tx=0).stop 2024-05-09T10:51:21.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.229+0000 7f1bcbfff640 1 -- 172.21.15.98:0/4291720999 reap_dead start 2024-05-09T10:51:21.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.229+0000 7f1bd1fa5640 1 -- 172.21.15.98:0/4291720999 shutdown_connections 2024-05-09T10:51:21.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.229+0000 7f1bd1fa5640 1 -- 172.21.15.98:0/4291720999 >> 172.21.15.98:0/4291720999 conn(0x7f1bcc0cf760 msgr2=0x7f1bcc0cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:21.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.229+0000 7f1bd1fa5640 1 -- 172.21.15.98:0/4291720999 shutdown_connections 2024-05-09T10:51:21.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.229+0000 7f1bd1fa5640 1 -- 172.21.15.98:0/4291720999 wait complete. 2024-05-09T10:51:21.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.231+0000 7f1bd1fa5640 1 Processor -- start 2024-05-09T10:51:21.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.231+0000 7f1bd1fa5640 1 -- start start 2024-05-09T10:51:21.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.232+0000 7f1bd1fa5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1bb4004800 0x7f1bcc15afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:21.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.232+0000 7f1bd1fa5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bcc0c6830 0x7f1bcc158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:21.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.232+0000 7f1bd1fa5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1bcc0b5000 0x7f1bcc0b5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:21.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.232+0000 7f1bd1fa5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc15c170 con 0x7f1bcc0c6830 2024-05-09T10:51:21.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.232+0000 7f1bd1fa5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc159b20 con 0x7f1bcc0b5000 2024-05-09T10:51:21.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.232+0000 7f1bd1fa5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc06abe0 con 0x7f1bb4004800 2024-05-09T10:51:21.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.232+0000 7f1bcaffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bcc0c6830 0x7f1bcc158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:21.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.232+0000 7f1bcaffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bcc0c6830 0x7f1bcc158320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57714/0 (socket says 172.21.15.98:57714) 2024-05-09T10:51:21.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.232+0000 7f1bcaffd640 1 -- 172.21.15.98:0/4099388090 learned_addr learned my addr 172.21.15.98:0/4099388090 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:21.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.232+0000 7f1bcbfff640 1 --2- 172.21.15.98:0/4099388090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1bcc0b5000 0x7f1bcc0b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:21.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.232+0000 7f1bcb7fe640 1 --2- 172.21.15.98:0/4099388090 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1bb4004800 0x7f1bcc15afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:21.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.232+0000 7f1bcaffd640 1 -- 172.21.15.98:0/4099388090 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1bb4004800 msgr2=0x7f1bcc15afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:21.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.232+0000 7f1bcaffd640 1 --2- 172.21.15.98:0/4099388090 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1bb4004800 0x7f1bcc15afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.232+0000 7f1bcaffd640 1 -- 172.21.15.98:0/4099388090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1bcc0b5000 msgr2=0x7f1bcc0b5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:21.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.232+0000 7f1bcaffd640 1 --2- 172.21.15.98:0/4099388090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1bcc0b5000 0x7f1bcc0b5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.232+0000 7f1bcaffd640 1 -- 172.21.15.98:0/4099388090 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bcc0b59c0 con 0x7f1bcc0c6830 2024-05-09T10:51:21.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.232+0000 7f1bcb7fe640 1 --2- 172.21.15.98:0/4099388090 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1bb4004800 0x7f1bcc15afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:21.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.232+0000 7f1bcaffd640 1 --2- 172.21.15.98:0/4099388090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bcc0c6830 0x7f1bcc158320 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000c330 tx=0x7f1bcc15c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:21.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.232+0000 7f1bb3fff640 1 -- 172.21.15.98:0/4099388090 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0038040 con 0x7f1bcc0c6830 2024-05-09T10:51:21.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.232+0000 7f1bb3fff640 1 -- 172.21.15.98:0/4099388090 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc0042070 con 0x7f1bcc0c6830 2024-05-09T10:51:21.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.232+0000 7f1bb3fff640 1 -- 172.21.15.98:0/4099388090 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0007470 con 0x7f1bcc0c6830 2024-05-09T10:51:21.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.232+0000 7f1bd1fa5640 1 -- 172.21.15.98:0/4099388090 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bcc0c7f10 con 0x7f1bcc0c6830 2024-05-09T10:51:21.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.232+0000 7f1bd1fa5640 1 -- 172.21.15.98:0/4099388090 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bcc1f29a0 con 0x7f1bcc0c6830 2024-05-09T10:51:21.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.233+0000 7f1bd1fa5640 1 -- 172.21.15.98:0/4099388090 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bcc067d50 con 0x7f1bcc0c6830 2024-05-09T10:51:21.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.234+0000 7f1bb3fff640 1 -- 172.21.15.98:0/4099388090 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1bc0047460 con 0x7f1bcc0c6830 2024-05-09T10:51:21.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.234+0000 7f1bb3fff640 1 --2- 172.21.15.98:0/4099388090 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1bb806b4d0 0x7f1bb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:21.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.234+0000 7f1bb3fff640 1 -- 172.21.15.98:0/4099388090 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1bc00aa020 con 0x7f1bcc0c6830 2024-05-09T10:51:21.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.234+0000 7f1bcb7fe640 1 --2- 172.21.15.98:0/4099388090 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1bb806b4d0 0x7f1bb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:21.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.235+0000 7f1bcb7fe640 1 --2- 172.21.15.98:0/4099388090 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1bb806b4d0 0x7f1bb806d990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc009bc0 tx=0x7f1bbc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:21.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.238+0000 7f1bb3fff640 1 -- 172.21.15.98:0/4099388090 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1bcc067d50 con 0x7f1bcc0c6830 2024-05-09T10:51:21.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.438+0000 7f1bd1fa5640 1 -- 172.21.15.98:0/4099388090 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol27112", "group_name": "subvol_grp59180", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bcc072c80 con 0x7f1bb806b4d0 2024-05-09T10:51:21.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.456+0000 7f1bb3fff640 1 -- 172.21.15.98:0/4099388090 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mgrmap(e 6) v1 ==== 89831+0+0 (secure 0 0 0) 0x7f1bc0075070 con 0x7f1bcc0c6830 2024-05-09T10:51:21.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.467+0000 7f1bb3fff640 1 -- 172.21.15.98:0/4099388090 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1bcc072c80 con 0x7f1bb806b4d0 2024-05-09T10:51:21.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.469+0000 7f1bd1fa5640 1 -- 172.21.15.98:0/4099388090 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1bb806b4d0 msgr2=0x7f1bb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:21.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.469+0000 7f1bd1fa5640 1 --2- 172.21.15.98:0/4099388090 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1bb806b4d0 0x7f1bb806d990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc009bc0 tx=0x7f1bbc008040 comp rx=0 tx=0).stop 2024-05-09T10:51:21.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.469+0000 7f1bd1fa5640 1 -- 172.21.15.98:0/4099388090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bcc0c6830 msgr2=0x7f1bcc158320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:21.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.469+0000 7f1bd1fa5640 1 --2- 172.21.15.98:0/4099388090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bcc0c6830 0x7f1bcc158320 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000c330 tx=0x7f1bcc15c650 comp rx=0 tx=0).stop 2024-05-09T10:51:21.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.469+0000 7f1bd1fa5640 1 -- 172.21.15.98:0/4099388090 shutdown_connections 2024-05-09T10:51:21.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.469+0000 7f1bd1fa5640 1 --2- 172.21.15.98:0/4099388090 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1bb806b4d0 0x7f1bb806d990 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.469+0000 7f1bd1fa5640 1 --2- 172.21.15.98:0/4099388090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1bcc0b5000 0x7f1bcc0b5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.469+0000 7f1bd1fa5640 1 --2- 172.21.15.98:0/4099388090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1bcc0c6830 0x7f1bcc158320 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.469+0000 7f1bd1fa5640 1 --2- 172.21.15.98:0/4099388090 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1bb4004800 0x7f1bcc15afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.469+0000 7f1bd1fa5640 1 -- 172.21.15.98:0/4099388090 >> 172.21.15.98:0/4099388090 conn(0x7f1bcc0cf760 msgr2=0x7f1bcc0cc170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:21.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.469+0000 7f1bd1fa5640 1 -- 172.21.15.98:0/4099388090 shutdown_connections 2024-05-09T10:51:21.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.469+0000 7f1bd1fa5640 1 -- 172.21.15.98:0/4099388090 wait complete. 2024-05-09T10:51:21.473 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol17470 subvol_grp59180 2024-05-09T10:51:21.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.583+0000 7f0ae493b640 1 Processor -- start 2024-05-09T10:51:21.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.583+0000 7f0ae493b640 1 -- start start 2024-05-09T10:51:21.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.584+0000 7f0ae493b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0ae015f270 0x7f0ae00cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:21.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.584+0000 7f0ae493b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ae00af770 0x7f0ae006abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:21.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.584+0000 7f0ae493b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0ae006b120 0x7f0ae00713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:21.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.584+0000 7f0ae493b640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ae00cd830 con 0x7f0ae006b120 2024-05-09T10:51:21.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.584+0000 7f0ae493b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ae00cf720 con 0x7f0ae015f270 2024-05-09T10:51:21.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.584+0000 7f0ae493b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae00afc00 con 0x7f0ae00af770 2024-05-09T10:51:21.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.585+0000 7f0aded9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0ae006b120 0x7f0ae00713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:21.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.585+0000 7f0aded9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0ae006b120 0x7f0ae00713f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57722/0 (socket says 172.21.15.98:57722) 2024-05-09T10:51:21.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.585+0000 7f0aded9d640 1 -- 172.21.15.98:0/1278753887 learned_addr learned my addr 172.21.15.98:0/1278753887 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:21.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.585+0000 7f0addd9b640 1 --2- 172.21.15.98:0/1278753887 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ae00af770 0x7f0ae006abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:21.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.585+0000 7f0ade59c640 1 --2- 172.21.15.98:0/1278753887 >> v2:172.21.15.105:3300/0 conn(0x7f0ae015f270 0x7f0ae00cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:21.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.585+0000 7f0aded9d640 1 -- 172.21.15.98:0/1278753887 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ae00af770 msgr2=0x7f0ae006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:21.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.585+0000 7f0aded9d640 1 --2- 172.21.15.98:0/1278753887 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ae00af770 0x7f0ae006abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.585+0000 7f0aded9d640 1 -- 172.21.15.98:0/1278753887 >> v2:172.21.15.105:3300/0 conn(0x7f0ae015f270 msgr2=0x7f0ae00cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:21.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.585+0000 7f0aded9d640 1 --2- 172.21.15.98:0/1278753887 >> v2:172.21.15.105:3300/0 conn(0x7f0ae015f270 0x7f0ae00cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.585+0000 7f0aded9d640 1 -- 172.21.15.98:0/1278753887 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae00afef0 con 0x7f0ae006b120 2024-05-09T10:51:21.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.586+0000 7f0aded9d640 1 --2- 172.21.15.98:0/1278753887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae006b120 0x7f0ae00713f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0074ba0 tx=0x7f0ad802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc143fdd5a952a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:21.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.586+0000 7f0add59a640 1 -- 172.21.15.98:0/1278753887 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8037070 con 0x7f0ae006b120 2024-05-09T10:51:21.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.586+0000 7f0add59a640 1 -- 172.21.15.98:0/1278753887 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad8032440 con 0x7f0ae006b120 2024-05-09T10:51:21.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.586+0000 7f0add59a640 1 -- 172.21.15.98:0/1278753887 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8032810 con 0x7f0ae006b120 2024-05-09T10:51:21.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.586+0000 7f0ae493b640 1 -- 172.21.15.98:0/1278753887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae006b120 msgr2=0x7f0ae00713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:21.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.586+0000 7f0ae493b640 1 --2- 172.21.15.98:0/1278753887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae006b120 0x7f0ae00713f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0074ba0 tx=0x7f0ad802a1e0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.586+0000 7f0ae493b640 1 -- 172.21.15.98:0/1278753887 shutdown_connections 2024-05-09T10:51:21.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.586+0000 7f0ae493b640 1 --2- 172.21.15.98:0/1278753887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae006b120 0x7f0ae00713f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.586+0000 7f0ae493b640 1 --2- 172.21.15.98:0/1278753887 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ae00af770 0x7f0ae006abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.586+0000 7f0ae493b640 1 --2- 172.21.15.98:0/1278753887 >> v2:172.21.15.105:3300/0 conn(0x7f0ae015f270 0x7f0ae00cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.586+0000 7f0ae493b640 1 -- 172.21.15.98:0/1278753887 >> 172.21.15.98:0/1278753887 conn(0x7f0ae00d0860 msgr2=0x7f0ae00d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:21.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.586+0000 7f0ae493b640 1 -- 172.21.15.98:0/1278753887 shutdown_connections 2024-05-09T10:51:21.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.586+0000 7f0ae493b640 1 -- 172.21.15.98:0/1278753887 wait complete. 2024-05-09T10:51:21.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.588+0000 7f0ae493b640 1 Processor -- start 2024-05-09T10:51:21.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.588+0000 7f0ae493b640 1 -- start start 2024-05-09T10:51:21.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.589+0000 7f0ae493b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ae006b120 0x7f0ae0158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:21.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.589+0000 7f0ae493b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ae015f270 0x7f0ae01f12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:21.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.589+0000 7f0ae493b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae0154b00 0x7f0ae0156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:21.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.589+0000 7f0ae493b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae00d1610 con 0x7f0ae0154b00 2024-05-09T10:51:21.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.589+0000 7f0ae493b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae00cf970 con 0x7f0ae006b120 2024-05-09T10:51:21.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.589+0000 7f0ae493b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae00cfe90 con 0x7f0ae015f270 2024-05-09T10:51:21.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.589+0000 7f0aded9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae0154b00 0x7f0ae0156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:21.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.589+0000 7f0aded9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae0154b00 0x7f0ae0156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57724/0 (socket says 172.21.15.98:57724) 2024-05-09T10:51:21.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.589+0000 7f0aded9d640 1 -- 172.21.15.98:0/1037508555 learned_addr learned my addr 172.21.15.98:0/1037508555 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:21.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.589+0000 7f0ade59c640 1 --2- 172.21.15.98:0/1037508555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ae006b120 0x7f0ae0158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:21.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.589+0000 7f0addd9b640 1 --2- 172.21.15.98:0/1037508555 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ae015f270 0x7f0ae01f12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:21.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.589+0000 7f0aded9d640 1 -- 172.21.15.98:0/1037508555 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ae015f270 msgr2=0x7f0ae01f12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:21.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.589+0000 7f0aded9d640 1 --2- 172.21.15.98:0/1037508555 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ae015f270 0x7f0ae01f12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.589+0000 7f0aded9d640 1 -- 172.21.15.98:0/1037508555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ae006b120 msgr2=0x7f0ae0158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:21.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.589+0000 7f0aded9d640 1 --2- 172.21.15.98:0/1037508555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ae006b120 0x7f0ae0158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.589+0000 7f0aded9d640 1 -- 172.21.15.98:0/1037508555 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae01595b0 con 0x7f0ae0154b00 2024-05-09T10:51:21.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.589+0000 7f0aded9d640 1 --2- 172.21.15.98:0/1037508555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae0154b00 0x7f0ae0156630 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80090b0 tx=0x7f0ad80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:21.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.589+0000 7f0ad6ffd640 1 -- 172.21.15.98:0/1037508555 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8041020 con 0x7f0ae0154b00 2024-05-09T10:51:21.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.589+0000 7f0ad6ffd640 1 -- 172.21.15.98:0/1037508555 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad80306c0 con 0x7f0ae0154b00 2024-05-09T10:51:21.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.589+0000 7f0ae493b640 1 -- 172.21.15.98:0/1037508555 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae0154020 con 0x7f0ae0154b00 2024-05-09T10:51:21.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.589+0000 7f0ad6ffd640 1 -- 172.21.15.98:0/1037508555 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8037040 con 0x7f0ae0154b00 2024-05-09T10:51:21.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.589+0000 7f0ae493b640 1 -- 172.21.15.98:0/1037508555 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae01f1830 con 0x7f0ae0154b00 2024-05-09T10:51:21.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.590+0000 7f0ae493b640 1 -- 172.21.15.98:0/1037508555 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ae00673a0 con 0x7f0ae0154b00 2024-05-09T10:51:21.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.591+0000 7f0ad6ffd640 1 -- 172.21.15.98:0/1037508555 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 6) v1 ==== 89831+0+0 (secure 0 0 0) 0x7f0ad8044020 con 0x7f0ae0154b00 2024-05-09T10:51:21.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.591+0000 7f0ad6ffd640 1 --2- 172.21.15.98:0/1037508555 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0ac006b590 0x7f0ac006da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:21.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.591+0000 7f0ade59c640 1 --2- 172.21.15.98:0/1037508555 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0ac006b590 0x7f0ac006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:21.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.591+0000 7f0ad6ffd640 1 -- 172.21.15.98:0/1037508555 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ad80aa850 con 0x7f0ae0154b00 2024-05-09T10:51:21.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.592+0000 7f0ade59c640 1 --2- 172.21.15.98:0/1037508555 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0ac006b590 0x7f0ac006da50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0003aa0 tx=0x7f0ad0009830 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:21.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.595+0000 7f0ad6ffd640 1 -- 172.21.15.98:0/1037508555 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ad804b030 con 0x7f0ae0154b00 2024-05-09T10:51:21.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.790+0000 7f0ae493b640 1 -- 172.21.15.98:0/1037508555 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol17470", "group_name": "subvol_grp59180", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ae0072f30 con 0x7f0ac006b590 2024-05-09T10:51:21.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.800+0000 7f0ad6ffd640 1 -- 172.21.15.98:0/1037508555 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0ae0072f30 con 0x7f0ac006b590 2024-05-09T10:51:21.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.802+0000 7f0ae493b640 1 -- 172.21.15.98:0/1037508555 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0ac006b590 msgr2=0x7f0ac006da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:21.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.802+0000 7f0ae493b640 1 --2- 172.21.15.98:0/1037508555 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0ac006b590 0x7f0ac006da50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0003aa0 tx=0x7f0ad0009830 comp rx=0 tx=0).stop 2024-05-09T10:51:21.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.802+0000 7f0ae493b640 1 -- 172.21.15.98:0/1037508555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae0154b00 msgr2=0x7f0ae0156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:21.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.802+0000 7f0ae493b640 1 --2- 172.21.15.98:0/1037508555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae0154b00 0x7f0ae0156630 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80090b0 tx=0x7f0ad80046c0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.802+0000 7f0ae493b640 1 -- 172.21.15.98:0/1037508555 shutdown_connections 2024-05-09T10:51:21.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.802+0000 7f0ae493b640 1 --2- 172.21.15.98:0/1037508555 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0ac006b590 0x7f0ac006da50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.802+0000 7f0ae493b640 1 --2- 172.21.15.98:0/1037508555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ae0154b00 0x7f0ae0156630 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.802+0000 7f0ae493b640 1 --2- 172.21.15.98:0/1037508555 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ae015f270 0x7f0ae01f12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.802+0000 7f0ae493b640 1 --2- 172.21.15.98:0/1037508555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ae006b120 0x7f0ae0158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.802+0000 7f0ae493b640 1 -- 172.21.15.98:0/1037508555 >> 172.21.15.98:0/1037508555 conn(0x7f0ae00d0860 msgr2=0x7f0ae006dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:21.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.802+0000 7f0ae493b640 1 -- 172.21.15.98:0/1037508555 shutdown_connections 2024-05-09T10:51:21.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.802+0000 7f0ae493b640 1 -- 172.21.15.98:0/1037508555 wait complete. 2024-05-09T10:51:21.797 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:51:21.803+0000 7fb2529d7640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T10:51:21.797 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:51:21.803+0000 7fb2529d7640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T10:51:21.797 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:51:21.803+0000 7fb2529d7640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T10:51:21.797 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:51:21.803+0000 7fb2529d7640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T10:51:21.797 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:51:21.803+0000 7fb2529d7640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T10:51:21.808 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol28581 subvol_grp59180 2024-05-09T10:51:21.809 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:51:21.816+0000 7fb2519d5640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T10:51:21.809 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:51:21.816+0000 7fb2519d5640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T10:51:21.809 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:51:21.816+0000 7fb2519d5640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T10:51:21.809 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:51:21.816+0000 7fb2519d5640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T10:51:21.809 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-05-09T10:51:21.816+0000 7fb2519d5640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T10:51:21.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.921+0000 7f7081363640 1 Processor -- start 2024-05-09T10:51:21.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.921+0000 7f7081363640 1 -- start start 2024-05-09T10:51:21.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.921+0000 7f7081363640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f707c0d2820 0x7f707c0d4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:21.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.921+0000 7f7081363640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f707c0b2a30 0x7f707c0b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:21.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.921+0000 7f7081363640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f707c161830 con 0x7f707c161450 2024-05-09T10:51:21.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.921+0000 7f7081363640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f707c0b0760 con 0x7f707c0b2a30 2024-05-09T10:51:21.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.921+0000 7f7081363640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f707c0cf2f0 con 0x7f707c0d2820 2024-05-09T10:51:21.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.921+0000 7f707b7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f707c0b2a30 0x7f707c0b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:21.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.922+0000 7f707affd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f707c161450 0x7f707c0d2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36920/0 (socket says 172.21.15.98:36920) 2024-05-09T10:51:21.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.922+0000 7f707affd640 1 -- 172.21.15.98:0/2015677207 learned_addr learned my addr 172.21.15.98:0/2015677207 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:21.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.922+0000 7f707a7fc640 1 --2- 172.21.15.98:0/2015677207 >> v2:172.21.15.105:3300/0 conn(0x7f707c0d2820 0x7f707c0d4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:21.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.922+0000 7f707b7fe640 1 -- 172.21.15.98:0/2015677207 >> v2:172.21.15.105:3300/0 conn(0x7f707c0d2820 msgr2=0x7f707c0d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:21.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.922+0000 7f707b7fe640 1 --2- 172.21.15.98:0/2015677207 >> v2:172.21.15.105:3300/0 conn(0x7f707c0d2820 0x7f707c0d4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.922+0000 7f707b7fe640 1 -- 172.21.15.98:0/2015677207 >> v1:172.21.15.98:6789/0 conn(0x7f707c161450 legacy=0x7f707c0d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:21.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.922+0000 7f707b7fe640 1 -- 172.21.15.98:0/2015677207 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c0cf5d0 con 0x7f707c0b2a30 2024-05-09T10:51:21.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.922+0000 7f7079ffb640 1 -- 172.21.15.98:0/2015677207 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284765381 0 0) 0x7f706c001490 con 0x7f707c161450 2024-05-09T10:51:21.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.923+0000 7f707b7fe640 1 --2- 172.21.15.98:0/2015677207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f707c0b2a30 0x7f707c0b0ed0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f707c08cc70 tx=0x7f707402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4296defe7959f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:21.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.923+0000 7f7079ffb640 1 -- 172.21.15.98:0/2015677207 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7074037070 con 0x7f707c0b2a30 2024-05-09T10:51:21.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.923+0000 7f7079ffb640 1 -- 172.21.15.98:0/2015677207 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7074032440 con 0x7f707c0b2a30 2024-05-09T10:51:21.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.923+0000 7f7079ffb640 1 -- 172.21.15.98:0/2015677207 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707402ae60 con 0x7f707c0b2a30 2024-05-09T10:51:21.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.923+0000 7f7081363640 1 -- 172.21.15.98:0/2015677207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f707c0b2a30 msgr2=0x7f707c0b0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:21.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.923+0000 7f7081363640 1 --2- 172.21.15.98:0/2015677207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f707c0b2a30 0x7f707c0b0ed0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f707c08cc70 tx=0x7f707402a110 comp rx=0 tx=0).stop 2024-05-09T10:51:21.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.923+0000 7f7081363640 1 -- 172.21.15.98:0/2015677207 shutdown_connections 2024-05-09T10:51:21.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.923+0000 7f7081363640 1 --2- 172.21.15.98:0/2015677207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f707c0b2a30 0x7f707c0b0ed0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.923+0000 7f7081363640 1 --2- 172.21.15.98:0/2015677207 >> v2:172.21.15.105:3300/0 conn(0x7f707c0d2820 0x7f707c0d4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.923+0000 7f7081363640 1 -- 172.21.15.98:0/2015677207 >> 172.21.15.98:0/2015677207 conn(0x7f707c0d3b30 msgr2=0x7f707c0d3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:21.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.923+0000 7f7081363640 1 -- 172.21.15.98:0/2015677207 shutdown_connections 2024-05-09T10:51:21.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.923+0000 7f7081363640 1 -- 172.21.15.98:0/2015677207 wait complete. 2024-05-09T10:51:21.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.925+0000 7f7081363640 1 Processor -- start 2024-05-09T10:51:21.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.925+0000 7f7081363640 1 -- start start 2024-05-09T10:51:21.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.925+0000 7f7081363640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f707c0b2a30 0x7f707c15d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:21.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.925+0000 7f7081363640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f707c0d2820 0x7f707c0cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:21.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.925+0000 7f7081363640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f707c161450 0x7f707c0cb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:21.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.925+0000 7f7081363640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c0b1410 con 0x7f707c0b2a30 2024-05-09T10:51:21.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.925+0000 7f7081363640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c0b09b0 con 0x7f707c161450 2024-05-09T10:51:21.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.925+0000 7f7081363640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707c0b1590 con 0x7f707c0d2820 2024-05-09T10:51:21.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.925+0000 7f707affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f707c0b2a30 0x7f707c15d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:21.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.925+0000 7f707affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f707c0b2a30 0x7f707c15d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57766/0 (socket says 172.21.15.98:57766) 2024-05-09T10:51:21.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.925+0000 7f707affd640 1 -- 172.21.15.98:0/2367256808 learned_addr learned my addr 172.21.15.98:0/2367256808 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:21.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.926+0000 7f707b7fe640 1 --2- 172.21.15.98:0/2367256808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f707c161450 0x7f707c0cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:21.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.926+0000 7f707a7fc640 1 --2- 172.21.15.98:0/2367256808 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f707c0d2820 0x7f707c0cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:21.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.926+0000 7f707affd640 1 -- 172.21.15.98:0/2367256808 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f707c0d2820 msgr2=0x7f707c0cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:21.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.926+0000 7f707affd640 1 --2- 172.21.15.98:0/2367256808 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f707c0d2820 0x7f707c0cac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.926+0000 7f707affd640 1 -- 172.21.15.98:0/2367256808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f707c161450 msgr2=0x7f707c0cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:21.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.926+0000 7f707affd640 1 --2- 172.21.15.98:0/2367256808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f707c161450 0x7f707c0cb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:21.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.926+0000 7f707affd640 1 -- 172.21.15.98:0/2367256808 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c0ccd90 con 0x7f707c0b2a30 2024-05-09T10:51:21.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.926+0000 7f707b7fe640 1 --2- 172.21.15.98:0/2367256808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f707c161450 0x7f707c0cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:21.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.926+0000 7f707affd640 1 --2- 172.21.15.98:0/2367256808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f707c0b2a30 0x7f707c15d4b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f706c00c7c0 tx=0x7f706c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:21.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.926+0000 7f70737fe640 1 -- 172.21.15.98:0/2367256808 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c018070 con 0x7f707c0b2a30 2024-05-09T10:51:21.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.926+0000 7f70737fe640 1 -- 172.21.15.98:0/2367256808 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706c00a640 con 0x7f707c0b2a30 2024-05-09T10:51:21.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.926+0000 7f7081363640 1 -- 172.21.15.98:0/2367256808 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f707c160a30 con 0x7f707c0b2a30 2024-05-09T10:51:21.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.926+0000 7f70737fe640 1 -- 172.21.15.98:0/2367256808 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c00aa30 con 0x7f707c0b2a30 2024-05-09T10:51:21.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.926+0000 7f7081363640 1 -- 172.21.15.98:0/2367256808 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f707c1605a0 con 0x7f707c0b2a30 2024-05-09T10:51:21.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.927+0000 7f7081363640 1 -- 172.21.15.98:0/2367256808 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f707c0a1ae0 con 0x7f707c0b2a30 2024-05-09T10:51:21.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.928+0000 7f70737fe640 1 -- 172.21.15.98:0/2367256808 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 6) v1 ==== 89831+0+0 (secure 0 0 0) 0x7f706c01b070 con 0x7f707c0b2a30 2024-05-09T10:51:21.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.928+0000 7f70737fe640 1 --2- 172.21.15.98:0/2367256808 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f705c06b560 0x7f705c06da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:21.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.928+0000 7f70737fe640 1 -- 172.21.15.98:0/2367256808 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f706c089850 con 0x7f707c0b2a30 2024-05-09T10:51:21.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.928+0000 7f707a7fc640 1 --2- 172.21.15.98:0/2367256808 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f705c06b560 0x7f705c06da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:21.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.928+0000 7f707a7fc640 1 --2- 172.21.15.98:0/2367256808 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f705c06b560 0x7f705c06da20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f707c15a440 tx=0x7f7064008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:21.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:21.932+0000 7f70737fe640 1 -- 172.21.15.98:0/2367256808 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f707c161830 con 0x7f707c0b2a30 2024-05-09T10:51:22.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.127+0000 7f7081363640 1 -- 172.21.15.98:0/2367256808 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol28581", "group_name": "subvol_grp59180", "target": ["mon-mgr", ""]}) v1 -- 0x7f707c0b27f0 con 0x7f705c06b560 2024-05-09T10:51:22.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.146+0000 7f70737fe640 1 -- 172.21.15.98:0/2367256808 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f707c0b27f0 con 0x7f705c06b560 2024-05-09T10:51:22.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.149+0000 7f7081363640 1 -- 172.21.15.98:0/2367256808 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f705c06b560 msgr2=0x7f705c06da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:22.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.149+0000 7f7081363640 1 --2- 172.21.15.98:0/2367256808 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f705c06b560 0x7f705c06da20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f707c15a440 tx=0x7f7064008040 comp rx=0 tx=0).stop 2024-05-09T10:51:22.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.149+0000 7f7081363640 1 -- 172.21.15.98:0/2367256808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f707c0b2a30 msgr2=0x7f707c15d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:22.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.149+0000 7f7081363640 1 --2- 172.21.15.98:0/2367256808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f707c0b2a30 0x7f707c15d4b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f706c00c7c0 tx=0x7f706c00cc80 comp rx=0 tx=0).stop 2024-05-09T10:51:22.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.149+0000 7f7081363640 1 -- 172.21.15.98:0/2367256808 shutdown_connections 2024-05-09T10:51:22.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.149+0000 7f7081363640 1 --2- 172.21.15.98:0/2367256808 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f705c06b560 0x7f705c06da20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.149+0000 7f7081363640 1 --2- 172.21.15.98:0/2367256808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f707c161450 0x7f707c0cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.149+0000 7f7081363640 1 --2- 172.21.15.98:0/2367256808 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f707c0d2820 0x7f707c0cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.149+0000 7f7081363640 1 --2- 172.21.15.98:0/2367256808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f707c0b2a30 0x7f707c15d4b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.149+0000 7f7081363640 1 -- 172.21.15.98:0/2367256808 >> 172.21.15.98:0/2367256808 conn(0x7f707c0d3b30 msgr2=0x7f707c0d0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:22.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.149+0000 7f7081363640 1 -- 172.21.15.98:0/2367256808 shutdown_connections 2024-05-09T10:51:22.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.149+0000 7f7081363640 1 -- 172.21.15.98:0/2367256808 wait complete. 2024-05-09T10:51:22.155 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol27112 subvol_grp59180 2024-05-09T10:51:22.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.265+0000 7fb33a61f640 1 Processor -- start 2024-05-09T10:51:22.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.265+0000 7fb33a61f640 1 -- start start 2024-05-09T10:51:22.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.266+0000 7fb33a61f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb33415bc80 0x7fb3340cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:22.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.266+0000 7fb33a61f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb3340d0660 0x7fb3340b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:22.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.266+0000 7fb33a61f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3340ce380 con 0x7fb33415c060 2024-05-09T10:51:22.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.266+0000 7fb33a61f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3340ce660 con 0x7fb33415bc80 2024-05-09T10:51:22.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.266+0000 7fb33a61f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3340ce940 con 0x7fb3340d0660 2024-05-09T10:51:22.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.266+0000 7fb333fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb33415bc80 0x7fb3340cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:22.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.266+0000 7fb333fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb33415bc80 0x7fb3340cd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57768/0 (socket says 172.21.15.98:57768) 2024-05-09T10:51:22.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.266+0000 7fb333fff640 1 -- 172.21.15.98:0/2024303678 learned_addr learned my addr 172.21.15.98:0/2024303678 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:22.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.266+0000 7fb338bbc640 1 --2- 172.21.15.98:0/2024303678 >> v2:172.21.15.105:3300/0 conn(0x7fb3340d0660 0x7fb3340b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:22.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.266+0000 7fb333fff640 1 -- 172.21.15.98:0/2024303678 >> v2:172.21.15.105:3300/0 conn(0x7fb3340d0660 msgr2=0x7fb3340b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:22.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.266+0000 7fb333fff640 1 --2- 172.21.15.98:0/2024303678 >> v2:172.21.15.105:3300/0 conn(0x7fb3340d0660 0x7fb3340b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.266+0000 7fb333fff640 1 -- 172.21.15.98:0/2024303678 >> v1:172.21.15.105:6789/0 conn(0x7fb33415c060 legacy=0x7fb3340cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:22.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.266+0000 7fb333fff640 1 -- 172.21.15.98:0/2024303678 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3340cec20 con 0x7fb33415bc80 2024-05-09T10:51:22.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.266+0000 7fb338bbc640 1 --2- 172.21.15.98:0/2024303678 >> v2:172.21.15.105:3300/0 conn(0x7fb3340d0660 0x7fb3340b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:22.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.267+0000 7fb333fff640 1 --2- 172.21.15.98:0/2024303678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb33415bc80 0x7fb3340cd090 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb334071b20 tx=0x7fb32402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a9000e69e269b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:22.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.267+0000 7fb3337fe640 1 -- 172.21.15.98:0/2024303678 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb324037070 con 0x7fb33415bc80 2024-05-09T10:51:22.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.267+0000 7fb3337fe640 1 -- 172.21.15.98:0/2024303678 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb324032440 con 0x7fb33415bc80 2024-05-09T10:51:22.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.267+0000 7fb3337fe640 1 -- 172.21.15.98:0/2024303678 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb324032810 con 0x7fb33415bc80 2024-05-09T10:51:22.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.268+0000 7fb33a61f640 1 -- 172.21.15.98:0/2024303678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb33415bc80 msgr2=0x7fb3340cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:22.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.268+0000 7fb33a61f640 1 --2- 172.21.15.98:0/2024303678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb33415bc80 0x7fb3340cd090 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb334071b20 tx=0x7fb32402a230 comp rx=0 tx=0).stop 2024-05-09T10:51:22.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.268+0000 7fb33a61f640 1 -- 172.21.15.98:0/2024303678 shutdown_connections 2024-05-09T10:51:22.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.268+0000 7fb33a61f640 1 --2- 172.21.15.98:0/2024303678 >> v2:172.21.15.105:3300/0 conn(0x7fb3340d0660 0x7fb3340b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.268+0000 7fb33a61f640 1 --2- 172.21.15.98:0/2024303678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb33415bc80 0x7fb3340cd090 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.268+0000 7fb33a61f640 1 -- 172.21.15.98:0/2024303678 >> 172.21.15.98:0/2024303678 conn(0x7fb3340d1970 msgr2=0x7fb3340d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:22.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.268+0000 7fb33a61f640 1 -- 172.21.15.98:0/2024303678 shutdown_connections 2024-05-09T10:51:22.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.268+0000 7fb33a61f640 1 -- 172.21.15.98:0/2024303678 wait complete. 2024-05-09T10:51:22.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.269+0000 7fb33a61f640 1 Processor -- start 2024-05-09T10:51:22.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.269+0000 7fb33a61f640 1 -- start start 2024-05-09T10:51:22.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.270+0000 7fb33a61f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3340d0660 0x7fb33414de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:22.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.270+0000 7fb33a61f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb33415bc80 0x7fb334149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:22.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.270+0000 7fb33a61f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb33415c060 0x7fb33409f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:22.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.270+0000 7fb33a61f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3340ce120 con 0x7fb33415c060 2024-05-09T10:51:22.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.270+0000 7fb33a61f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3340cdfa0 con 0x7fb33415bc80 2024-05-09T10:51:22.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.270+0000 7fb33a61f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3340cd5d0 con 0x7fb3340d0660 2024-05-09T10:51:22.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.270+0000 7fb338bbc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb33415c060 0x7fb33409f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:22.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.270+0000 7fb338bbc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb33415c060 0x7fb33409f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57784/0 (socket says 172.21.15.98:57784) 2024-05-09T10:51:22.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.270+0000 7fb338bbc640 1 -- 172.21.15.98:0/1077394619 learned_addr learned my addr 172.21.15.98:0/1077394619 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:22.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.270+0000 7fb333fff640 1 --2- 172.21.15.98:0/1077394619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3340d0660 0x7fb33414de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:22.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.270+0000 7fb32bfff640 1 --2- 172.21.15.98:0/1077394619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb33415bc80 0x7fb334149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:22.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.270+0000 7fb338bbc640 1 -- 172.21.15.98:0/1077394619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3340d0660 msgr2=0x7fb33414de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:22.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.270+0000 7fb338bbc640 1 --2- 172.21.15.98:0/1077394619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3340d0660 0x7fb33414de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.270+0000 7fb338bbc640 1 -- 172.21.15.98:0/1077394619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb33415bc80 msgr2=0x7fb334149d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:22.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.270+0000 7fb338bbc640 1 --2- 172.21.15.98:0/1077394619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb33415bc80 0x7fb334149d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.270+0000 7fb338bbc640 1 -- 172.21.15.98:0/1077394619 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33414a3b0 con 0x7fb33415c060 2024-05-09T10:51:22.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.270+0000 7fb333fff640 1 --2- 172.21.15.98:0/1077394619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3340d0660 0x7fb33414de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:22.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.270+0000 7fb338bbc640 1 --2- 172.21.15.98:0/1077394619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb33415c060 0x7fb33409f330 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00ce80 tx=0x7fb32c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:22.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.270+0000 7fb3317fa640 1 -- 172.21.15.98:0/1077394619 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c01a070 con 0x7fb33415c060 2024-05-09T10:51:22.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.270+0000 7fb3317fa640 1 -- 172.21.15.98:0/1077394619 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32c00d880 con 0x7fb33415c060 2024-05-09T10:51:22.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.270+0000 7fb33a61f640 1 -- 172.21.15.98:0/1077394619 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33414a690 con 0x7fb33415c060 2024-05-09T10:51:22.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.270+0000 7fb3317fa640 1 -- 172.21.15.98:0/1077394619 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c00dc70 con 0x7fb33415c060 2024-05-09T10:51:22.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.271+0000 7fb33a61f640 1 -- 172.21.15.98:0/1077394619 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33409b970 con 0x7fb33415c060 2024-05-09T10:51:22.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.271+0000 7fb33a61f640 1 -- 172.21.15.98:0/1077394619 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33406ba50 con 0x7fb33415c060 2024-05-09T10:51:22.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.272+0000 7fb3317fa640 1 -- 172.21.15.98:0/1077394619 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 6) v1 ==== 89831+0+0 (secure 0 0 0) 0x7fb32c01d070 con 0x7fb33415c060 2024-05-09T10:51:22.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.273+0000 7fb3317fa640 1 --2- 172.21.15.98:0/1077394619 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb31406b560 0x7fb31406da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:22.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.273+0000 7fb333fff640 1 --2- 172.21.15.98:0/1077394619 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb31406b560 0x7fb31406da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:22.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.273+0000 7fb3317fa640 1 -- 172.21.15.98:0/1077394619 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb32c08a910 con 0x7fb33415c060 2024-05-09T10:51:22.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.273+0000 7fb333fff640 1 --2- 172.21.15.98:0/1077394619 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb31406b560 0x7fb31406da20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb324034000 tx=0x7fb32402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:22.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.277+0000 7fb3317fa640 1 -- 172.21.15.98:0/1077394619 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb32c0591e0 con 0x7fb33415c060 2024-05-09T10:51:22.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.480+0000 7fb33a61f640 1 -- 172.21.15.98:0/1077394619 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol27112", "group_name": "subvol_grp59180", "target": ["mon-mgr", ""]}) v1 -- 0x7fb33409bb70 con 0x7fb31406b560 2024-05-09T10:51:22.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.492+0000 7fb3317fa640 1 -- 172.21.15.98:0/1077394619 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb33409bb70 con 0x7fb31406b560 2024-05-09T10:51:22.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.495+0000 7fb33a61f640 1 -- 172.21.15.98:0/1077394619 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb31406b560 msgr2=0x7fb31406da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:22.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.495+0000 7fb33a61f640 1 --2- 172.21.15.98:0/1077394619 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb31406b560 0x7fb31406da20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb324034000 tx=0x7fb32402e000 comp rx=0 tx=0).stop 2024-05-09T10:51:22.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.495+0000 7fb33a61f640 1 -- 172.21.15.98:0/1077394619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb33415c060 msgr2=0x7fb33409f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:22.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.495+0000 7fb33a61f640 1 --2- 172.21.15.98:0/1077394619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb33415c060 0x7fb33409f330 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00ce80 tx=0x7fb32c00a710 comp rx=0 tx=0).stop 2024-05-09T10:51:22.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.495+0000 7fb33a61f640 1 -- 172.21.15.98:0/1077394619 shutdown_connections 2024-05-09T10:51:22.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.495+0000 7fb33a61f640 1 --2- 172.21.15.98:0/1077394619 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb31406b560 0x7fb31406da20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.495+0000 7fb33a61f640 1 --2- 172.21.15.98:0/1077394619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb33415c060 0x7fb33409f330 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.495+0000 7fb33a61f640 1 --2- 172.21.15.98:0/1077394619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb33415bc80 0x7fb334149d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.495+0000 7fb33a61f640 1 --2- 172.21.15.98:0/1077394619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb3340d0660 0x7fb33414de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.495+0000 7fb33a61f640 1 -- 172.21.15.98:0/1077394619 >> 172.21.15.98:0/1077394619 conn(0x7fb3340d1970 msgr2=0x7fb3340b03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:22.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.495+0000 7fb33a61f640 1 -- 172.21.15.98:0/1077394619 shutdown_connections 2024-05-09T10:51:22.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.495+0000 7fb33a61f640 1 -- 172.21.15.98:0/1077394619 wait complete. 2024-05-09T10:51:22.501 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp59180 2024-05-09T10:51:22.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.611+0000 7fcc67db7640 1 Processor -- start 2024-05-09T10:51:22.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.611+0000 7fcc67db7640 1 -- start start 2024-05-09T10:51:22.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.612+0000 7fcc67db7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcc6015a100 0x7fcc6006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:22.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.612+0000 7fcc67db7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc6015a4e0 0x7fcc600c4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:22.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.612+0000 7fcc67db7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcc600c4b60 0x7fcc600ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:22.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.612+0000 7fcc67db7640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc600cd660 con 0x7fcc600c4b60 2024-05-09T10:51:22.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.612+0000 7fcc67db7640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc600cd920 con 0x7fcc6015a100 2024-05-09T10:51:22.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.612+0000 7fcc67db7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc600cdc00 con 0x7fcc6015a4e0 2024-05-09T10:51:22.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.612+0000 7fcc66354640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcc600c4b60 0x7fcc600ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:22.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.612+0000 7fcc66354640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcc600c4b60 0x7fcc600ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57798/0 (socket says 172.21.15.98:57798) 2024-05-09T10:51:22.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.612+0000 7fcc66354640 1 -- 172.21.15.98:0/3100443481 learned_addr learned my addr 172.21.15.98:0/3100443481 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:22.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.612+0000 7fcc65352640 1 --2- 172.21.15.98:0/3100443481 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc6015a4e0 0x7fcc600c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:22.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.612+0000 7fcc65b53640 1 --2- 172.21.15.98:0/3100443481 >> v2:172.21.15.105:3300/0 conn(0x7fcc6015a100 0x7fcc6006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:22.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.612+0000 7fcc66354640 1 -- 172.21.15.98:0/3100443481 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc6015a4e0 msgr2=0x7fcc600c4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:22.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.612+0000 7fcc66354640 1 --2- 172.21.15.98:0/3100443481 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc6015a4e0 0x7fcc600c4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.612+0000 7fcc66354640 1 -- 172.21.15.98:0/3100443481 >> v2:172.21.15.105:3300/0 conn(0x7fcc6015a100 msgr2=0x7fcc6006d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:22.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.612+0000 7fcc66354640 1 --2- 172.21.15.98:0/3100443481 >> v2:172.21.15.105:3300/0 conn(0x7fcc6015a100 0x7fcc6006d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.612+0000 7fcc66354640 1 -- 172.21.15.98:0/3100443481 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc600cdee0 con 0x7fcc600c4b60 2024-05-09T10:51:22.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.613+0000 7fcc66354640 1 --2- 172.21.15.98:0/3100443481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc600c4b60 0x7fcc600ce6e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcc600cf2c0 tx=0x7fcc5802c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1107088185630d0d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:22.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.613+0000 7fcc64b51640 1 -- 172.21.15.98:0/3100443481 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc58009600 con 0x7fcc600c4b60 2024-05-09T10:51:22.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.613+0000 7fcc64b51640 1 -- 172.21.15.98:0/3100443481 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc58037070 con 0x7fcc600c4b60 2024-05-09T10:51:22.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.613+0000 7fcc64b51640 1 -- 172.21.15.98:0/3100443481 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc580099c0 con 0x7fcc600c4b60 2024-05-09T10:51:22.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.613+0000 7fcc67db7640 1 -- 172.21.15.98:0/3100443481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc600c4b60 msgr2=0x7fcc600ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:22.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.613+0000 7fcc67db7640 1 --2- 172.21.15.98:0/3100443481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc600c4b60 0x7fcc600ce6e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcc600cf2c0 tx=0x7fcc5802c930 comp rx=0 tx=0).stop 2024-05-09T10:51:22.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.614+0000 7fcc67db7640 1 -- 172.21.15.98:0/3100443481 shutdown_connections 2024-05-09T10:51:22.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.614+0000 7fcc67db7640 1 --2- 172.21.15.98:0/3100443481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc600c4b60 0x7fcc600ce6e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.614+0000 7fcc67db7640 1 --2- 172.21.15.98:0/3100443481 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc6015a4e0 0x7fcc600c4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.614+0000 7fcc67db7640 1 --2- 172.21.15.98:0/3100443481 >> v2:172.21.15.105:3300/0 conn(0x7fcc6015a100 0x7fcc6006d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.614+0000 7fcc67db7640 1 -- 172.21.15.98:0/3100443481 >> 172.21.15.98:0/3100443481 conn(0x7fcc600cf760 msgr2=0x7fcc600cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:22.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.614+0000 7fcc67db7640 1 -- 172.21.15.98:0/3100443481 shutdown_connections 2024-05-09T10:51:22.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.614+0000 7fcc67db7640 1 -- 172.21.15.98:0/3100443481 wait complete. 2024-05-09T10:51:22.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.615+0000 7fcc67db7640 1 Processor -- start 2024-05-09T10:51:22.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.615+0000 7fcc67db7640 1 -- start start 2024-05-09T10:51:22.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.615+0000 7fcc67db7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc600c4b60 0x7fcc601ef390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:22.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.615+0000 7fcc67db7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc6015a100 0x7fcc60070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:22.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.615+0000 7fcc67db7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc6015a4e0 0x7fcc60070a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:22.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.615+0000 7fcc67db7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6006c4b0 con 0x7fcc6015a100 2024-05-09T10:51:22.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.615+0000 7fcc67db7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6006e460 con 0x7fcc6015a4e0 2024-05-09T10:51:22.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.615+0000 7fcc67db7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6006c630 con 0x7fcc600c4b60 2024-05-09T10:51:22.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.616+0000 7fcc65352640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc6015a100 0x7fcc60070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:22.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.616+0000 7fcc65352640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc6015a100 0x7fcc60070540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57814/0 (socket says 172.21.15.98:57814) 2024-05-09T10:51:22.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.616+0000 7fcc65352640 1 -- 172.21.15.98:0/2451737873 learned_addr learned my addr 172.21.15.98:0/2451737873 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:22.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.616+0000 7fcc65b53640 1 --2- 172.21.15.98:0/2451737873 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc600c4b60 0x7fcc601ef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:22.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.616+0000 7fcc66354640 1 --2- 172.21.15.98:0/2451737873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc6015a4e0 0x7fcc60070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:22.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.616+0000 7fcc65352640 1 -- 172.21.15.98:0/2451737873 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc600c4b60 msgr2=0x7fcc601ef390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:22.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.616+0000 7fcc65352640 1 --2- 172.21.15.98:0/2451737873 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc600c4b60 0x7fcc601ef390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.616+0000 7fcc65352640 1 -- 172.21.15.98:0/2451737873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc6015a4e0 msgr2=0x7fcc60070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:22.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.616+0000 7fcc65352640 1 --2- 172.21.15.98:0/2451737873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc6015a4e0 0x7fcc60070a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.616+0000 7fcc65352640 1 -- 172.21.15.98:0/2451737873 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc600c78b0 con 0x7fcc6015a100 2024-05-09T10:51:22.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.616+0000 7fcc66354640 1 --2- 172.21.15.98:0/2451737873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc6015a4e0 0x7fcc60070a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:22.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.616+0000 7fcc65352640 1 --2- 172.21.15.98:0/2451737873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc6015a100 0x7fcc60070540 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000d7d0 tx=0x7fcc5000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:22.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.616+0000 7fcc567fc640 1 -- 172.21.15.98:0/2451737873 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5000b6c0 con 0x7fcc6015a100 2024-05-09T10:51:22.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.616+0000 7fcc567fc640 1 -- 172.21.15.98:0/2451737873 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc50019070 con 0x7fcc6015a100 2024-05-09T10:51:22.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.616+0000 7fcc567fc640 1 -- 172.21.15.98:0/2451737873 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5000ba80 con 0x7fcc6015a100 2024-05-09T10:51:22.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.616+0000 7fcc67db7640 1 -- 172.21.15.98:0/2451737873 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc600c6830 con 0x7fcc6015a100 2024-05-09T10:51:22.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.616+0000 7fcc67db7640 1 -- 172.21.15.98:0/2451737873 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc600c6dd0 con 0x7fcc6015a100 2024-05-09T10:51:22.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.617+0000 7fcc67db7640 1 -- 172.21.15.98:0/2451737873 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc60072c80 con 0x7fcc6015a100 2024-05-09T10:51:22.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.618+0000 7fcc567fc640 1 -- 172.21.15.98:0/2451737873 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 6) v1 ==== 89831+0+0 (secure 0 0 0) 0x7fcc5000bc20 con 0x7fcc6015a100 2024-05-09T10:51:22.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.618+0000 7fcc567fc640 1 --2- 172.21.15.98:0/2451737873 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fcc4406b560 0x7fcc4406da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:22.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.618+0000 7fcc567fc640 1 -- 172.21.15.98:0/2451737873 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc5008a620 con 0x7fcc6015a100 2024-05-09T10:51:22.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.618+0000 7fcc65b53640 1 --2- 172.21.15.98:0/2451737873 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fcc4406b560 0x7fcc4406da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:22.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.618+0000 7fcc65b53640 1 --2- 172.21.15.98:0/2451737873 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fcc4406b560 0x7fcc4406da20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c006fd0 tx=0x7fcc5c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:22.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.622+0000 7fcc567fc640 1 -- 172.21.15.98:0/2451737873 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcc500594b0 con 0x7fcc6015a100 2024-05-09T10:51:22.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.819+0000 7fcc67db7640 1 -- 172.21.15.98:0/2451737873 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp59180", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc600c7af0 con 0x7fcc4406b560 2024-05-09T10:51:22.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.822+0000 7fcc567fc640 1 -- 172.21.15.98:0/2451737873 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcc600c7af0 con 0x7fcc4406b560 2024-05-09T10:51:22.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.824+0000 7fcc67db7640 1 -- 172.21.15.98:0/2451737873 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fcc4406b560 msgr2=0x7fcc4406da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:22.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.824+0000 7fcc67db7640 1 --2- 172.21.15.98:0/2451737873 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fcc4406b560 0x7fcc4406da20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c006fd0 tx=0x7fcc5c008040 comp rx=0 tx=0).stop 2024-05-09T10:51:22.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.824+0000 7fcc67db7640 1 -- 172.21.15.98:0/2451737873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc6015a100 msgr2=0x7fcc60070540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:22.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.824+0000 7fcc67db7640 1 --2- 172.21.15.98:0/2451737873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc6015a100 0x7fcc60070540 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000d7d0 tx=0x7fcc5000dc90 comp rx=0 tx=0).stop 2024-05-09T10:51:22.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.824+0000 7fcc67db7640 1 -- 172.21.15.98:0/2451737873 shutdown_connections 2024-05-09T10:51:22.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.824+0000 7fcc67db7640 1 --2- 172.21.15.98:0/2451737873 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fcc4406b560 0x7fcc4406da20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.824+0000 7fcc67db7640 1 --2- 172.21.15.98:0/2451737873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc6015a4e0 0x7fcc60070a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.824+0000 7fcc67db7640 1 --2- 172.21.15.98:0/2451737873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc6015a100 0x7fcc60070540 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.824+0000 7fcc67db7640 1 --2- 172.21.15.98:0/2451737873 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcc600c4b60 0x7fcc601ef390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:22.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.824+0000 7fcc67db7640 1 -- 172.21.15.98:0/2451737873 >> 172.21.15.98:0/2451737873 conn(0x7fcc600cf760 msgr2=0x7fcc60159340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:22.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.824+0000 7fcc67db7640 1 -- 172.21.15.98:0/2451737873 shutdown_connections 2024-05-09T10:51:22.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:22.824+0000 7fcc67db7640 1 -- 172.21.15.98:0/2451737873 wait complete. 2024-05-09T10:51:22.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:22.829 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-05-09T10:51:22.849 INFO:teuthology.orchestra.run.smithi098.stdout:2 2024-05-09T10:51:22.849 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-09T10:51:22.849 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T10:51:22.925 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T10:51:22.968 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-09T10:51:22.968 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:51:22.968 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:51:22.984 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:51:22.984 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:51:23.040 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:51:23.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.186+0000 7fb9f23d5640 1 Processor -- start 2024-05-09T10:51:23.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.187+0000 7fb9f23d5640 1 -- start start 2024-05-09T10:51:23.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.187+0000 7fb9f23d5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb9ec155000 0x7fb9ec0713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:23.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.187+0000 7fb9f23d5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9ec071930 0x7fb9ec06e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:23.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.187+0000 7fb9f23d5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ec1553e0 con 0x7fb9ec06eb70 2024-05-09T10:51:23.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.187+0000 7fb9f23d5640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9ec071d90 con 0x7fb9ec155000 2024-05-09T10:51:23.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.187+0000 7fb9f23d5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec0b4430 con 0x7fb9ec071930 2024-05-09T10:51:23.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.187+0000 7fb9ebfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb9ec155000 0x7fb9ec0713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:23.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.188+0000 7fb9ebfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb9ec155000 0x7fb9ec0713f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57824/0 (socket says 172.21.15.98:57824) 2024-05-09T10:51:23.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.188+0000 7fb9ebfff640 1 -- 172.21.15.98:0/433810598 learned_addr learned my addr 172.21.15.98:0/433810598 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:23.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.188+0000 7fb9e3fff640 1 --2- 172.21.15.98:0/433810598 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9ec071930 0x7fb9ec06e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:23.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.188+0000 7fb9ebfff640 1 -- 172.21.15.98:0/433810598 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9ec071930 msgr2=0x7fb9ec06e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:23.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.188+0000 7fb9ebfff640 1 --2- 172.21.15.98:0/433810598 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9ec071930 0x7fb9ec06e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.188+0000 7fb9ebfff640 1 -- 172.21.15.98:0/433810598 >> v1:172.21.15.98:6789/0 conn(0x7fb9ec06eb70 legacy=0x7fb9ec0af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:23.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.188+0000 7fb9ebfff640 1 -- 172.21.15.98:0/433810598 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ec0b3860 con 0x7fb9ec155000 2024-05-09T10:51:23.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.188+0000 7fb9e3fff640 1 --2- 172.21.15.98:0/433810598 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9ec071930 0x7fb9ec06e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:23.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.188+0000 7fb9ebfff640 1 --2- 172.21.15.98:0/433810598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9ec155000 0x7fb9ec0713f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0b45b0 tx=0x7fb9dc02c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54a293dda8d6d12 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:23.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.189+0000 7fb9eb7fe640 1 -- 172.21.15.98:0/433810598 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc036020 con 0x7fb9ec155000 2024-05-09T10:51:23.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.189+0000 7fb9eb7fe640 1 -- 172.21.15.98:0/433810598 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9dc032800 con 0x7fb9ec155000 2024-05-09T10:51:23.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.189+0000 7fb9eb7fe640 1 -- 172.21.15.98:0/433810598 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc032bd0 con 0x7fb9ec155000 2024-05-09T10:51:23.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.189+0000 7fb9f23d5640 1 -- 172.21.15.98:0/433810598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9ec155000 msgr2=0x7fb9ec0713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:23.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.189+0000 7fb9f23d5640 1 --2- 172.21.15.98:0/433810598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9ec155000 0x7fb9ec0713f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0b45b0 tx=0x7fb9dc02c900 comp rx=0 tx=0).stop 2024-05-09T10:51:23.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.189+0000 7fb9f23d5640 1 -- 172.21.15.98:0/433810598 shutdown_connections 2024-05-09T10:51:23.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.189+0000 7fb9f23d5640 1 --2- 172.21.15.98:0/433810598 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9ec071930 0x7fb9ec06e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.189+0000 7fb9f23d5640 1 --2- 172.21.15.98:0/433810598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9ec155000 0x7fb9ec0713f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.189+0000 7fb9f23d5640 1 -- 172.21.15.98:0/433810598 >> 172.21.15.98:0/433810598 conn(0x7fb9ec0b24c0 msgr2=0x7fb9ec0b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:23.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.189+0000 7fb9f23d5640 1 -- 172.21.15.98:0/433810598 shutdown_connections 2024-05-09T10:51:23.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.189+0000 7fb9f23d5640 1 -- 172.21.15.98:0/433810598 wait complete. 2024-05-09T10:51:23.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.190+0000 7fb9f23d5640 1 Processor -- start 2024-05-09T10:51:23.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.190+0000 7fb9f23d5640 1 -- start start 2024-05-09T10:51:23.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.191+0000 7fb9f23d5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9ec06eb70 0x7fb9ec149c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:23.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.191+0000 7fb9f23d5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9ec071930 0x7fb9ec157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:23.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.191+0000 7fb9f23d5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9ec155000 0x7fb9ec06abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:23.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.191+0000 7fb9f23d5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec0b3de0 con 0x7fb9ec06eb70 2024-05-09T10:51:23.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.191+0000 7fb9f23d5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec06efd0 con 0x7fb9ec155000 2024-05-09T10:51:23.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.191+0000 7fb9f23d5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec0b40c0 con 0x7fb9ec071930 2024-05-09T10:51:23.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.191+0000 7fb9ebfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9ec06eb70 0x7fb9ec149c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:23.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.191+0000 7fb9ebfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9ec06eb70 0x7fb9ec149c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57830/0 (socket says 172.21.15.98:57830) 2024-05-09T10:51:23.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.191+0000 7fb9ebfff640 1 -- 172.21.15.98:0/1970970626 learned_addr learned my addr 172.21.15.98:0/1970970626 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:23.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.191+0000 7fb9ebfff640 1 -- 172.21.15.98:0/1970970626 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9ec071930 msgr2=0x7fb9ec157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:23.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.191+0000 7fb9e3fff640 1 --2- 172.21.15.98:0/1970970626 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9ec071930 0x7fb9ec157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:23.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.191+0000 7fb9ebfff640 1 --2- 172.21.15.98:0/1970970626 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9ec071930 0x7fb9ec157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.191+0000 7fb9ebfff640 1 -- 172.21.15.98:0/1970970626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9ec155000 msgr2=0x7fb9ec06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:23.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.191+0000 7fb9f0972640 1 --2- 172.21.15.98:0/1970970626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9ec155000 0x7fb9ec06abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:23.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.192+0000 7fb9ebfff640 1 --2- 172.21.15.98:0/1970970626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9ec155000 0x7fb9ec06abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.192+0000 7fb9ebfff640 1 -- 172.21.15.98:0/1970970626 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ec14a190 con 0x7fb9ec06eb70 2024-05-09T10:51:23.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.192+0000 7fb9e3fff640 1 --2- 172.21.15.98:0/1970970626 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9ec071930 0x7fb9ec157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:23.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.192+0000 7fb9f0972640 1 --2- 172.21.15.98:0/1970970626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9ec155000 0x7fb9ec06abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:23.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.192+0000 7fb9ebfff640 1 --2- 172.21.15.98:0/1970970626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9ec06eb70 0x7fb9ec149c50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc02c940 tx=0x7fb9dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:23.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.192+0000 7fb9e97fa640 1 -- 172.21.15.98:0/1970970626 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc042070 con 0x7fb9ec06eb70 2024-05-09T10:51:23.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.192+0000 7fb9e97fa640 1 -- 172.21.15.98:0/1970970626 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9dc032620 con 0x7fb9ec06eb70 2024-05-09T10:51:23.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.192+0000 7fb9f23d5640 1 -- 172.21.15.98:0/1970970626 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ec1f3680 con 0x7fb9ec06eb70 2024-05-09T10:51:23.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.192+0000 7fb9e97fa640 1 -- 172.21.15.98:0/1970970626 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc036040 con 0x7fb9ec06eb70 2024-05-09T10:51:23.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.192+0000 7fb9f23d5640 1 -- 172.21.15.98:0/1970970626 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ec1f3c20 con 0x7fb9ec06eb70 2024-05-09T10:51:23.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.193+0000 7fb9f23d5640 1 -- 172.21.15.98:0/1970970626 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9ec069cb0 con 0x7fb9ec06eb70 2024-05-09T10:51:23.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.193+0000 7fb9e97fa640 1 -- 172.21.15.98:0/1970970626 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 6) v1 ==== 89831+0+0 (secure 0 0 0) 0x7fb9dc045070 con 0x7fb9ec06eb70 2024-05-09T10:51:23.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.194+0000 7fb9e97fa640 1 --2- 172.21.15.98:0/1970970626 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb9cc06b560 0x7fb9cc06da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:23.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.194+0000 7fb9e97fa640 1 -- 172.21.15.98:0/1970970626 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9dc0aaf10 con 0x7fb9ec06eb70 2024-05-09T10:51:23.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.194+0000 7fb9e3fff640 1 --2- 172.21.15.98:0/1970970626 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb9cc06b560 0x7fb9cc06da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:23.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.194+0000 7fb9e3fff640 1 --2- 172.21.15.98:0/1970970626 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb9cc06b560 0x7fb9cc06da20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4006fd0 tx=0x7fb9d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:23.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.198+0000 7fb9e97fa640 1 -- 172.21.15.98:0/1970970626 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb9dc0327d0 con 0x7fb9ec06eb70 2024-05-09T10:51:23.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.371+0000 7fb9f23d5640 1 -- 172.21.15.98:0/1970970626 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9ec15bf00 con 0x7fb9ec06eb70 2024-05-09T10:51:23.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.372+0000 7fb9e97fa640 1 -- 172.21.15.98:0/1970970626 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+16669 (secure 0 0 0) 0x7fb9dc079d20 con 0x7fb9ec06eb70 2024-05-09T10:51:23.365 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:23.365 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":31,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:51:08.400895+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:51:05.150410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-09T10:51:05.708391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:51:23.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.374+0000 7fb9f23d5640 1 -- 172.21.15.98:0/1970970626 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb9cc06b560 msgr2=0x7fb9cc06da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:23.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.374+0000 7fb9f23d5640 1 --2- 172.21.15.98:0/1970970626 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb9cc06b560 0x7fb9cc06da20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4006fd0 tx=0x7fb9d4008040 comp rx=0 tx=0).stop 2024-05-09T10:51:23.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.374+0000 7fb9f23d5640 1 -- 172.21.15.98:0/1970970626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9ec06eb70 msgr2=0x7fb9ec149c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:23.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.374+0000 7fb9f23d5640 1 --2- 172.21.15.98:0/1970970626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9ec06eb70 0x7fb9ec149c50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc02c940 tx=0x7fb9dc004730 comp rx=0 tx=0).stop 2024-05-09T10:51:23.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.374+0000 7fb9f23d5640 1 -- 172.21.15.98:0/1970970626 shutdown_connections 2024-05-09T10:51:23.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.374+0000 7fb9f23d5640 1 --2- 172.21.15.98:0/1970970626 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fb9cc06b560 0x7fb9cc06da20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.374+0000 7fb9f23d5640 1 --2- 172.21.15.98:0/1970970626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9ec155000 0x7fb9ec06abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.374+0000 7fb9f23d5640 1 --2- 172.21.15.98:0/1970970626 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9ec071930 0x7fb9ec157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.374+0000 7fb9f23d5640 1 --2- 172.21.15.98:0/1970970626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9ec06eb70 0x7fb9ec149c50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.374+0000 7fb9f23d5640 1 -- 172.21.15.98:0/1970970626 >> 172.21.15.98:0/1970970626 conn(0x7fb9ec0b24c0 msgr2=0x7fb9ec0b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:23.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.374+0000 7fb9f23d5640 1 -- 172.21.15.98:0/1970970626 shutdown_connections 2024-05-09T10:51:23.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.374+0000 7fb9f23d5640 1 -- 172.21.15.98:0/1970970626 wait complete. 2024-05-09T10:51:23.379 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T10:50:07.396423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T10:51:05.150410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T10:51:05.708391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T10:51:23.379 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T10:51:23.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.494+0000 7f19f0d9e640 1 Processor -- start 2024-05-09T10:51:23.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.494+0000 7f19f0d9e640 1 -- start start 2024-05-09T10:51:23.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.495+0000 7f19f0d9e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19ec0d3950 0x7f19ec0d3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:23.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.495+0000 7f19f0d9e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ec0cf150 con 0x7f19ec0ce5d0 2024-05-09T10:51:23.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.495+0000 7f19f0d9e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ec0cf400 con 0x7f19ec15dd60 2024-05-09T10:51:23.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.495+0000 7f19f0d9e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0cf9c0 con 0x7f19ec0d3950 2024-05-09T10:51:23.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.495+0000 7f19ebfff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f19ec0ce5d0 0x7f19ec0cea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46920/0 (socket says 172.21.15.98:46920) 2024-05-09T10:51:23.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.495+0000 7f19ebfff640 1 -- 172.21.15.98:0/2948850816 learned_addr learned my addr 172.21.15.98:0/2948850816 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:23.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19ea7fc640 1 -- 172.21.15.98:0/2948850816 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3418682381 0 0) 0x7f19ec0cf150 con 0x7f19ec0ce5d0 2024-05-09T10:51:23.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19eaffd640 1 --2- 172.21.15.98:0/2948850816 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19ec0d3950 0x7f19ec0d3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:23.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19ea7fc640 1 -- 172.21.15.98:0/2948850816 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19d4003520 con 0x7f19ec0ce5d0 2024-05-09T10:51:23.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19ea7fc640 1 -- 172.21.15.98:0/2948850816 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4223537994 0 0) 0x7f19d4003520 con 0x7f19ec0ce5d0 2024-05-09T10:51:23.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19ea7fc640 1 -- 172.21.15.98:0/2948850816 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19ec0d3950 msgr2=0x7f19ec0d3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:23.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19ea7fc640 1 --2- 172.21.15.98:0/2948850816 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19ec0d3950 0x7f19ec0d3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19ea7fc640 1 -- 172.21.15.98:0/2948850816 >> v1:172.21.15.105:6789/0 conn(0x7f19ec15dd60 legacy=0x7f19ec06b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:23.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19ea7fc640 1 -- 172.21.15.98:0/2948850816 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec0cf6e0 con 0x7f19ec0ce5d0 2024-05-09T10:51:23.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19ea7fc640 1 -- 172.21.15.98:0/2948850816 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f19e4001670 con 0x7f19ec0ce5d0 2024-05-09T10:51:23.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19ea7fc640 1 -- 172.21.15.98:0/2948850816 >> v1:172.21.15.98:6789/0 conn(0x7f19ec0ce5d0 legacy=0x7f19ec0cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:23.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19ea7fc640 1 --2- 172.21.15.98:0/2948850816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19d40039e0 0x7f19d4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19ea7fc640 1 --2- 172.21.15.98:0/2948850816 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19ec0d3950 0x7f19d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19ea7fc640 1 --2- 172.21.15.98:0/2948850816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19d4004d40 0x7f19d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19ea7fc640 1 -- 172.21.15.98:0/2948850816 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0cf9c0 con 0x7f19d4004d40 2024-05-09T10:51:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19ea7fc640 1 -- 172.21.15.98:0/2948850816 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d400b9c0 con 0x7f19d40039e0 2024-05-09T10:51:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19ea7fc640 1 -- 172.21.15.98:0/2948850816 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d400bbc0 con 0x7f19ec0d3950 2024-05-09T10:51:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19ebfff640 1 --2- 172.21.15.98:0/2948850816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19d4004d40 0x7f19d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19eb7fe640 1 --2- 172.21.15.98:0/2948850816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19d40039e0 0x7f19d4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19eaffd640 1 --2- 172.21.15.98:0/2948850816 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19ec0d3950 0x7f19d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19ebfff640 1 -- 172.21.15.98:0/2948850816 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19ec0d3950 msgr2=0x7f19d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19ebfff640 1 --2- 172.21.15.98:0/2948850816 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19ec0d3950 0x7f19d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19ebfff640 1 -- 172.21.15.98:0/2948850816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19d40039e0 msgr2=0x7f19d4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19ebfff640 1 --2- 172.21.15.98:0/2948850816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19d40039e0 0x7f19d4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.496+0000 7f19ebfff640 1 -- 172.21.15.98:0/2948850816 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d4003520 con 0x7f19d4004d40 2024-05-09T10:51:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.497+0000 7f19ebfff640 1 --2- 172.21.15.98:0/2948850816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19d4004d40 0x7f19d40051c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f19e400c290 tx=0x7f19d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb8ac514b90384f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.497+0000 7f19ea7fc640 1 -- 172.21.15.98:0/2948850816 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4007d50 con 0x7f19d4004d40 2024-05-09T10:51:23.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.497+0000 7f19ea7fc640 1 -- 172.21.15.98:0/2948850816 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e4005af0 con 0x7f19d4004d40 2024-05-09T10:51:23.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.497+0000 7f19ea7fc640 1 -- 172.21.15.98:0/2948850816 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4007070 con 0x7f19d4004d40 2024-05-09T10:51:23.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.498+0000 7f19f0d9e640 1 -- 172.21.15.98:0/2948850816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19d4004d40 msgr2=0x7f19d40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:23.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.498+0000 7f19f0d9e640 1 --2- 172.21.15.98:0/2948850816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19d4004d40 0x7f19d40051c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f19e400c290 tx=0x7f19d4005830 comp rx=0 tx=0).stop 2024-05-09T10:51:23.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.498+0000 7f19ebfff640 1 -- 172.21.15.98:0/2948850816 reap_dead start 2024-05-09T10:51:23.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.498+0000 7f19f0d9e640 1 -- 172.21.15.98:0/2948850816 shutdown_connections 2024-05-09T10:51:23.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.498+0000 7f19f0d9e640 1 -- 172.21.15.98:0/2948850816 >> 172.21.15.98:0/2948850816 conn(0x7f19ec08d3f0 msgr2=0x7f19ec09cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:23.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.498+0000 7f19f0d9e640 1 -- 172.21.15.98:0/2948850816 shutdown_connections 2024-05-09T10:51:23.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.498+0000 7f19f0d9e640 1 -- 172.21.15.98:0/2948850816 wait complete. 2024-05-09T10:51:23.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.499+0000 7f19f0d9e640 1 Processor -- start 2024-05-09T10:51:23.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.499+0000 7f19f0d9e640 1 -- start start 2024-05-09T10:51:23.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.500+0000 7f19f0d9e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19d4004d40 0x7f19ec0cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:23.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.500+0000 7f19f0d9e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19ec0c9fc0 0x7f19ec0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:23.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.500+0000 7f19f0d9e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19ec15a110 0x7f19ec15a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:23.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.500+0000 7f19f0d9e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0d3d90 con 0x7f19d4004d40 2024-05-09T10:51:23.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.500+0000 7f19f0d9e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0d19a0 con 0x7f19ec15a110 2024-05-09T10:51:23.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.500+0000 7f19f0d9e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0d3f10 con 0x7f19ec0c9fc0 2024-05-09T10:51:23.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.500+0000 7f19eb7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19d4004d40 0x7f19ec0cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:23.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.500+0000 7f19eb7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19d4004d40 0x7f19ec0cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59992/0 (socket says 172.21.15.98:59992) 2024-05-09T10:51:23.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.500+0000 7f19eb7fe640 1 -- 172.21.15.98:0/800395992 learned_addr learned my addr 172.21.15.98:0/800395992 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:23.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.500+0000 7f19eaffd640 1 --2- 172.21.15.98:0/800395992 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19ec0c9fc0 0x7f19ec0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:23.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.500+0000 7f19ebfff640 1 --2- 172.21.15.98:0/800395992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19ec15a110 0x7f19ec15a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:23.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.500+0000 7f19eb7fe640 1 -- 172.21.15.98:0/800395992 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19ec0c9fc0 msgr2=0x7f19ec0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.500+0000 7f19eb7fe640 1 --2- 172.21.15.98:0/800395992 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19ec0c9fc0 0x7f19ec0cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.500+0000 7f19eb7fe640 1 -- 172.21.15.98:0/800395992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19ec15a110 msgr2=0x7f19ec15a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.500+0000 7f19eb7fe640 1 --2- 172.21.15.98:0/800395992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19ec15a110 0x7f19ec15a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.500+0000 7f19eb7fe640 1 -- 172.21.15.98:0/800395992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec0cbcf0 con 0x7f19d4004d40 2024-05-09T10:51:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.500+0000 7f19eaffd640 1 --2- 172.21.15.98:0/800395992 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19ec0c9fc0 0x7f19ec0cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.500+0000 7f19ebfff640 1 --2- 172.21.15.98:0/800395992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19ec15a110 0x7f19ec15a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.501+0000 7f19eb7fe640 1 --2- 172.21.15.98:0/800395992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19d4004d40 0x7f19ec0cb4a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00fbc0 tx=0x7f19dc00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.501+0000 7f19d3fff640 1 -- 172.21.15.98:0/800395992 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc00dbd0 con 0x7f19d4004d40 2024-05-09T10:51:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.501+0000 7f19d3fff640 1 -- 172.21.15.98:0/800395992 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc018070 con 0x7f19d4004d40 2024-05-09T10:51:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.501+0000 7f19d3fff640 1 -- 172.21.15.98:0/800395992 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc003970 con 0x7f19d4004d40 2024-05-09T10:51:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.501+0000 7f19f0d9e640 1 -- 172.21.15.98:0/800395992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ec0cbfd0 con 0x7f19d4004d40 2024-05-09T10:51:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.501+0000 7f19f0d9e640 1 -- 172.21.15.98:0/800395992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec0cb9e0 con 0x7f19d4004d40 2024-05-09T10:51:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.502+0000 7f19f0d9e640 1 -- 172.21.15.98:0/800395992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ec0cc4f0 con 0x7f19d4004d40 2024-05-09T10:51:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.502+0000 7f19d3fff640 1 -- 172.21.15.98:0/800395992 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f19dc003b10 con 0x7f19d4004d40 2024-05-09T10:51:23.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.502+0000 7f19d3fff640 1 --2- 172.21.15.98:0/800395992 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f19d806b680 0x7f19d806db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:23.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.503+0000 7f19d3fff640 1 -- 172.21.15.98:0/800395992 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f19dc08a7c0 con 0x7f19d4004d40 2024-05-09T10:51:23.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.503+0000 7f19eaffd640 1 --2- 172.21.15.98:0/800395992 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f19d806b680 0x7f19d806db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:23.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.503+0000 7f19eaffd640 1 --2- 172.21.15.98:0/800395992 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f19d806b680 0x7f19d806db40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0b2660 tx=0x7f19e0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:23.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.507+0000 7f19d3fff640 1 -- 172.21.15.98:0/800395992 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f19ec0cc4f0 con 0x7f19d4004d40 2024-05-09T10:51:23.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.680+0000 7f19f0d9e640 1 -- 172.21.15.98:0/800395992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f19ec04ef20 con 0x7f19d4004d40 2024-05-09T10:51:23.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.680+0000 7f19d3fff640 1 -- 172.21.15.98:0/800395992 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) v1 ==== 93+0+10 (secure 0 0 0) 0x7f19dc0593c0 con 0x7f19d4004d40 2024-05-09T10:51:23.674 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-05-09T10:51:23.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.682+0000 7f19f0d9e640 1 -- 172.21.15.98:0/800395992 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f19d806b680 msgr2=0x7f19d806db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:23.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.682+0000 7f19f0d9e640 1 --2- 172.21.15.98:0/800395992 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f19d806b680 0x7f19d806db40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0b2660 tx=0x7f19e0008040 comp rx=0 tx=0).stop 2024-05-09T10:51:23.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.682+0000 7f19f0d9e640 1 -- 172.21.15.98:0/800395992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19d4004d40 msgr2=0x7f19ec0cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:23.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.682+0000 7f19f0d9e640 1 --2- 172.21.15.98:0/800395992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19d4004d40 0x7f19ec0cb4a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00fbc0 tx=0x7f19dc00d520 comp rx=0 tx=0).stop 2024-05-09T10:51:23.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.682+0000 7f19f0d9e640 1 -- 172.21.15.98:0/800395992 shutdown_connections 2024-05-09T10:51:23.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.682+0000 7f19f0d9e640 1 --2- 172.21.15.98:0/800395992 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f19d806b680 0x7f19d806db40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.682+0000 7f19f0d9e640 1 --2- 172.21.15.98:0/800395992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19ec15a110 0x7f19ec15a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.682+0000 7f19f0d9e640 1 --2- 172.21.15.98:0/800395992 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19ec0c9fc0 0x7f19ec0cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.682+0000 7f19f0d9e640 1 --2- 172.21.15.98:0/800395992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19d4004d40 0x7f19ec0cb4a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.682+0000 7f19f0d9e640 1 -- 172.21.15.98:0/800395992 >> 172.21.15.98:0/800395992 conn(0x7f19ec08d3f0 msgr2=0x7f19ec0d08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:23.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.682+0000 7f19f0d9e640 1 -- 172.21.15.98:0/800395992 shutdown_connections 2024-05-09T10:51:23.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.682+0000 7f19f0d9e640 1 -- 172.21.15.98:0/800395992 wait complete. 2024-05-09T10:51:23.686 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T10:51:23.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.795+0000 7f25bc037640 1 Processor -- start 2024-05-09T10:51:23.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.795+0000 7f25bc037640 1 -- start start 2024-05-09T10:51:23.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.796+0000 7f25bc037640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f25b40ca530 0x7f25b415f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:23.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.796+0000 7f25bc037640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b40ae960 con 0x7f25b415a8f0 2024-05-09T10:51:23.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.796+0000 7f25bc037640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b40ca9c0 con 0x7f25b40cee80 2024-05-09T10:51:23.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.796+0000 7f25bc037640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b415f5d0 con 0x7f25b40ca530 2024-05-09T10:51:23.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.796+0000 7f25b95d2640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f25b40ca530 0x7f25b415f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:23.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.796+0000 7f25b9dd3640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f25b415a8f0 0x7f25b40ce770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46934/0 (socket says 172.21.15.98:46934) 2024-05-09T10:51:23.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.796+0000 7f25b95d2640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f25b40ca530 0x7f25b415f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60004/0 (socket says 172.21.15.98:60004) 2024-05-09T10:51:23.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.796+0000 7f25b9dd3640 1 -- 172.21.15.98:0/3215591437 learned_addr learned my addr 172.21.15.98:0/3215591437 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:23.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.796+0000 7f25b95d2640 1 -- 172.21.15.98:0/3215591437 >> v1:172.21.15.105:6789/0 conn(0x7f25b40cee80 legacy=0x7f25b40c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:23.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.796+0000 7f25b95d2640 1 -- 172.21.15.98:0/3215591437 >> v1:172.21.15.98:6789/0 conn(0x7f25b415a8f0 legacy=0x7f25b40ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:23.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.796+0000 7f25b95d2640 1 -- 172.21.15.98:0/3215591437 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b40caca0 con 0x7f25b40ca530 2024-05-09T10:51:23.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.796+0000 7f25b8dd1640 1 -- 172.21.15.98:0/3215591437 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2794350838 0 0) 0x7f25b40ae960 con 0x7f25b415a8f0 2024-05-09T10:51:23.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.797+0000 7f25b95d2640 1 --2- 172.21.15.98:0/3215591437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25b40ca530 0x7f25b415f090 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f25a40097a0 tx=0x7f25a402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5248ac40fc14b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:23.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.797+0000 7f25b8dd1640 1 -- 172.21.15.98:0/3215591437 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4037070 con 0x7f25b40ca530 2024-05-09T10:51:23.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.797+0000 7f25b8dd1640 1 -- 172.21.15.98:0/3215591437 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a4032440 con 0x7f25b40ca530 2024-05-09T10:51:23.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.797+0000 7f25b8dd1640 1 -- 172.21.15.98:0/3215591437 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4032810 con 0x7f25b40ca530 2024-05-09T10:51:23.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.798+0000 7f25bc037640 1 -- 172.21.15.98:0/3215591437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25b40ca530 msgr2=0x7f25b415f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:23.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.798+0000 7f25bc037640 1 --2- 172.21.15.98:0/3215591437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25b40ca530 0x7f25b415f090 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f25a40097a0 tx=0x7f25a402a1e0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.798+0000 7f25bc037640 1 -- 172.21.15.98:0/3215591437 shutdown_connections 2024-05-09T10:51:23.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.798+0000 7f25bc037640 1 --2- 172.21.15.98:0/3215591437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25b40ca530 0x7f25b415f090 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.798+0000 7f25bc037640 1 -- 172.21.15.98:0/3215591437 >> 172.21.15.98:0/3215591437 conn(0x7f25b40cf760 msgr2=0x7f25b40cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:23.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.798+0000 7f25bc037640 1 -- 172.21.15.98:0/3215591437 shutdown_connections 2024-05-09T10:51:23.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.798+0000 7f25bc037640 1 -- 172.21.15.98:0/3215591437 wait complete. 2024-05-09T10:51:23.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.799+0000 7f25bc037640 1 Processor -- start 2024-05-09T10:51:23.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.799+0000 7f25bc037640 1 -- start start 2024-05-09T10:51:23.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.800+0000 7f25bc037640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25b40cee80 0x7f25b415bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:23.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.800+0000 7f25bc037640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25b415a8f0 0x7f25b4147490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:23.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.800+0000 7f25bc037640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25b4149e00 0x7f25b4155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:23.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.800+0000 7f25bc037640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b415acd0 con 0x7f25b40cee80 2024-05-09T10:51:23.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.800+0000 7f25bc037640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40cb260 con 0x7f25b415a8f0 2024-05-09T10:51:23.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.800+0000 7f25bc037640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b415ae50 con 0x7f25b4149e00 2024-05-09T10:51:23.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.800+0000 7f25b9dd3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25b40cee80 0x7f25b415bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:23.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.800+0000 7f25b9dd3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25b40cee80 0x7f25b415bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60018/0 (socket says 172.21.15.98:60018) 2024-05-09T10:51:23.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.800+0000 7f25b9dd3640 1 -- 172.21.15.98:0/3821562268 learned_addr learned my addr 172.21.15.98:0/3821562268 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:23.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.800+0000 7f25b9dd3640 1 -- 172.21.15.98:0/3821562268 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25b4149e00 msgr2=0x7f25b4155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:23.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.800+0000 7f25b9dd3640 1 --2- 172.21.15.98:0/3821562268 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25b4149e00 0x7f25b4155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.800+0000 7f25b9dd3640 1 -- 172.21.15.98:0/3821562268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25b415a8f0 msgr2=0x7f25b4147490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:23.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.800+0000 7f25b9dd3640 1 --2- 172.21.15.98:0/3821562268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25b415a8f0 0x7f25b4147490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.800+0000 7f25b9dd3640 1 -- 172.21.15.98:0/3821562268 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b4147d70 con 0x7f25b40cee80 2024-05-09T10:51:23.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.800+0000 7f25b9dd3640 1 --2- 172.21.15.98:0/3821562268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25b40cee80 0x7f25b415bad0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f25b000c430 tx=0x7f25b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:23.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.800+0000 7f25aa7fc640 1 -- 172.21.15.98:0/3821562268 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b00074e0 con 0x7f25b40cee80 2024-05-09T10:51:23.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.800+0000 7f25aa7fc640 1 -- 172.21.15.98:0/3821562268 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b00047e0 con 0x7f25b40cee80 2024-05-09T10:51:23.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.800+0000 7f25aa7fc640 1 -- 172.21.15.98:0/3821562268 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b0004bd0 con 0x7f25b40cee80 2024-05-09T10:51:23.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.800+0000 7f25bc037640 1 -- 172.21.15.98:0/3821562268 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b414a300 con 0x7f25b40cee80 2024-05-09T10:51:23.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.800+0000 7f25bc037640 1 -- 172.21.15.98:0/3821562268 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b414a8c0 con 0x7f25b40cee80 2024-05-09T10:51:23.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.801+0000 7f25bc037640 1 -- 172.21.15.98:0/3821562268 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b4072c80 con 0x7f25b40cee80 2024-05-09T10:51:23.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.802+0000 7f25aa7fc640 1 -- 172.21.15.98:0/3821562268 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f25b0011070 con 0x7f25b40cee80 2024-05-09T10:51:23.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.802+0000 7f25aa7fc640 1 --2- 172.21.15.98:0/3821562268 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f259806b680 0x7f259806db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:23.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.802+0000 7f25aa7fc640 1 -- 172.21.15.98:0/3821562268 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25b008a180 con 0x7f25b40cee80 2024-05-09T10:51:23.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.802+0000 7f25b95d2640 1 --2- 172.21.15.98:0/3821562268 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f259806b680 0x7f259806db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:23.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.803+0000 7f25b95d2640 1 --2- 172.21.15.98:0/3821562268 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f259806b680 0x7f259806db40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f25a4034000 tx=0x7f25a402c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:23.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.807+0000 7f25aa7fc640 1 -- 172.21.15.98:0/3821562268 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f25b0058d80 con 0x7f25b40cee80 2024-05-09T10:51:23.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.977+0000 7f25bc037640 1 -- 172.21.15.98:0/3821562268 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f25b404f000 con 0x7f25b40cee80 2024-05-09T10:51:23.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.978+0000 7f25aa7fc640 1 -- 172.21.15.98:0/3821562268 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v31) v1 ==== 104+0+11 (secure 0 0 0) 0x7f25b0058680 con 0x7f25b40cee80 2024-05-09T10:51:23.971 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2024-05-09T10:51:23.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.979+0000 7f25bc037640 1 -- 172.21.15.98:0/3821562268 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f259806b680 msgr2=0x7f259806db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:23.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.979+0000 7f25bc037640 1 --2- 172.21.15.98:0/3821562268 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f259806b680 0x7f259806db40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f25a4034000 tx=0x7f25a402c000 comp rx=0 tx=0).stop 2024-05-09T10:51:23.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.979+0000 7f25bc037640 1 -- 172.21.15.98:0/3821562268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25b40cee80 msgr2=0x7f25b415bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:23.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.979+0000 7f25bc037640 1 --2- 172.21.15.98:0/3821562268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25b40cee80 0x7f25b415bad0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f25b000c430 tx=0x7f25b000c8f0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.980+0000 7f25bc037640 1 -- 172.21.15.98:0/3821562268 shutdown_connections 2024-05-09T10:51:23.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.980+0000 7f25bc037640 1 --2- 172.21.15.98:0/3821562268 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f259806b680 0x7f259806db40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.980+0000 7f25bc037640 1 --2- 172.21.15.98:0/3821562268 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25b4149e00 0x7f25b4155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.980+0000 7f25bc037640 1 --2- 172.21.15.98:0/3821562268 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25b415a8f0 0x7f25b4147490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.980+0000 7f25bc037640 1 --2- 172.21.15.98:0/3821562268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f25b40cee80 0x7f25b415bad0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:23.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.980+0000 7f25bc037640 1 -- 172.21.15.98:0/3821562268 >> 172.21.15.98:0/3821562268 conn(0x7f25b40cf760 msgr2=0x7f25b4071800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:23.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.980+0000 7f25bc037640 1 -- 172.21.15.98:0/3821562268 shutdown_connections 2024-05-09T10:51:23.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:23.980+0000 7f25bc037640 1 -- 172.21.15.98:0/3821562268 wait complete. 2024-05-09T10:51:23.984 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T10:51:24.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.093+0000 7fe7afbce640 1 Processor -- start 2024-05-09T10:51:24.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.093+0000 7fe7afbce640 1 -- start start 2024-05-09T10:51:24.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.094+0000 7fe7afbce640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe7a815dd60 0x7fe7a80d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:24.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.094+0000 7fe7afbce640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe7a815e140 0x7fe7a80cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:24.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.094+0000 7fe7afbce640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a80cf8e0 con 0x7fe7a80d1de0 2024-05-09T10:51:24.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.094+0000 7fe7afbce640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a80cf6b0 con 0x7fe7a815dd60 2024-05-09T10:51:24.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.094+0000 7fe7afbce640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a80b2f50 con 0x7fe7a815e140 2024-05-09T10:51:24.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.094+0000 7fe7ad96a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe7a815dd60 0x7fe7a80d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:24.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.094+0000 7fe7ae16b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe7a80d1de0 0x7fe7a80d21c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46948/0 (socket says 172.21.15.98:46948) 2024-05-09T10:51:24.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.094+0000 7fe7ae16b640 1 -- 172.21.15.98:0/1141767025 learned_addr learned my addr 172.21.15.98:0/1141767025 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:24.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.094+0000 7fe7ad169640 1 --2- 172.21.15.98:0/1141767025 >> v2:172.21.15.105:3300/0 conn(0x7fe7a815e140 0x7fe7a80cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:24.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.095+0000 7fe7ad96a640 1 -- 172.21.15.98:0/1141767025 >> v2:172.21.15.105:3300/0 conn(0x7fe7a815e140 msgr2=0x7fe7a80cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:24.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.095+0000 7fe7ad96a640 1 --2- 172.21.15.98:0/1141767025 >> v2:172.21.15.105:3300/0 conn(0x7fe7a815e140 0x7fe7a80cf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:24.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.095+0000 7fe7ad96a640 1 -- 172.21.15.98:0/1141767025 >> v1:172.21.15.98:6789/0 conn(0x7fe7a80d1de0 legacy=0x7fe7a80d21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:24.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.095+0000 7fe7ad96a640 1 -- 172.21.15.98:0/1141767025 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a80b31a0 con 0x7fe7a815dd60 2024-05-09T10:51:24.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.095+0000 7fe7ac968640 1 -- 172.21.15.98:0/1141767025 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3235578818 0 0) 0x7fe7a80cf8e0 con 0x7fe7a80d1de0 2024-05-09T10:51:24.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.095+0000 7fe7ad96a640 1 --2- 172.21.15.98:0/1141767025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7a815dd60 0x7fe7a80d1790 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe7a80b30d0 tx=0x7fe7a402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceebe8b30848417b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:24.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.095+0000 7fe7ac968640 1 -- 172.21.15.98:0/1141767025 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4007740 con 0x7fe7a815dd60 2024-05-09T10:51:24.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.095+0000 7fe7ac968640 1 -- 172.21.15.98:0/1141767025 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a40078e0 con 0x7fe7a815dd60 2024-05-09T10:51:24.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.095+0000 7fe7ac968640 1 -- 172.21.15.98:0/1141767025 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4007cb0 con 0x7fe7a815dd60 2024-05-09T10:51:24.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.096+0000 7fe7afbce640 1 -- 172.21.15.98:0/1141767025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7a815dd60 msgr2=0x7fe7a80d1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:24.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.096+0000 7fe7afbce640 1 --2- 172.21.15.98:0/1141767025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7a815dd60 0x7fe7a80d1790 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe7a80b30d0 tx=0x7fe7a402a660 comp rx=0 tx=0).stop 2024-05-09T10:51:24.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.096+0000 7fe7afbce640 1 -- 172.21.15.98:0/1141767025 shutdown_connections 2024-05-09T10:51:24.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.096+0000 7fe7afbce640 1 --2- 172.21.15.98:0/1141767025 >> v2:172.21.15.105:3300/0 conn(0x7fe7a815e140 0x7fe7a80cf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:24.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.096+0000 7fe7afbce640 1 --2- 172.21.15.98:0/1141767025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7a815dd60 0x7fe7a80d1790 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:24.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.096+0000 7fe7afbce640 1 -- 172.21.15.98:0/1141767025 >> 172.21.15.98:0/1141767025 conn(0x7fe7a80d3b50 msgr2=0x7fe7a80d3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:24.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.096+0000 7fe7afbce640 1 -- 172.21.15.98:0/1141767025 shutdown_connections 2024-05-09T10:51:24.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.096+0000 7fe7afbce640 1 -- 172.21.15.98:0/1141767025 wait complete. 2024-05-09T10:51:24.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.097+0000 7fe7afbce640 1 Processor -- start 2024-05-09T10:51:24.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.097+0000 7fe7afbce640 1 -- start start 2024-05-09T10:51:24.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.098+0000 7fe7afbce640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7a80d1de0 0x7fe7a815a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:24.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.098+0000 7fe7afbce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7a815dd60 0x7fe7a80b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:24.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.098+0000 7fe7afbce640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7a81f3950 0x7fe7a81f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:24.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.098+0000 7fe7afbce640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80b3c80 con 0x7fe7a81f3950 2024-05-09T10:51:24.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.098+0000 7fe7afbce640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80b3760 con 0x7fe7a815dd60 2024-05-09T10:51:24.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.098+0000 7fe7afbce640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80b3e00 con 0x7fe7a80d1de0 2024-05-09T10:51:24.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.098+0000 7fe7ae16b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7a81f3950 0x7fe7a81f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:24.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.098+0000 7fe7ae16b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7a81f3950 0x7fe7a81f3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60034/0 (socket says 172.21.15.98:60034) 2024-05-09T10:51:24.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.098+0000 7fe7ae16b640 1 -- 172.21.15.98:0/565126709 learned_addr learned my addr 172.21.15.98:0/565126709 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:24.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.098+0000 7fe7ad96a640 1 --2- 172.21.15.98:0/565126709 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7a80d1de0 0x7fe7a815a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:24.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.098+0000 7fe7ad169640 1 --2- 172.21.15.98:0/565126709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7a815dd60 0x7fe7a80b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:24.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.098+0000 7fe7ae16b640 1 -- 172.21.15.98:0/565126709 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7a80d1de0 msgr2=0x7fe7a815a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:24.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.098+0000 7fe7ae16b640 1 --2- 172.21.15.98:0/565126709 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7a80d1de0 0x7fe7a815a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:24.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.098+0000 7fe7ae16b640 1 -- 172.21.15.98:0/565126709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7a815dd60 msgr2=0x7fe7a80b71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:24.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.098+0000 7fe7ae16b640 1 --2- 172.21.15.98:0/565126709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7a815dd60 0x7fe7a80b71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:24.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.098+0000 7fe7ae16b640 1 -- 172.21.15.98:0/565126709 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a80b6710 con 0x7fe7a81f3950 2024-05-09T10:51:24.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.099+0000 7fe7ae16b640 1 --2- 172.21.15.98:0/565126709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7a81f3950 0x7fe7a81f3de0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000c480 tx=0x7fe7a000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:24.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.099+0000 7fe79e7fc640 1 -- 172.21.15.98:0/565126709 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0018070 con 0x7fe7a81f3950 2024-05-09T10:51:24.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.099+0000 7fe79e7fc640 1 -- 172.21.15.98:0/565126709 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a0007500 con 0x7fe7a81f3950 2024-05-09T10:51:24.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.099+0000 7fe7afbce640 1 -- 172.21.15.98:0/565126709 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a81f4480 con 0x7fe7a81f3950 2024-05-09T10:51:24.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.099+0000 7fe79e7fc640 1 -- 172.21.15.98:0/565126709 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0004a20 con 0x7fe7a81f3950 2024-05-09T10:51:24.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.099+0000 7fe7afbce640 1 -- 172.21.15.98:0/565126709 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a80b69d0 con 0x7fe7a81f3950 2024-05-09T10:51:24.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.100+0000 7fe7afbce640 1 -- 172.21.15.98:0/565126709 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a80a1b60 con 0x7fe7a81f3950 2024-05-09T10:51:24.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.100+0000 7fe79e7fc640 1 -- 172.21.15.98:0/565126709 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7fe7a001b070 con 0x7fe7a81f3950 2024-05-09T10:51:24.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.101+0000 7fe79e7fc640 1 --2- 172.21.15.98:0/565126709 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe78c06b6b0 0x7fe78c06db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:24.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.101+0000 7fe79e7fc640 1 -- 172.21.15.98:0/565126709 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7a0089760 con 0x7fe7a81f3950 2024-05-09T10:51:24.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.101+0000 7fe7ad96a640 1 --2- 172.21.15.98:0/565126709 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe78c06b6b0 0x7fe78c06db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:24.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.101+0000 7fe7ad96a640 1 --2- 172.21.15.98:0/565126709 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe78c06b6b0 0x7fe78c06db70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4034000 tx=0x7fe7a402f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:24.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.105+0000 7fe79e7fc640 1 -- 172.21.15.98:0/565126709 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe7a00584b0 con 0x7fe7a81f3950 2024-05-09T10:51:24.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.274+0000 7fe7afbce640 1 -- 172.21.15.98:0/565126709 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe7a804f000 con 0x7fe7a81f3950 2024-05-09T10:51:24.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.275+0000 7fe79e7fc640 1 -- 172.21.15.98:0/565126709 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v31) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe7a005ce20 con 0x7fe7a81f3950 2024-05-09T10:51:24.268 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2024-05-09T10:51:24.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.277+0000 7fe7afbce640 1 -- 172.21.15.98:0/565126709 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe78c06b6b0 msgr2=0x7fe78c06db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:24.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.277+0000 7fe7afbce640 1 --2- 172.21.15.98:0/565126709 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe78c06b6b0 0x7fe78c06db70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4034000 tx=0x7fe7a402f000 comp rx=0 tx=0).stop 2024-05-09T10:51:24.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.277+0000 7fe7afbce640 1 -- 172.21.15.98:0/565126709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7a81f3950 msgr2=0x7fe7a81f3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:24.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.277+0000 7fe7afbce640 1 --2- 172.21.15.98:0/565126709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7a81f3950 0x7fe7a81f3de0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000c480 tx=0x7fe7a000c940 comp rx=0 tx=0).stop 2024-05-09T10:51:24.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.277+0000 7fe7afbce640 1 -- 172.21.15.98:0/565126709 shutdown_connections 2024-05-09T10:51:24.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.277+0000 7fe7afbce640 1 --2- 172.21.15.98:0/565126709 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fe78c06b6b0 0x7fe78c06db70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:24.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.277+0000 7fe7afbce640 1 --2- 172.21.15.98:0/565126709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7a81f3950 0x7fe7a81f3de0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:24.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.277+0000 7fe7afbce640 1 --2- 172.21.15.98:0/565126709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7a815dd60 0x7fe7a80b71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:24.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.277+0000 7fe7afbce640 1 --2- 172.21.15.98:0/565126709 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7a80d1de0 0x7fe7a815a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:24.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.277+0000 7fe7afbce640 1 -- 172.21.15.98:0/565126709 >> 172.21.15.98:0/565126709 conn(0x7fe7a80d3b50 msgr2=0x7fe7a806bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:24.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.277+0000 7fe7afbce640 1 -- 172.21.15.98:0/565126709 shutdown_connections 2024-05-09T10:51:24.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.277+0000 7fe7afbce640 1 -- 172.21.15.98:0/565126709 wait complete. 2024-05-09T10:51:24.281 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T10:51:24.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.389+0000 7fa65375c640 1 Processor -- start 2024-05-09T10:51:24.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.390+0000 7fa65375c640 1 -- start start 2024-05-09T10:51:24.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.390+0000 7fa65375c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa64c15f4e0 0x7fa64c0ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:24.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.390+0000 7fa65375c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa64c0afe40 0x7fa64c0b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:24.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.390+0000 7fa65375c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa64c0aeda0 con 0x7fa64c0b07e0 2024-05-09T10:51:24.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.390+0000 7fa65375c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa64c0af080 con 0x7fa64c0afe40 2024-05-09T10:51:24.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.390+0000 7fa65375c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0af360 con 0x7fa64c15f4e0 2024-05-09T10:51:24.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.391+0000 7fa6514f8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa64c15f4e0 0x7fa64c0ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:24.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.391+0000 7fa650cf7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa64c0afe40 0x7fa64c0b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:24.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.391+0000 7fa650cf7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa64c0afe40 0x7fa64c0b02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.98:60730/0 (socket says 172.21.15.98:60730) 2024-05-09T10:51:24.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.391+0000 7fa650cf7640 1 -- 172.21.15.98:0/3509855698 learned_addr learned my addr 172.21.15.98:0/3509855698 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:24.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.391+0000 7fa650cf7640 1 -- 172.21.15.98:0/3509855698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa64c15f4e0 msgr2=0x7fa64c0ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:24.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.391+0000 7fa650cf7640 1 --2- 172.21.15.98:0/3509855698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa64c15f4e0 0x7fa64c0ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:24.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.391+0000 7fa650cf7640 1 -- 172.21.15.98:0/3509855698 >> v1:172.21.15.105:6789/0 conn(0x7fa64c0b07e0 legacy=0x7fa64c164c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:24.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.391+0000 7fa650cf7640 1 -- 172.21.15.98:0/3509855698 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64c0af640 con 0x7fa64c0afe40 2024-05-09T10:51:24.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.392+0000 7fa650cf7640 1 --2- 172.21.15.98:0/3509855698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa64c0afe40 0x7fa64c0b02a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa640009e50 tx=0x7fa64002a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eeebab2e28bbcc81 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:24.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.392+0000 7fa64bfff640 1 -- 172.21.15.98:0/3509855698 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640036070 con 0x7fa64c0afe40 2024-05-09T10:51:24.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.392+0000 7fa64bfff640 1 -- 172.21.15.98:0/3509855698 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa640031600 con 0x7fa64c0afe40 2024-05-09T10:51:24.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.392+0000 7fa64bfff640 1 -- 172.21.15.98:0/3509855698 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6400319d0 con 0x7fa64c0afe40 2024-05-09T10:51:24.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.393+0000 7fa65375c640 1 -- 172.21.15.98:0/3509855698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa64c0afe40 msgr2=0x7fa64c0b02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:24.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.393+0000 7fa65375c640 1 --2- 172.21.15.98:0/3509855698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa64c0afe40 0x7fa64c0b02a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa640009e50 tx=0x7fa64002a680 comp rx=0 tx=0).stop 2024-05-09T10:51:24.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.393+0000 7fa65375c640 1 -- 172.21.15.98:0/3509855698 shutdown_connections 2024-05-09T10:51:24.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.393+0000 7fa65375c640 1 --2- 172.21.15.98:0/3509855698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa64c0afe40 0x7fa64c0b02a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:24.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.393+0000 7fa65375c640 1 --2- 172.21.15.98:0/3509855698 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa64c15f4e0 0x7fa64c0ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:24.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.393+0000 7fa65375c640 1 -- 172.21.15.98:0/3509855698 >> 172.21.15.98:0/3509855698 conn(0x7fa64c0b34e0 msgr2=0x7fa64c0b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:24.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.393+0000 7fa65375c640 1 -- 172.21.15.98:0/3509855698 shutdown_connections 2024-05-09T10:51:24.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.393+0000 7fa65375c640 1 -- 172.21.15.98:0/3509855698 wait complete. 2024-05-09T10:51:24.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.394+0000 7fa65375c640 1 Processor -- start 2024-05-09T10:51:24.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.394+0000 7fa65375c640 1 -- start start 2024-05-09T10:51:24.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.395+0000 7fa65375c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa64c0afe40 0x7fa64c0b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:24.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.395+0000 7fa65375c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa64c0b07e0 0x7fa64c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:24.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.395+0000 7fa65375c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa64c15f4e0 0x7fa64c0b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:24.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.395+0000 7fa65375c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c15f8c0 con 0x7fa64c0b07e0 2024-05-09T10:51:24.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.395+0000 7fa65375c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0cbcf0 con 0x7fa64c0afe40 2024-05-09T10:51:24.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.395+0000 7fa65375c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0b0c50 con 0x7fa64c15f4e0 2024-05-09T10:51:24.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.395+0000 7fa650cf7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa64c0b07e0 0x7fa64c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:24.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.395+0000 7fa650cf7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa64c0b07e0 0x7fa64c0b6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60036/0 (socket says 172.21.15.98:60036) 2024-05-09T10:51:24.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.395+0000 7fa6514f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa64c0afe40 0x7fa64c0b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:24.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.395+0000 7fa650cf7640 1 -- 172.21.15.98:0/1026710589 learned_addr learned my addr 172.21.15.98:0/1026710589 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:24.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.395+0000 7fa651cf9640 1 --2- 172.21.15.98:0/1026710589 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa64c15f4e0 0x7fa64c0b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:24.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.395+0000 7fa650cf7640 1 -- 172.21.15.98:0/1026710589 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa64c15f4e0 msgr2=0x7fa64c0b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:24.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.395+0000 7fa650cf7640 1 --2- 172.21.15.98:0/1026710589 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa64c15f4e0 0x7fa64c0b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:24.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.395+0000 7fa650cf7640 1 -- 172.21.15.98:0/1026710589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa64c0afe40 msgr2=0x7fa64c0b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:24.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.395+0000 7fa650cf7640 1 --2- 172.21.15.98:0/1026710589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa64c0afe40 0x7fa64c0b42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:24.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.395+0000 7fa650cf7640 1 -- 172.21.15.98:0/1026710589 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64c0b47e0 con 0x7fa64c0b07e0 2024-05-09T10:51:24.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.395+0000 7fa6514f8640 1 --2- 172.21.15.98:0/1026710589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa64c0afe40 0x7fa64c0b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:51:24.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.395+0000 7fa650cf7640 1 --2- 172.21.15.98:0/1026710589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa64c0b07e0 0x7fa64c0b6170 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa640034040 tx=0x7fa640004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:24.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.395+0000 7fa649ffb640 1 -- 172.21.15.98:0/1026710589 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64003f020 con 0x7fa64c0b07e0 2024-05-09T10:51:24.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.395+0000 7fa649ffb640 1 -- 172.21.15.98:0/1026710589 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa640037040 con 0x7fa64c0b07e0 2024-05-09T10:51:24.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.395+0000 7fa65375c640 1 -- 172.21.15.98:0/1026710589 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64c06c2d0 con 0x7fa64c0b07e0 2024-05-09T10:51:24.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.395+0000 7fa649ffb640 1 -- 172.21.15.98:0/1026710589 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640036070 con 0x7fa64c0b07e0 2024-05-09T10:51:24.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.396+0000 7fa65375c640 1 -- 172.21.15.98:0/1026710589 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64c06c890 con 0x7fa64c0b07e0 2024-05-09T10:51:24.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.397+0000 7fa65375c640 1 -- 172.21.15.98:0/1026710589 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa64c0b7770 con 0x7fa64c0b07e0 2024-05-09T10:51:24.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.397+0000 7fa649ffb640 1 -- 172.21.15.98:0/1026710589 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7fa640042020 con 0x7fa64c0b07e0 2024-05-09T10:51:24.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.398+0000 7fa649ffb640 1 --2- 172.21.15.98:0/1026710589 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa63806b680 0x7fa63806db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:24.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.398+0000 7fa649ffb640 1 -- 172.21.15.98:0/1026710589 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6400aa8f0 con 0x7fa64c0b07e0 2024-05-09T10:51:24.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.398+0000 7fa6514f8640 1 --2- 172.21.15.98:0/1026710589 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa63806b680 0x7fa63806db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:24.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.398+0000 7fa6514f8640 1 --2- 172.21.15.98:0/1026710589 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa63806b680 0x7fa63806db40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa644009e90 tx=0x7fa644009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:24.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.402+0000 7fa649ffb640 1 -- 172.21.15.98:0/1026710589 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa64c0b7770 con 0x7fa64c0b07e0 2024-05-09T10:51:24.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.588+0000 7fa65375c640 1 -- 172.21.15.98:0/1026710589 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa64c0b66b0 con 0x7fa64c0b07e0 2024-05-09T10:51:24.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.588+0000 7fa649ffb640 1 -- 172.21.15.98:0/1026710589 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v31) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa640049030 con 0x7fa64c0b07e0 2024-05-09T10:51:24.582 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-05-09T10:51:24.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.590+0000 7fa65375c640 1 -- 172.21.15.98:0/1026710589 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa63806b680 msgr2=0x7fa63806db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:24.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.590+0000 7fa65375c640 1 --2- 172.21.15.98:0/1026710589 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa63806b680 0x7fa63806db40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa644009e90 tx=0x7fa644009190 comp rx=0 tx=0).stop 2024-05-09T10:51:24.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.590+0000 7fa65375c640 1 -- 172.21.15.98:0/1026710589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa64c0b07e0 msgr2=0x7fa64c0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:24.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.590+0000 7fa65375c640 1 --2- 172.21.15.98:0/1026710589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa64c0b07e0 0x7fa64c0b6170 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa640034040 tx=0x7fa640004a40 comp rx=0 tx=0).stop 2024-05-09T10:51:24.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.590+0000 7fa65375c640 1 -- 172.21.15.98:0/1026710589 shutdown_connections 2024-05-09T10:51:24.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.590+0000 7fa65375c640 1 --2- 172.21.15.98:0/1026710589 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa63806b680 0x7fa63806db40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:24.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.590+0000 7fa65375c640 1 --2- 172.21.15.98:0/1026710589 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa64c15f4e0 0x7fa64c0b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:24.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.590+0000 7fa65375c640 1 --2- 172.21.15.98:0/1026710589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa64c0b07e0 0x7fa64c0b6170 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:24.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.590+0000 7fa65375c640 1 --2- 172.21.15.98:0/1026710589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa64c0afe40 0x7fa64c0b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:24.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.590+0000 7fa65375c640 1 -- 172.21.15.98:0/1026710589 >> 172.21.15.98:0/1026710589 conn(0x7fa64c0b34e0 msgr2=0x7fa64c06e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:24.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.590+0000 7fa65375c640 1 -- 172.21.15.98:0/1026710589 shutdown_connections 2024-05-09T10:51:24.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.591+0000 7fa65375c640 1 -- 172.21.15.98:0/1026710589 wait complete. 2024-05-09T10:51:24.595 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-09T10:51:24.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:24.595 DEBUG:teuthology.orchestra.run.smithi098:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-09T10:51:24.623 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-05-09T10:51:24.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:24.624 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T10:51:24.680 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:51:24.680 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:51:24.700 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:51:24.700 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:51:24.758 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:51:24.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.865+0000 7f1d98e07640 1 Processor -- start 2024-05-09T10:51:24.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.865+0000 7f1d98e07640 1 -- start start 2024-05-09T10:51:24.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.866+0000 7f1d98e07640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1d9406bbf0 0x7f1d9406bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:24.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.866+0000 7f1d98e07640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1d94072a20 0x7f1d940ce750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:24.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.866+0000 7f1d98e07640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d940cbf50 con 0x7f1d940ce2c0 2024-05-09T10:51:24.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.866+0000 7f1d98e07640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d940cc1c0 con 0x7f1d94072a20 2024-05-09T10:51:24.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.866+0000 7f1d98e07640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d9406f2c0 con 0x7f1d9406bbf0 2024-05-09T10:51:24.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.866+0000 7f1d92d9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1d94072a20 0x7f1d940ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:24.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.866+0000 7f1d89d9b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f1d940ce2c0 0x7f1d94072310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46956/0 (socket says 172.21.15.98:46956) 2024-05-09T10:51:24.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.866+0000 7f1d89d9b640 1 -- 172.21.15.98:0/82902735 learned_addr learned my addr 172.21.15.98:0/82902735 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:24.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.866+0000 7f1d92d9d640 1 -- 172.21.15.98:0/82902735 >> v2:172.21.15.105:3300/0 conn(0x7f1d9406bbf0 msgr2=0x7f1d9406bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:24.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.866+0000 7f1d9259c640 1 --2- 172.21.15.98:0/82902735 >> v2:172.21.15.105:3300/0 conn(0x7f1d9406bbf0 0x7f1d9406bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:24.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.866+0000 7f1d92d9d640 1 --2- 172.21.15.98:0/82902735 >> v2:172.21.15.105:3300/0 conn(0x7f1d9406bbf0 0x7f1d9406bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:24.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.866+0000 7f1d91d9b640 1 -- 172.21.15.98:0/82902735 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1956527858 0 0) 0x7f1d940cbf50 con 0x7f1d940ce2c0 2024-05-09T10:51:24.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.866+0000 7f1d92d9d640 1 -- 172.21.15.98:0/82902735 >> v1:172.21.15.98:6789/0 conn(0x7f1d940ce2c0 legacy=0x7f1d94072310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:24.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.866+0000 7f1d92d9d640 1 -- 172.21.15.98:0/82902735 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9406c510 con 0x7f1d94072a20 2024-05-09T10:51:24.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.866+0000 7f1d9259c640 1 --2- 172.21.15.98:0/82902735 >> v2:172.21.15.105:3300/0 conn(0x7f1d9406bbf0 0x7f1d9406bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:24.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.867+0000 7f1d92d9d640 1 --2- 172.21.15.98:0/82902735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d94072a20 0x7f1d940ce750 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1d94070730 tx=0x7f1d8c02a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6652f66a0816f57 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:24.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.867+0000 7f1d91d9b640 1 -- 172.21.15.98:0/82902735 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c007740 con 0x7f1d94072a20 2024-05-09T10:51:24.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.867+0000 7f1d91d9b640 1 -- 172.21.15.98:0/82902735 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c0078e0 con 0x7f1d94072a20 2024-05-09T10:51:24.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.867+0000 7f1d91d9b640 1 -- 172.21.15.98:0/82902735 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c007cb0 con 0x7f1d94072a20 2024-05-09T10:51:24.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.868+0000 7f1d98e07640 1 -- 172.21.15.98:0/82902735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d94072a20 msgr2=0x7f1d940ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:24.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.868+0000 7f1d98e07640 1 --2- 172.21.15.98:0/82902735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d94072a20 0x7f1d940ce750 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1d94070730 tx=0x7f1d8c02a560 comp rx=0 tx=0).stop 2024-05-09T10:51:24.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.868+0000 7f1d98e07640 1 -- 172.21.15.98:0/82902735 shutdown_connections 2024-05-09T10:51:24.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.868+0000 7f1d98e07640 1 --2- 172.21.15.98:0/82902735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d94072a20 0x7f1d940ce750 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:24.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.868+0000 7f1d98e07640 1 --2- 172.21.15.98:0/82902735 >> v2:172.21.15.105:3300/0 conn(0x7f1d9406bbf0 0x7f1d9406bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:24.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.868+0000 7f1d98e07640 1 -- 172.21.15.98:0/82902735 >> 172.21.15.98:0/82902735 conn(0x7f1d940ccaa0 msgr2=0x7f1d940cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:24.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.868+0000 7f1d98e07640 1 -- 172.21.15.98:0/82902735 shutdown_connections 2024-05-09T10:51:24.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.868+0000 7f1d98e07640 1 -- 172.21.15.98:0/82902735 wait complete. 2024-05-09T10:51:24.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.869+0000 7f1d98e07640 1 Processor -- start 2024-05-09T10:51:24.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.869+0000 7f1d98e07640 1 -- start start 2024-05-09T10:51:24.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.870+0000 7f1d98e07640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d9406bbf0 0x7f1d941f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:24.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.870+0000 7f1d98e07640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d94072a20 0x7f1d940c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:24.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.870+0000 7f1d98e07640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1d940ce2c0 0x7f1d940c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:24.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.870+0000 7f1d98e07640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d940c37d0 con 0x7f1d94072a20 2024-05-09T10:51:24.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.870+0000 7f1d98e07640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d940cb820 con 0x7f1d9406bbf0 2024-05-09T10:51:24.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.870+0000 7f1d98e07640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d940c3950 con 0x7f1d940ce2c0 2024-05-09T10:51:24.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.870+0000 7f1d89d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d94072a20 0x7f1d940c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:24.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.870+0000 7f1d89d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d94072a20 0x7f1d940c3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60054/0 (socket says 172.21.15.98:60054) 2024-05-09T10:51:24.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.870+0000 7f1d89d9b640 1 -- 172.21.15.98:0/2222471935 learned_addr learned my addr 172.21.15.98:0/2222471935 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:24.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.870+0000 7f1d9259c640 1 --2- 172.21.15.98:0/2222471935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d9406bbf0 0x7f1d941f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:24.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.870+0000 7f1d92d9d640 1 --2- 172.21.15.98:0/2222471935 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1d940ce2c0 0x7f1d940c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:24.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.870+0000 7f1d89d9b640 1 -- 172.21.15.98:0/2222471935 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1d940ce2c0 msgr2=0x7f1d940c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:24.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.870+0000 7f1d89d9b640 1 --2- 172.21.15.98:0/2222471935 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1d940ce2c0 0x7f1d940c8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:24.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.871+0000 7f1d89d9b640 1 -- 172.21.15.98:0/2222471935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d9406bbf0 msgr2=0x7f1d941f67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:24.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.871+0000 7f1d89d9b640 1 --2- 172.21.15.98:0/2222471935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d9406bbf0 0x7f1d941f67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:24.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.871+0000 7f1d89d9b640 1 -- 172.21.15.98:0/2222471935 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d940c43f0 con 0x7f1d94072a20 2024-05-09T10:51:24.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.871+0000 7f1d9259c640 1 --2- 172.21.15.98:0/2222471935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d9406bbf0 0x7f1d941f67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:24.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.871+0000 7f1d89d9b640 1 --2- 172.21.15.98:0/2222471935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d94072a20 0x7f1d940c3eb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00c420 tx=0x7f1d7c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:24.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.871+0000 7f1d8b7fe640 1 -- 172.21.15.98:0/2222471935 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c0074e0 con 0x7f1d94072a20 2024-05-09T10:51:24.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.871+0000 7f1d8b7fe640 1 -- 172.21.15.98:0/2222471935 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7c0047e0 con 0x7f1d94072a20 2024-05-09T10:51:24.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.871+0000 7f1d98e07640 1 -- 172.21.15.98:0/2222471935 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9409b030 con 0x7f1d94072a20 2024-05-09T10:51:24.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.871+0000 7f1d8b7fe640 1 -- 172.21.15.98:0/2222471935 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c004c10 con 0x7f1d94072a20 2024-05-09T10:51:24.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.871+0000 7f1d98e07640 1 -- 172.21.15.98:0/2222471935 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9409b5f0 con 0x7f1d94072a20 2024-05-09T10:51:24.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.872+0000 7f1d98e07640 1 -- 172.21.15.98:0/2222471935 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d940cbe90 con 0x7f1d94072a20 2024-05-09T10:51:24.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.872+0000 7f1d8b7fe640 1 -- 172.21.15.98:0/2222471935 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f1d7c011070 con 0x7f1d94072a20 2024-05-09T10:51:24.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.873+0000 7f1d8b7fe640 1 --2- 172.21.15.98:0/2222471935 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1d7406b680 0x7f1d7406db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:24.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.873+0000 7f1d8b7fe640 1 -- 172.21.15.98:0/2222471935 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d7c08a130 con 0x7f1d94072a20 2024-05-09T10:51:24.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.873+0000 7f1d9259c640 1 --2- 172.21.15.98:0/2222471935 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1d7406b680 0x7f1d7406db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:24.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.873+0000 7f1d9259c640 1 --2- 172.21.15.98:0/2222471935 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1d7406b680 0x7f1d7406db40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1d9409cb50 tx=0x7f1d84008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:24.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:24.877+0000 7f1d8b7fe640 1 -- 172.21.15.98:0/2222471935 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1d7c00e280 con 0x7f1d94072a20 2024-05-09T10:51:25.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.050+0000 7f1d98e07640 1 -- 172.21.15.98:0/2222471935 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d94069cb0 con 0x7f1d94072a20 2024-05-09T10:51:25.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.051+0000 7f1d8b7fe640 1 -- 172.21.15.98:0/2222471935 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+16669 (secure 0 0 0) 0x7f1d7c058d30 con 0x7f1d94072a20 2024-05-09T10:51:25.045 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:25.045 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":31,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:51:08.400895+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:51:05.150410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-09T10:51:05.708391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:51:25.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.053+0000 7f1d98e07640 1 -- 172.21.15.98:0/2222471935 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1d7406b680 msgr2=0x7f1d7406db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:25.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.053+0000 7f1d98e07640 1 --2- 172.21.15.98:0/2222471935 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1d7406b680 0x7f1d7406db40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1d9409cb50 tx=0x7f1d84008040 comp rx=0 tx=0).stop 2024-05-09T10:51:25.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.053+0000 7f1d98e07640 1 -- 172.21.15.98:0/2222471935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d94072a20 msgr2=0x7f1d940c3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:25.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.053+0000 7f1d98e07640 1 --2- 172.21.15.98:0/2222471935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d94072a20 0x7f1d940c3eb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00c420 tx=0x7f1d7c00c8e0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.053+0000 7f1d98e07640 1 -- 172.21.15.98:0/2222471935 shutdown_connections 2024-05-09T10:51:25.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.053+0000 7f1d98e07640 1 --2- 172.21.15.98:0/2222471935 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1d7406b680 0x7f1d7406db40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.053+0000 7f1d98e07640 1 --2- 172.21.15.98:0/2222471935 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1d940ce2c0 0x7f1d940c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.053+0000 7f1d98e07640 1 --2- 172.21.15.98:0/2222471935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d94072a20 0x7f1d940c3eb0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.053+0000 7f1d98e07640 1 --2- 172.21.15.98:0/2222471935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d9406bbf0 0x7f1d941f67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.053+0000 7f1d98e07640 1 -- 172.21.15.98:0/2222471935 >> 172.21.15.98:0/2222471935 conn(0x7f1d940ccaa0 msgr2=0x7f1d9406e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:25.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.053+0000 7f1d98e07640 1 -- 172.21.15.98:0/2222471935 shutdown_connections 2024-05-09T10:51:25.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.053+0000 7f1d98e07640 1 -- 172.21.15.98:0/2222471935 wait complete. 2024-05-09T10:51:25.057 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T10:50:07.396423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T10:51:05.150410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T10:51:05.708391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T10:51:25.058 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T10:51:25.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.168+0000 7f23e1f6a640 1 Processor -- start 2024-05-09T10:51:25.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.168+0000 7f23e1f6a640 1 -- start start 2024-05-09T10:51:25.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.168+0000 7f23e1f6a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23dc0d2820 0x7f23dc0d4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:25.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.168+0000 7f23e1f6a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23dc161830 con 0x7f23dc161450 2024-05-09T10:51:25.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.168+0000 7f23e1f6a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23dc0cf2f0 con 0x7f23dc0b2a30 2024-05-09T10:51:25.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.168+0000 7f23e1f6a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc0b0760 con 0x7f23dc0d2820 2024-05-09T10:51:25.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.169+0000 7f23db7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f23dc161450 0x7f23dc0d2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46970/0 (socket says 172.21.15.98:46970) 2024-05-09T10:51:25.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.169+0000 7f23db7fe640 1 -- 172.21.15.98:0/2473794665 learned_addr learned my addr 172.21.15.98:0/2473794665 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:25.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.169+0000 7f23d3fff640 1 --2- 172.21.15.98:0/2473794665 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23dc0d2820 0x7f23dc0d4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:25.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.169+0000 7f23daffd640 1 -- 172.21.15.98:0/2473794665 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 93924093 0 0) 0x7f23cc001110 con 0x7f23dc161450 2024-05-09T10:51:25.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.169+0000 7f23daffd640 1 -- 172.21.15.98:0/2473794665 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23c8003520 con 0x7f23dc161450 2024-05-09T10:51:25.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.169+0000 7f23daffd640 1 -- 172.21.15.98:0/2473794665 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 476525532 0 0) 0x7f23c8003520 con 0x7f23dc161450 2024-05-09T10:51:25.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.169+0000 7f23daffd640 1 -- 172.21.15.98:0/2473794665 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23dc0d2820 msgr2=0x7f23dc0d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:25.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.169+0000 7f23daffd640 1 --2- 172.21.15.98:0/2473794665 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23dc0d2820 0x7f23dc0d4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.169+0000 7f23daffd640 1 -- 172.21.15.98:0/2473794665 >> v1:172.21.15.105:6789/0 conn(0x7f23dc0b2a30 legacy=0x7f23dc0b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:25.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.169+0000 7f23daffd640 1 -- 172.21.15.98:0/2473794665 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23dc0cf5d0 con 0x7f23dc161450 2024-05-09T10:51:25.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.169+0000 7f23daffd640 1 -- 172.21.15.98:0/2473794665 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f23cc001840 con 0x7f23dc161450 2024-05-09T10:51:25.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.169+0000 7f23daffd640 1 -- 172.21.15.98:0/2473794665 >> v1:172.21.15.98:6789/0 conn(0x7f23dc161450 legacy=0x7f23dc0d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:25.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.169+0000 7f23daffd640 1 --2- 172.21.15.98:0/2473794665 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23dc0d2820 0x7f23c80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:25.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.169+0000 7f23daffd640 1 --2- 172.21.15.98:0/2473794665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23c8004010 0x7f23c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:25.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.169+0000 7f23daffd640 1 --2- 172.21.15.98:0/2473794665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23c8004d40 0x7f23c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:25.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.169+0000 7f23daffd640 1 -- 172.21.15.98:0/2473794665 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc0b0760 con 0x7f23c8004010 2024-05-09T10:51:25.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.169+0000 7f23daffd640 1 -- 172.21.15.98:0/2473794665 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c800bc00 con 0x7f23c8004d40 2024-05-09T10:51:25.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.169+0000 7f23daffd640 1 -- 172.21.15.98:0/2473794665 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c800be00 con 0x7f23dc0d2820 2024-05-09T10:51:25.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.169+0000 7f23d3fff640 1 --2- 172.21.15.98:0/2473794665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23c8004010 0x7f23c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:25.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.169+0000 7f23db7fe640 1 --2- 172.21.15.98:0/2473794665 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23dc0d2820 0x7f23c80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:25.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.169+0000 7f23dbfff640 1 --2- 172.21.15.98:0/2473794665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23c8004d40 0x7f23c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:25.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.170+0000 7f23d3fff640 1 -- 172.21.15.98:0/2473794665 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23dc0d2820 msgr2=0x7f23c80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:25.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.170+0000 7f23d3fff640 1 --2- 172.21.15.98:0/2473794665 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23dc0d2820 0x7f23c80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.170+0000 7f23d3fff640 1 -- 172.21.15.98:0/2473794665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23c8004d40 msgr2=0x7f23c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:25.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.170+0000 7f23d3fff640 1 --2- 172.21.15.98:0/2473794665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23c8004d40 0x7f23c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.170+0000 7f23d3fff640 1 -- 172.21.15.98:0/2473794665 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c8003520 con 0x7f23c8004010 2024-05-09T10:51:25.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.170+0000 7f23d3fff640 1 --2- 172.21.15.98:0/2473794665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23c8004010 0x7f23c8004800 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f23c400ba30 tx=0x7f23c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28806d21ecf89eac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:25.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.170+0000 7f23daffd640 1 -- 172.21.15.98:0/2473794665 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c4030070 con 0x7f23c8004010 2024-05-09T10:51:25.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.170+0000 7f23daffd640 1 -- 172.21.15.98:0/2473794665 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c402ade0 con 0x7f23c8004010 2024-05-09T10:51:25.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.170+0000 7f23daffd640 1 -- 172.21.15.98:0/2473794665 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c400b600 con 0x7f23c8004010 2024-05-09T10:51:25.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.171+0000 7f23e1f6a640 1 -- 172.21.15.98:0/2473794665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23c8004010 msgr2=0x7f23c8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:25.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.171+0000 7f23e1f6a640 1 --2- 172.21.15.98:0/2473794665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23c8004010 0x7f23c8004800 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f23c400ba30 tx=0x7f23c8005830 comp rx=0 tx=0).stop 2024-05-09T10:51:25.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.171+0000 7f23dbfff640 1 -- 172.21.15.98:0/2473794665 reap_dead start 2024-05-09T10:51:25.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.171+0000 7f23e1f6a640 1 -- 172.21.15.98:0/2473794665 shutdown_connections 2024-05-09T10:51:25.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.171+0000 7f23e1f6a640 1 -- 172.21.15.98:0/2473794665 >> 172.21.15.98:0/2473794665 conn(0x7f23dc0d3b30 msgr2=0x7f23dc0d3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:25.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.171+0000 7f23e1f6a640 1 -- 172.21.15.98:0/2473794665 shutdown_connections 2024-05-09T10:51:25.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.171+0000 7f23e1f6a640 1 -- 172.21.15.98:0/2473794665 wait complete. 2024-05-09T10:51:25.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.172+0000 7f23e1f6a640 1 Processor -- start 2024-05-09T10:51:25.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.172+0000 7f23e1f6a640 1 -- start start 2024-05-09T10:51:25.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.173+0000 7f23e1f6a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23c8004010 0x7f23dc0b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:25.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.173+0000 7f23e1f6a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23dc15abf0 0x7f23dc0c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:25.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.173+0000 7f23e1f6a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23dc15d180 0x7f23dc15d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:25.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.173+0000 7f23e1f6a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc0d4650 con 0x7f23dc15d180 2024-05-09T10:51:25.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.173+0000 7f23e1f6a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc0b09b0 con 0x7f23dc15abf0 2024-05-09T10:51:25.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.173+0000 7f23e1f6a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc0d47d0 con 0x7f23c8004010 2024-05-09T10:51:25.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.173+0000 7f23dbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23dc15d180 0x7f23dc15d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:25.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.173+0000 7f23dbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23dc15d180 0x7f23dc15d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60066/0 (socket says 172.21.15.98:60066) 2024-05-09T10:51:25.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.173+0000 7f23dbfff640 1 -- 172.21.15.98:0/1341883771 learned_addr learned my addr 172.21.15.98:0/1341883771 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:25.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.173+0000 7f23d3fff640 1 --2- 172.21.15.98:0/1341883771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23dc15abf0 0x7f23dc0c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:25.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.173+0000 7f23dbfff640 1 -- 172.21.15.98:0/1341883771 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23c8004010 msgr2=0x7f23dc0b70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:25.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.173+0000 7f23dbfff640 1 --2- 172.21.15.98:0/1341883771 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23c8004010 0x7f23dc0b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.173+0000 7f23dbfff640 1 -- 172.21.15.98:0/1341883771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23dc15abf0 msgr2=0x7f23dc0c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:25.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.173+0000 7f23dbfff640 1 --2- 172.21.15.98:0/1341883771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23dc15abf0 0x7f23dc0c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.173+0000 7f23dbfff640 1 -- 172.21.15.98:0/1341883771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23dc0ca440 con 0x7f23dc15d180 2024-05-09T10:51:25.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.173+0000 7f23d3fff640 1 --2- 172.21.15.98:0/1341883771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23dc15abf0 0x7f23dc0c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:25.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.173+0000 7f23dbfff640 1 --2- 172.21.15.98:0/1341883771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23dc15d180 0x7f23dc15d600 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f23dc08e310 tx=0x7f23d400fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:25.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.174+0000 7f23d8ff9640 1 -- 172.21.15.98:0/1341883771 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d4003b80 con 0x7f23dc15d180 2024-05-09T10:51:25.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.174+0000 7f23d8ff9640 1 -- 172.21.15.98:0/1341883771 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d4018070 con 0x7f23dc15d180 2024-05-09T10:51:25.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.174+0000 7f23d8ff9640 1 -- 172.21.15.98:0/1341883771 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d40050d0 con 0x7f23dc15d180 2024-05-09T10:51:25.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.174+0000 7f23e1f6a640 1 -- 172.21.15.98:0/1341883771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23dc0b7760 con 0x7f23dc15d180 2024-05-09T10:51:25.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.174+0000 7f23e1f6a640 1 -- 172.21.15.98:0/1341883771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23dc0b7d20 con 0x7f23dc15d180 2024-05-09T10:51:25.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.175+0000 7f23e1f6a640 1 -- 172.21.15.98:0/1341883771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23dc0a1ae0 con 0x7f23dc15d180 2024-05-09T10:51:25.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.175+0000 7f23d8ff9640 1 -- 172.21.15.98:0/1341883771 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f23d4005270 con 0x7f23dc15d180 2024-05-09T10:51:25.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.176+0000 7f23d8ff9640 1 --2- 172.21.15.98:0/1341883771 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f23bc06b680 0x7f23bc06db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:25.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.176+0000 7f23d8ff9640 1 -- 172.21.15.98:0/1341883771 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23d408a430 con 0x7f23dc15d180 2024-05-09T10:51:25.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.176+0000 7f23db7fe640 1 --2- 172.21.15.98:0/1341883771 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f23bc06b680 0x7f23bc06db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:25.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.176+0000 7f23db7fe640 1 --2- 172.21.15.98:0/1341883771 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f23bc06b680 0x7f23bc06db40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f23cc0012b0 tx=0x7f23cc001d50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:25.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.180+0000 7f23d8ff9640 1 -- 172.21.15.98:0/1341883771 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f23d401a310 con 0x7f23dc15d180 2024-05-09T10:51:25.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.352+0000 7f23e1f6a640 1 -- 172.21.15.98:0/1341883771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f23dc15b640 con 0x7f23dc15d180 2024-05-09T10:51:25.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.353+0000 7f23d8ff9640 1 -- 172.21.15.98:0/1341883771 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) v1 ==== 93+0+10 (secure 0 0 0) 0x7f23d4058fa0 con 0x7f23dc15d180 2024-05-09T10:51:25.346 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-05-09T10:51:25.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.355+0000 7f23e1f6a640 1 -- 172.21.15.98:0/1341883771 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f23bc06b680 msgr2=0x7f23bc06db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:25.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.355+0000 7f23e1f6a640 1 --2- 172.21.15.98:0/1341883771 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f23bc06b680 0x7f23bc06db40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f23cc0012b0 tx=0x7f23cc001d50 comp rx=0 tx=0).stop 2024-05-09T10:51:25.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.355+0000 7f23e1f6a640 1 -- 172.21.15.98:0/1341883771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23dc15d180 msgr2=0x7f23dc15d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:25.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.355+0000 7f23e1f6a640 1 --2- 172.21.15.98:0/1341883771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23dc15d180 0x7f23dc15d600 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f23dc08e310 tx=0x7f23d400fd40 comp rx=0 tx=0).stop 2024-05-09T10:51:25.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.355+0000 7f23e1f6a640 1 -- 172.21.15.98:0/1341883771 shutdown_connections 2024-05-09T10:51:25.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.355+0000 7f23e1f6a640 1 --2- 172.21.15.98:0/1341883771 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f23bc06b680 0x7f23bc06db40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.355+0000 7f23e1f6a640 1 --2- 172.21.15.98:0/1341883771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23dc15d180 0x7f23dc15d600 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.355+0000 7f23e1f6a640 1 --2- 172.21.15.98:0/1341883771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23dc15abf0 0x7f23dc0c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.355+0000 7f23e1f6a640 1 --2- 172.21.15.98:0/1341883771 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23c8004010 0x7f23dc0b70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.355+0000 7f23e1f6a640 1 -- 172.21.15.98:0/1341883771 >> 172.21.15.98:0/1341883771 conn(0x7f23dc0d3b30 msgr2=0x7f23dc0cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:25.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.355+0000 7f23e1f6a640 1 -- 172.21.15.98:0/1341883771 shutdown_connections 2024-05-09T10:51:25.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.355+0000 7f23e1f6a640 1 -- 172.21.15.98:0/1341883771 wait complete. 2024-05-09T10:51:25.359 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T10:51:25.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.470+0000 7f8f3c041640 1 Processor -- start 2024-05-09T10:51:25.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.470+0000 7f8f3c041640 1 -- start start 2024-05-09T10:51:25.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.471+0000 7f8f3c041640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8f34161450 0x7f8f340d4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:25.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.471+0000 7f8f3c041640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f34161830 con 0x7f8f340b2a30 2024-05-09T10:51:25.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.471+0000 7f8f3c041640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f340cf2f0 con 0x7f8f340d2820 2024-05-09T10:51:25.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.471+0000 7f8f3c041640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f340b0760 con 0x7f8f34161450 2024-05-09T10:51:25.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.471+0000 7f8f395dc640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f8f340b2a30 0x7f8f340d2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46984/0 (socket says 172.21.15.98:46984) 2024-05-09T10:51:25.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.471+0000 7f8f395dc640 1 -- 172.21.15.98:0/4064087315 learned_addr learned my addr 172.21.15.98:0/4064087315 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:25.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.471+0000 7f8f38ddb640 1 -- 172.21.15.98:0/4064087315 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4030744627 0 0) 0x7f8f28001110 con 0x7f8f340b2a30 2024-05-09T10:51:25.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.471+0000 7f8f38ddb640 1 -- 172.21.15.98:0/4064087315 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f1c003520 con 0x7f8f340b2a30 2024-05-09T10:51:25.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.471+0000 7f8f39ddd640 1 --2- 172.21.15.98:0/4064087315 >> v2:172.21.15.105:3300/0 conn(0x7f8f34161450 0x7f8f340d4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:25.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f38ddb640 1 -- 172.21.15.98:0/4064087315 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3793647518 0 0) 0x7f8f1c003520 con 0x7f8f340b2a30 2024-05-09T10:51:25.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f38ddb640 1 -- 172.21.15.98:0/4064087315 >> v2:172.21.15.105:3300/0 conn(0x7f8f34161450 msgr2=0x7f8f340d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:25.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f38ddb640 1 --2- 172.21.15.98:0/4064087315 >> v2:172.21.15.105:3300/0 conn(0x7f8f34161450 0x7f8f340d4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f38ddb640 1 -- 172.21.15.98:0/4064087315 >> v1:172.21.15.105:6789/0 conn(0x7f8f340d2820 legacy=0x7f8f340b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:25.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f38ddb640 1 -- 172.21.15.98:0/4064087315 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f340cf5d0 con 0x7f8f340b2a30 2024-05-09T10:51:25.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f38ddb640 1 -- 172.21.15.98:0/4064087315 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f8f28001840 con 0x7f8f340b2a30 2024-05-09T10:51:25.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f38ddb640 1 -- 172.21.15.98:0/4064087315 >> v1:172.21.15.98:6789/0 conn(0x7f8f340b2a30 legacy=0x7f8f340d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:25.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f38ddb640 1 --2- 172.21.15.98:0/4064087315 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f1c0039b0 0x7f8f1c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:25.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f38ddb640 1 --2- 172.21.15.98:0/4064087315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f1c004800 0x7f8f1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:25.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f38ddb640 1 --2- 172.21.15.98:0/4064087315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f1c0051c0 0x7f8f1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:25.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f38ddb640 1 -- 172.21.15.98:0/4064087315 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f340b0760 con 0x7f8f1c004800 2024-05-09T10:51:25.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f38ddb640 1 -- 172.21.15.98:0/4064087315 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c00c160 con 0x7f8f1c0051c0 2024-05-09T10:51:25.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f38ddb640 1 -- 172.21.15.98:0/4064087315 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c014b70 con 0x7f8f1c0039b0 2024-05-09T10:51:25.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f395dc640 1 --2- 172.21.15.98:0/4064087315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f1c004800 0x7f8f1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:25.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f39ddd640 1 --2- 172.21.15.98:0/4064087315 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f1c0039b0 0x7f8f1c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:25.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f3a5de640 1 --2- 172.21.15.98:0/4064087315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f1c0051c0 0x7f8f1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:25.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f395dc640 1 -- 172.21.15.98:0/4064087315 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f1c0039b0 msgr2=0x7f8f1c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:25.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f395dc640 1 --2- 172.21.15.98:0/4064087315 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f1c0039b0 0x7f8f1c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f395dc640 1 -- 172.21.15.98:0/4064087315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f1c0051c0 msgr2=0x7f8f1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:25.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f395dc640 1 --2- 172.21.15.98:0/4064087315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f1c0051c0 0x7f8f1c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f395dc640 1 -- 172.21.15.98:0/4064087315 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1c003520 con 0x7f8f1c004800 2024-05-09T10:51:25.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f3a5de640 1 --2- 172.21.15.98:0/4064087315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f1c0051c0 0x7f8f1c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:25.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f3a5de640 1 -- 172.21.15.98:0/4064087315 reap_dead start 2024-05-09T10:51:25.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.472+0000 7f8f39ddd640 1 --2- 172.21.15.98:0/4064087315 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f1c0039b0 0x7f8f1c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:25.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.473+0000 7f8f395dc640 1 --2- 172.21.15.98:0/4064087315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f1c004800 0x7f8f1c004c80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8f2800c4c0 tx=0x7f8f1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf11f870ed5674d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:25.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.473+0000 7f8f38ddb640 1 -- 172.21.15.98:0/4064087315 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2800c0e0 con 0x7f8f1c004800 2024-05-09T10:51:25.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.473+0000 7f8f38ddb640 1 -- 172.21.15.98:0/4064087315 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f28034070 con 0x7f8f1c004800 2024-05-09T10:51:25.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.473+0000 7f8f38ddb640 1 -- 172.21.15.98:0/4064087315 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f28005d70 con 0x7f8f1c004800 2024-05-09T10:51:25.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.474+0000 7f8f3c041640 1 -- 172.21.15.98:0/4064087315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f1c004800 msgr2=0x7f8f1c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:25.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.474+0000 7f8f3c041640 1 --2- 172.21.15.98:0/4064087315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f1c004800 0x7f8f1c004c80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8f2800c4c0 tx=0x7f8f1c005830 comp rx=0 tx=0).stop 2024-05-09T10:51:25.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.474+0000 7f8f3c041640 1 -- 172.21.15.98:0/4064087315 shutdown_connections 2024-05-09T10:51:25.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.474+0000 7f8f3c041640 1 --2- 172.21.15.98:0/4064087315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f1c004800 0x7f8f1c004c80 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.474+0000 7f8f3c041640 1 -- 172.21.15.98:0/4064087315 >> 172.21.15.98:0/4064087315 conn(0x7f8f340d3b30 msgr2=0x7f8f340d3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:25.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.474+0000 7f8f3c041640 1 -- 172.21.15.98:0/4064087315 shutdown_connections 2024-05-09T10:51:25.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.474+0000 7f8f3c041640 1 -- 172.21.15.98:0/4064087315 wait complete. 2024-05-09T10:51:25.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.475+0000 7f8f3c041640 1 Processor -- start 2024-05-09T10:51:25.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.475+0000 7f8f3c041640 1 -- start start 2024-05-09T10:51:25.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.476+0000 7f8f3c041640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f1c004800 0x7f8f34157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:25.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.476+0000 7f8f3c041640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f340d0aa0 0x7f8f340c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:25.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.476+0000 7f8f3c041640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f34159540 0x7f8f34160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:25.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.476+0000 7f8f3c041640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f340d4650 con 0x7f8f34159540 2024-05-09T10:51:25.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.476+0000 7f8f3c041640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f340b09b0 con 0x7f8f1c004800 2024-05-09T10:51:25.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.476+0000 7f8f3c041640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f340d47d0 con 0x7f8f340d0aa0 2024-05-09T10:51:25.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.476+0000 7f8f3a5de640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f34159540 0x7f8f34160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:25.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.476+0000 7f8f3a5de640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f34159540 0x7f8f34160280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60084/0 (socket says 172.21.15.98:60084) 2024-05-09T10:51:25.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.476+0000 7f8f3a5de640 1 -- 172.21.15.98:0/4058990118 learned_addr learned my addr 172.21.15.98:0/4058990118 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:25.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.476+0000 7f8f395dc640 1 --2- 172.21.15.98:0/4058990118 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f340d0aa0 0x7f8f340c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:25.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.476+0000 7f8f39ddd640 1 --2- 172.21.15.98:0/4058990118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f1c004800 0x7f8f34157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:25.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.476+0000 7f8f3a5de640 1 -- 172.21.15.98:0/4058990118 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f340d0aa0 msgr2=0x7f8f340c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:25.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.476+0000 7f8f3a5de640 1 --2- 172.21.15.98:0/4058990118 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f340d0aa0 0x7f8f340c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.476+0000 7f8f3a5de640 1 -- 172.21.15.98:0/4058990118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f1c004800 msgr2=0x7f8f34157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:25.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.476+0000 7f8f3a5de640 1 --2- 172.21.15.98:0/4058990118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f1c004800 0x7f8f34157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.476+0000 7f8f3a5de640 1 -- 172.21.15.98:0/4058990118 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f34158aa0 con 0x7f8f34159540 2024-05-09T10:51:25.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.476+0000 7f8f3a5de640 1 --2- 172.21.15.98:0/4058990118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f34159540 0x7f8f34160280 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8f3408cec0 tx=0x7f8f2400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:25.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.476+0000 7f8f327fc640 1 -- 172.21.15.98:0/4058990118 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f24003970 con 0x7f8f34159540 2024-05-09T10:51:25.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.476+0000 7f8f327fc640 1 -- 172.21.15.98:0/4058990118 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f24018070 con 0x7f8f34159540 2024-05-09T10:51:25.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.476+0000 7f8f327fc640 1 -- 172.21.15.98:0/4058990118 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f24003d50 con 0x7f8f34159540 2024-05-09T10:51:25.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.476+0000 7f8f3c041640 1 -- 172.21.15.98:0/4058990118 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f340c7f90 con 0x7f8f34159540 2024-05-09T10:51:25.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.477+0000 7f8f3c041640 1 -- 172.21.15.98:0/4058990118 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f34159020 con 0x7f8f34159540 2024-05-09T10:51:25.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.477+0000 7f8f3c041640 1 -- 172.21.15.98:0/4058990118 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f340a1ae0 con 0x7f8f34159540 2024-05-09T10:51:25.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.478+0000 7f8f327fc640 1 -- 172.21.15.98:0/4058990118 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f8f240050d0 con 0x7f8f34159540 2024-05-09T10:51:25.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.478+0000 7f8f327fc640 1 --2- 172.21.15.98:0/4058990118 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8f2006b680 0x7f8f2006db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:25.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.478+0000 7f8f327fc640 1 -- 172.21.15.98:0/4058990118 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f24089fd0 con 0x7f8f34159540 2024-05-09T10:51:25.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.478+0000 7f8f39ddd640 1 --2- 172.21.15.98:0/4058990118 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8f2006b680 0x7f8f2006db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:25.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.479+0000 7f8f39ddd640 1 --2- 172.21.15.98:0/4058990118 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8f2006b680 0x7f8f2006db40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8f3408ca10 tx=0x7f8f2c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:25.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.483+0000 7f8f327fc640 1 -- 172.21.15.98:0/4058990118 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8f24058640 con 0x7f8f34159540 2024-05-09T10:51:25.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.653+0000 7f8f3c041640 1 -- 172.21.15.98:0/4058990118 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8f3406c530 con 0x7f8f34159540 2024-05-09T10:51:25.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.653+0000 7f8f327fc640 1 -- 172.21.15.98:0/4058990118 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v31) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8f2405c6e0 con 0x7f8f34159540 2024-05-09T10:51:25.646 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2024-05-09T10:51:25.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.655+0000 7f8f3c041640 1 -- 172.21.15.98:0/4058990118 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8f2006b680 msgr2=0x7f8f2006db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:25.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.655+0000 7f8f3c041640 1 --2- 172.21.15.98:0/4058990118 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8f2006b680 0x7f8f2006db40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8f3408ca10 tx=0x7f8f2c008040 comp rx=0 tx=0).stop 2024-05-09T10:51:25.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.655+0000 7f8f3c041640 1 -- 172.21.15.98:0/4058990118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f34159540 msgr2=0x7f8f34160280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:25.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.655+0000 7f8f3c041640 1 --2- 172.21.15.98:0/4058990118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f34159540 0x7f8f34160280 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8f3408cec0 tx=0x7f8f2400eb90 comp rx=0 tx=0).stop 2024-05-09T10:51:25.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.655+0000 7f8f3c041640 1 -- 172.21.15.98:0/4058990118 shutdown_connections 2024-05-09T10:51:25.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.655+0000 7f8f3c041640 1 --2- 172.21.15.98:0/4058990118 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8f2006b680 0x7f8f2006db40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.655+0000 7f8f3c041640 1 --2- 172.21.15.98:0/4058990118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f34159540 0x7f8f34160280 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.655+0000 7f8f3c041640 1 --2- 172.21.15.98:0/4058990118 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8f340d0aa0 0x7f8f340c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.655+0000 7f8f3c041640 1 --2- 172.21.15.98:0/4058990118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f1c004800 0x7f8f34157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.655+0000 7f8f3c041640 1 -- 172.21.15.98:0/4058990118 >> 172.21.15.98:0/4058990118 conn(0x7f8f340d3b30 msgr2=0x7f8f340b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:25.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.655+0000 7f8f3c041640 1 -- 172.21.15.98:0/4058990118 shutdown_connections 2024-05-09T10:51:25.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.655+0000 7f8f3c041640 1 -- 172.21.15.98:0/4058990118 wait complete. 2024-05-09T10:51:25.659 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T10:51:25.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.769+0000 7f262db29640 1 Processor -- start 2024-05-09T10:51:25.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.769+0000 7f262db29640 1 -- start start 2024-05-09T10:51:25.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.770+0000 7f262db29640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26280720d0 0x7f26280af720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:25.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.770+0000 7f262db29640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f262806abe0 0x7f262806aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:25.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.770+0000 7f262db29640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2628072690 con 0x7f262815db90 2024-05-09T10:51:25.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.770+0000 7f262db29640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26280afc60 con 0x7f262806abe0 2024-05-09T10:51:25.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.770+0000 7f262db29640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26280b5d70 con 0x7f26280720d0 2024-05-09T10:51:25.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.770+0000 7f2627fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f262806abe0 0x7f262806aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:25.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.770+0000 7f2627fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f262806abe0 0x7f262806aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60092/0 (socket says 172.21.15.98:60092) 2024-05-09T10:51:25.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.770+0000 7f2627fff640 1 -- 172.21.15.98:0/3231752105 learned_addr learned my addr 172.21.15.98:0/3231752105 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:25.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.770+0000 7f2626ffd640 1 --2- 172.21.15.98:0/3231752105 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26280720d0 0x7f26280af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:25.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.770+0000 7f2627fff640 1 -- 172.21.15.98:0/3231752105 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26280720d0 msgr2=0x7f26280af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:25.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.770+0000 7f2627fff640 1 --2- 172.21.15.98:0/3231752105 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26280720d0 0x7f26280af720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.770+0000 7f2627fff640 1 -- 172.21.15.98:0/3231752105 >> v1:172.21.15.105:6789/0 conn(0x7f262815db90 legacy=0x7f262806d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:25.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.770+0000 7f2627fff640 1 -- 172.21.15.98:0/3231752105 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262806b530 con 0x7f262806abe0 2024-05-09T10:51:25.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.771+0000 7f2627fff640 1 --2- 172.21.15.98:0/3231752105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f262806abe0 0x7f262806aff0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2620009870 tx=0x7f262002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=968604e7174205b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:25.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.771+0000 7f26267fc640 1 -- 172.21.15.98:0/3231752105 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2620037070 con 0x7f262806abe0 2024-05-09T10:51:25.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.771+0000 7f26267fc640 1 -- 172.21.15.98:0/3231752105 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2620032440 con 0x7f262806abe0 2024-05-09T10:51:25.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.771+0000 7f26267fc640 1 -- 172.21.15.98:0/3231752105 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2620032810 con 0x7f262806abe0 2024-05-09T10:51:25.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.772+0000 7f262db29640 1 -- 172.21.15.98:0/3231752105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f262806abe0 msgr2=0x7f262806aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:25.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.772+0000 7f262db29640 1 --2- 172.21.15.98:0/3231752105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f262806abe0 0x7f262806aff0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2620009870 tx=0x7f262002a180 comp rx=0 tx=0).stop 2024-05-09T10:51:25.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.772+0000 7f262db29640 1 -- 172.21.15.98:0/3231752105 shutdown_connections 2024-05-09T10:51:25.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.772+0000 7f262db29640 1 --2- 172.21.15.98:0/3231752105 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f26280720d0 0x7f26280af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.772+0000 7f262db29640 1 --2- 172.21.15.98:0/3231752105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f262806abe0 0x7f262806aff0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.772+0000 7f262db29640 1 -- 172.21.15.98:0/3231752105 >> 172.21.15.98:0/3231752105 conn(0x7f2628066fd0 msgr2=0x7f262806bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:25.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.772+0000 7f262db29640 1 -- 172.21.15.98:0/3231752105 shutdown_connections 2024-05-09T10:51:25.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.772+0000 7f262db29640 1 -- 172.21.15.98:0/3231752105 wait complete. 2024-05-09T10:51:25.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.773+0000 7f262db29640 1 Processor -- start 2024-05-09T10:51:25.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.773+0000 7f262db29640 1 -- start start 2024-05-09T10:51:25.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.774+0000 7f262db29640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f262806abe0 0x7f262814b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:25.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.774+0000 7f262db29640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26280720d0 0x7f26280adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:25.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.774+0000 7f262db29640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f262815db90 0x7f26280adff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:25.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.774+0000 7f262db29640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26280d0670 con 0x7f262806abe0 2024-05-09T10:51:25.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.774+0000 7f262db29640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26280b5a90 con 0x7f26280720d0 2024-05-09T10:51:25.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.774+0000 7f262db29640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f262815df70 con 0x7f262815db90 2024-05-09T10:51:25.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.774+0000 7f26277fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f262806abe0 0x7f262814b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:25.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.774+0000 7f26277fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f262806abe0 0x7f262814b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60108/0 (socket says 172.21.15.98:60108) 2024-05-09T10:51:25.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.774+0000 7f26277fe640 1 -- 172.21.15.98:0/2617615106 learned_addr learned my addr 172.21.15.98:0/2617615106 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:25.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.774+0000 7f2627fff640 1 --2- 172.21.15.98:0/2617615106 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f262815db90 0x7f26280adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:25.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.774+0000 7f2626ffd640 1 --2- 172.21.15.98:0/2617615106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26280720d0 0x7f26280adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:25.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.774+0000 7f26277fe640 1 -- 172.21.15.98:0/2617615106 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f262815db90 msgr2=0x7f26280adff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:25.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.774+0000 7f26277fe640 1 --2- 172.21.15.98:0/2617615106 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f262815db90 0x7f26280adff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.774+0000 7f26277fe640 1 -- 172.21.15.98:0/2617615106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26280720d0 msgr2=0x7f26280adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:25.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.774+0000 7f26277fe640 1 --2- 172.21.15.98:0/2617615106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26280720d0 0x7f26280adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.774+0000 7f26277fe640 1 -- 172.21.15.98:0/2617615106 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2628155d40 con 0x7f262806abe0 2024-05-09T10:51:25.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.774+0000 7f2626ffd640 1 --2- 172.21.15.98:0/2617615106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26280720d0 0x7f26280adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:25.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.774+0000 7f26277fe640 1 --2- 172.21.15.98:0/2617615106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f262806abe0 0x7f262814b3e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f261800ba50 tx=0x7f261800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:25.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.774+0000 7f260ffff640 1 -- 172.21.15.98:0/2617615106 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618019070 con 0x7f262806abe0 2024-05-09T10:51:25.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.775+0000 7f260ffff640 1 -- 172.21.15.98:0/2617615106 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261800cc20 con 0x7f262806abe0 2024-05-09T10:51:25.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.775+0000 7f262db29640 1 -- 172.21.15.98:0/2617615106 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2628156020 con 0x7f262806abe0 2024-05-09T10:51:25.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.775+0000 7f260ffff640 1 -- 172.21.15.98:0/2617615106 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618014680 con 0x7f262806abe0 2024-05-09T10:51:25.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.775+0000 7f262db29640 1 -- 172.21.15.98:0/2617615106 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26281565e0 con 0x7f262806abe0 2024-05-09T10:51:25.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.775+0000 7f262db29640 1 -- 172.21.15.98:0/2617615106 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2628072dd0 con 0x7f262806abe0 2024-05-09T10:51:25.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.776+0000 7f260ffff640 1 -- 172.21.15.98:0/2617615106 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f261801c070 con 0x7f262806abe0 2024-05-09T10:51:25.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.776+0000 7f260ffff640 1 --2- 172.21.15.98:0/2617615106 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f261406b680 0x7f261406db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:25.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.777+0000 7f260ffff640 1 -- 172.21.15.98:0/2617615106 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f261808a180 con 0x7f262806abe0 2024-05-09T10:51:25.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.777+0000 7f2626ffd640 1 --2- 172.21.15.98:0/2617615106 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f261406b680 0x7f261406db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:25.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.777+0000 7f2626ffd640 1 --2- 172.21.15.98:0/2617615106 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f261406b680 0x7f261406db40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f26280b1ba0 tx=0x7f261c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:25.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.780+0000 7f260ffff640 1 -- 172.21.15.98:0/2617615106 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2618058760 con 0x7f262806abe0 2024-05-09T10:51:25.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.953+0000 7f262db29640 1 -- 172.21.15.98:0/2617615106 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f262804f000 con 0x7f262806abe0 2024-05-09T10:51:25.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.954+0000 7f260ffff640 1 -- 172.21.15.98:0/2617615106 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v31) v1 ==== 100+0+11 (secure 0 0 0) 0x7f261805c800 con 0x7f262806abe0 2024-05-09T10:51:25.947 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2024-05-09T10:51:25.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.955+0000 7f262db29640 1 -- 172.21.15.98:0/2617615106 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f261406b680 msgr2=0x7f261406db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:25.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.955+0000 7f262db29640 1 --2- 172.21.15.98:0/2617615106 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f261406b680 0x7f261406db40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f26280b1ba0 tx=0x7f261c008040 comp rx=0 tx=0).stop 2024-05-09T10:51:25.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.956+0000 7f262db29640 1 -- 172.21.15.98:0/2617615106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f262806abe0 msgr2=0x7f262814b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:25.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.956+0000 7f262db29640 1 --2- 172.21.15.98:0/2617615106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f262806abe0 0x7f262814b3e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f261800ba50 tx=0x7f261800bf10 comp rx=0 tx=0).stop 2024-05-09T10:51:25.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.956+0000 7f262db29640 1 -- 172.21.15.98:0/2617615106 shutdown_connections 2024-05-09T10:51:25.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.956+0000 7f262db29640 1 --2- 172.21.15.98:0/2617615106 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f261406b680 0x7f261406db40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.956+0000 7f262db29640 1 --2- 172.21.15.98:0/2617615106 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f262815db90 0x7f26280adff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.956+0000 7f262db29640 1 --2- 172.21.15.98:0/2617615106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26280720d0 0x7f26280adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.956+0000 7f262db29640 1 --2- 172.21.15.98:0/2617615106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f262806abe0 0x7f262814b3e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:25.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.956+0000 7f262db29640 1 -- 172.21.15.98:0/2617615106 >> 172.21.15.98:0/2617615106 conn(0x7f2628066fd0 msgr2=0x7f26281616e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:25.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.956+0000 7f262db29640 1 -- 172.21.15.98:0/2617615106 shutdown_connections 2024-05-09T10:51:25.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:25.956+0000 7f262db29640 1 -- 172.21.15.98:0/2617615106 wait complete. 2024-05-09T10:51:25.960 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T10:51:26.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.067+0000 7ff8453f9640 1 Processor -- start 2024-05-09T10:51:26.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.067+0000 7ff8453f9640 1 -- start start 2024-05-09T10:51:26.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.068+0000 7ff8453f9640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff84015de40 0x7ff8400d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:26.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.068+0000 7ff8453f9640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff84015e220 con 0x7ff8400cf250 2024-05-09T10:51:26.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.068+0000 7ff8453f9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8400d19e0 con 0x7ff8400d0b80 2024-05-09T10:51:26.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.068+0000 7ff8453f9640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8400d1790 con 0x7ff84015de40 2024-05-09T10:51:26.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.068+0000 7ff837fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff8400cf250 0x7ff8400d0470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46986/0 (socket says 172.21.15.98:46986) 2024-05-09T10:51:26.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.068+0000 7ff837fff640 1 -- 172.21.15.98:0/216340397 learned_addr learned my addr 172.21.15.98:0/216340397 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:26.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.068+0000 7ff83effd640 1 --2- 172.21.15.98:0/216340397 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff84015de40 0x7ff8400d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:26.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.068+0000 7ff83e7fc640 1 -- 172.21.15.98:0/216340397 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 711747321 0 0) 0x7ff84015e220 con 0x7ff8400cf250 2024-05-09T10:51:26.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.068+0000 7ff83e7fc640 1 -- 172.21.15.98:0/216340397 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff82c003520 con 0x7ff8400cf250 2024-05-09T10:51:26.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.068+0000 7ff83e7fc640 1 -- 172.21.15.98:0/216340397 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2508224445 0 0) 0x7ff82c003520 con 0x7ff8400cf250 2024-05-09T10:51:26.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.068+0000 7ff83e7fc640 1 -- 172.21.15.98:0/216340397 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff84015de40 msgr2=0x7ff8400d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:26.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.068+0000 7ff83e7fc640 1 --2- 172.21.15.98:0/216340397 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff84015de40 0x7ff8400d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:26.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.068+0000 7ff83e7fc640 1 -- 172.21.15.98:0/216340397 >> v1:172.21.15.105:6789/0 conn(0x7ff8400d0b80 legacy=0x7ff8400b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:26.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.068+0000 7ff83e7fc640 1 -- 172.21.15.98:0/216340397 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8400d1cc0 con 0x7ff8400cf250 2024-05-09T10:51:26.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.068+0000 7ff83e7fc640 1 -- 172.21.15.98:0/216340397 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7ff828001670 con 0x7ff8400cf250 2024-05-09T10:51:26.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.068+0000 7ff83e7fc640 1 -- 172.21.15.98:0/216340397 >> v1:172.21.15.98:6789/0 conn(0x7ff8400cf250 legacy=0x7ff8400d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:26.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.068+0000 7ff83e7fc640 1 --2- 172.21.15.98:0/216340397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff82c0039e0 0x7ff82c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:26.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.068+0000 7ff83e7fc640 1 --2- 172.21.15.98:0/216340397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff82c004390 0x7ff82c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:26.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.068+0000 7ff83e7fc640 1 --2- 172.21.15.98:0/216340397 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff84015de40 0x7ff82c0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:26.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.068+0000 7ff83e7fc640 1 -- 172.21.15.98:0/216340397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8400d1790 con 0x7ff82c0039e0 2024-05-09T10:51:26.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.068+0000 7ff83e7fc640 1 -- 172.21.15.98:0/216340397 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c00b9c0 con 0x7ff82c004390 2024-05-09T10:51:26.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.068+0000 7ff83e7fc640 1 -- 172.21.15.98:0/216340397 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c00bbc0 con 0x7ff84015de40 2024-05-09T10:51:26.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.069+0000 7ff83effd640 1 --2- 172.21.15.98:0/216340397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff82c0039e0 0x7ff82c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:26.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.069+0000 7ff837fff640 1 --2- 172.21.15.98:0/216340397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff82c004390 0x7ff82c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:26.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.069+0000 7ff83f7fe640 1 --2- 172.21.15.98:0/216340397 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff84015de40 0x7ff82c0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:26.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.069+0000 7ff83effd640 1 -- 172.21.15.98:0/216340397 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff84015de40 msgr2=0x7ff82c0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:26.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.069+0000 7ff83effd640 1 --2- 172.21.15.98:0/216340397 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff84015de40 0x7ff82c0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:26.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.069+0000 7ff83effd640 1 -- 172.21.15.98:0/216340397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff82c004390 msgr2=0x7ff82c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:26.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.069+0000 7ff83effd640 1 --2- 172.21.15.98:0/216340397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff82c004390 0x7ff82c004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:26.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.069+0000 7ff83effd640 1 -- 172.21.15.98:0/216340397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c003520 con 0x7ff82c0039e0 2024-05-09T10:51:26.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.069+0000 7ff83f7fe640 1 --2- 172.21.15.98:0/216340397 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff84015de40 0x7ff82c0050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:26.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.069+0000 7ff83effd640 1 --2- 172.21.15.98:0/216340397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff82c0039e0 0x7ff82c003dc0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff830009ac0 tx=0x7ff82c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9564a1a6add1940f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:26.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.070+0000 7ff83e7fc640 1 -- 172.21.15.98:0/216340397 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff830036070 con 0x7ff82c0039e0 2024-05-09T10:51:26.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.070+0000 7ff83e7fc640 1 -- 172.21.15.98:0/216340397 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff830031400 con 0x7ff82c0039e0 2024-05-09T10:51:26.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.070+0000 7ff83e7fc640 1 -- 172.21.15.98:0/216340397 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff830009690 con 0x7ff82c0039e0 2024-05-09T10:51:26.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.070+0000 7ff8453f9640 1 -- 172.21.15.98:0/216340397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff82c0039e0 msgr2=0x7ff82c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:26.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.070+0000 7ff8453f9640 1 --2- 172.21.15.98:0/216340397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff82c0039e0 0x7ff82c003dc0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff830009ac0 tx=0x7ff82c005830 comp rx=0 tx=0).stop 2024-05-09T10:51:26.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.070+0000 7ff83f7fe640 1 -- 172.21.15.98:0/216340397 reap_dead start 2024-05-09T10:51:26.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.070+0000 7ff8453f9640 1 -- 172.21.15.98:0/216340397 shutdown_connections 2024-05-09T10:51:26.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.070+0000 7ff8453f9640 1 -- 172.21.15.98:0/216340397 >> 172.21.15.98:0/216340397 conn(0x7ff84009ce90 msgr2=0x7ff8400d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:26.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.070+0000 7ff8453f9640 1 -- 172.21.15.98:0/216340397 shutdown_connections 2024-05-09T10:51:26.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.070+0000 7ff8453f9640 1 -- 172.21.15.98:0/216340397 wait complete. 2024-05-09T10:51:26.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.071+0000 7ff8453f9640 1 Processor -- start 2024-05-09T10:51:26.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.071+0000 7ff8453f9640 1 -- start start 2024-05-09T10:51:26.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.072+0000 7ff8453f9640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff82c0039e0 0x7ff840155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:26.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.072+0000 7ff8453f9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8401e93d0 0x7ff8401e9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:26.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.072+0000 7ff8453f9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8401e9d90 0x7ff8401f7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:26.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.072+0000 7ff8453f9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8400d4680 con 0x7ff8401e93d0 2024-05-09T10:51:26.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.072+0000 7ff8453f9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8400d2280 con 0x7ff8401e9d90 2024-05-09T10:51:26.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.072+0000 7ff8453f9640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8400d4800 con 0x7ff82c0039e0 2024-05-09T10:51:26.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.072+0000 7ff837fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8401e93d0 0x7ff8401e9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:26.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.072+0000 7ff837fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8401e93d0 0x7ff8401e9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60112/0 (socket says 172.21.15.98:60112) 2024-05-09T10:51:26.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.072+0000 7ff837fff640 1 -- 172.21.15.98:0/1767235556 learned_addr learned my addr 172.21.15.98:0/1767235556 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:26.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.073+0000 7ff837fff640 1 -- 172.21.15.98:0/1767235556 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff82c0039e0 msgr2=0x7ff840155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:26.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.073+0000 7ff837fff640 1 --2- 172.21.15.98:0/1767235556 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff82c0039e0 0x7ff840155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:26.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.073+0000 7ff837fff640 1 -- 172.21.15.98:0/1767235556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8401e9d90 msgr2=0x7ff8401f7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:26.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.073+0000 7ff837fff640 1 --2- 172.21.15.98:0/1767235556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8401e9d90 0x7ff8401f7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:26.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.073+0000 7ff837fff640 1 -- 172.21.15.98:0/1767235556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8400c7a70 con 0x7ff8401e93d0 2024-05-09T10:51:26.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.073+0000 7ff837fff640 1 --2- 172.21.15.98:0/1767235556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8401e93d0 0x7ff8401e9850 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff82800e8c0 tx=0x7ff82800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:26.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.073+0000 7ff8377fe640 1 -- 172.21.15.98:0/1767235556 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82800c870 con 0x7ff8401e93d0 2024-05-09T10:51:26.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.073+0000 7ff8377fe640 1 -- 172.21.15.98:0/1767235556 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff828018070 con 0x7ff8401e93d0 2024-05-09T10:51:26.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.073+0000 7ff8377fe640 1 -- 172.21.15.98:0/1767235556 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82800cc70 con 0x7ff8401e93d0 2024-05-09T10:51:26.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.073+0000 7ff8453f9640 1 -- 172.21.15.98:0/1767235556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8400cd600 con 0x7ff8401e93d0 2024-05-09T10:51:26.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.073+0000 7ff8453f9640 1 -- 172.21.15.98:0/1767235556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8400c7d50 con 0x7ff8401e93d0 2024-05-09T10:51:26.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.074+0000 7ff8453f9640 1 -- 172.21.15.98:0/1767235556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff840069cb0 con 0x7ff8401e93d0 2024-05-09T10:51:26.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.074+0000 7ff8377fe640 1 -- 172.21.15.98:0/1767235556 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7ff828013420 con 0x7ff8401e93d0 2024-05-09T10:51:26.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.075+0000 7ff8377fe640 1 --2- 172.21.15.98:0/1767235556 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff82006b6b0 0x7ff82006db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:26.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.075+0000 7ff8377fe640 1 -- 172.21.15.98:0/1767235556 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff82808a0e0 con 0x7ff8401e93d0 2024-05-09T10:51:26.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.075+0000 7ff83effd640 1 --2- 172.21.15.98:0/1767235556 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff82006b6b0 0x7ff82006db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:26.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.075+0000 7ff83effd640 1 --2- 172.21.15.98:0/1767235556 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff82006b6b0 0x7ff82006db70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff830009ac0 tx=0x7ff83002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:26.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.079+0000 7ff8377fe640 1 -- 172.21.15.98:0/1767235556 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff828058d70 con 0x7ff8401e93d0 2024-05-09T10:51:26.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.262+0000 7ff8453f9640 1 -- 172.21.15.98:0/1767235556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff8400cb840 con 0x7ff8401e93d0 2024-05-09T10:51:26.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.262+0000 7ff8377fe640 1 -- 172.21.15.98:0/1767235556 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v31) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff828058670 con 0x7ff8401e93d0 2024-05-09T10:51:26.256 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-05-09T10:51:26.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.264+0000 7ff8453f9640 1 -- 172.21.15.98:0/1767235556 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff82006b6b0 msgr2=0x7ff82006db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:26.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.264+0000 7ff8453f9640 1 --2- 172.21.15.98:0/1767235556 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff82006b6b0 0x7ff82006db70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff830009ac0 tx=0x7ff83002e000 comp rx=0 tx=0).stop 2024-05-09T10:51:26.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.264+0000 7ff8453f9640 1 -- 172.21.15.98:0/1767235556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8401e93d0 msgr2=0x7ff8401e9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:26.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.264+0000 7ff8453f9640 1 --2- 172.21.15.98:0/1767235556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8401e93d0 0x7ff8401e9850 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff82800e8c0 tx=0x7ff82800ed80 comp rx=0 tx=0).stop 2024-05-09T10:51:26.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.264+0000 7ff8453f9640 1 -- 172.21.15.98:0/1767235556 shutdown_connections 2024-05-09T10:51:26.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.264+0000 7ff8453f9640 1 --2- 172.21.15.98:0/1767235556 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7ff82006b6b0 0x7ff82006db70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:26.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.264+0000 7ff8453f9640 1 --2- 172.21.15.98:0/1767235556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8401e9d90 0x7ff8401f7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:26.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.264+0000 7ff8453f9640 1 --2- 172.21.15.98:0/1767235556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff8401e93d0 0x7ff8401e9850 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:26.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.264+0000 7ff8453f9640 1 --2- 172.21.15.98:0/1767235556 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff82c0039e0 0x7ff840155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:26.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.264+0000 7ff8453f9640 1 -- 172.21.15.98:0/1767235556 >> 172.21.15.98:0/1767235556 conn(0x7ff84009ce90 msgr2=0x7ff84015bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:26.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.264+0000 7ff8453f9640 1 -- 172.21.15.98:0/1767235556 shutdown_connections 2024-05-09T10:51:26.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.264+0000 7ff8453f9640 1 -- 172.21.15.98:0/1767235556 wait complete. 2024-05-09T10:51:26.268 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-09T10:51:26.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:26.269 DEBUG:teuthology.orchestra.run.smithi105:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-09T10:51:26.303 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2024-05-09T10:51:26.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:26.303 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-09T10:51:26.362 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:51:26.362 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:51:26.378 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056333,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18772 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T10:51:26.378 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:51:26.378 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:51:26.420 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056332,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20554 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T10:51:26.420 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T10:51:26.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.527+0000 7f277ab09640 1 Processor -- start 2024-05-09T10:51:26.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.527+0000 7f277ab09640 1 -- start start 2024-05-09T10:51:26.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.527+0000 7f277ab09640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f27740d0b80 0x7f27740b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:26.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.527+0000 7f277ab09640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27740d2c30 con 0x7f27740d27a0 2024-05-09T10:51:26.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.527+0000 7f277ab09640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f277415e220 con 0x7f277415de40 2024-05-09T10:51:26.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.527+0000 7f277ab09640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27740d1790 con 0x7f27740d0b80 2024-05-09T10:51:26.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f2773fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f27740d27a0 0x7f27740d0470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46992/0 (socket says 172.21.15.98:46992) 2024-05-09T10:51:26.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f2773fff640 1 -- 172.21.15.98:0/3343055015 learned_addr learned my addr 172.21.15.98:0/3343055015 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:26.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f27737fe640 1 -- 172.21.15.98:0/3343055015 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4199112769 0 0) 0x7f27740d2c30 con 0x7f27740d27a0 2024-05-09T10:51:26.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f27790a6640 1 --2- 172.21.15.98:0/3343055015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f27740d0b80 0x7f27740b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:26.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f27737fe640 1 -- 172.21.15.98:0/3343055015 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2760003520 con 0x7f27740d27a0 2024-05-09T10:51:26.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f27737fe640 1 -- 172.21.15.98:0/3343055015 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 865398184 0 0) 0x7f2760003520 con 0x7f27740d27a0 2024-05-09T10:51:26.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f27737fe640 1 -- 172.21.15.98:0/3343055015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f27740d0b80 msgr2=0x7f27740b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:26.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f27737fe640 1 --2- 172.21.15.98:0/3343055015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f27740d0b80 0x7f27740b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:26.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f27737fe640 1 -- 172.21.15.98:0/3343055015 >> v1:172.21.15.105:6789/0 conn(0x7f277415de40 legacy=0x7f27740cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:26.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f27737fe640 1 -- 172.21.15.98:0/3343055015 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27740d19e0 con 0x7f27740d27a0 2024-05-09T10:51:26.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f27737fe640 1 -- 172.21.15.98:0/3343055015 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f275c001670 con 0x7f27740d27a0 2024-05-09T10:51:26.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f27790a6640 1 --2- 172.21.15.98:0/3343055015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f27740d0b80 0x7f27740b6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:26.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f27737fe640 1 -- 172.21.15.98:0/3343055015 >> v1:172.21.15.98:6789/0 conn(0x7f27740d27a0 legacy=0x7f27740d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:26.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f27737fe640 1 --2- 172.21.15.98:0/3343055015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f27740d0b80 0x7f27600039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:26.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f27737fe640 1 --2- 172.21.15.98:0/3343055015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2760003fd0 0x7f2760004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:26.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f27737fe640 1 --2- 172.21.15.98:0/3343055015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2760004d40 0x7f27600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:26.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f27737fe640 1 -- 172.21.15.98:0/3343055015 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27740d1790 con 0x7f2760003fd0 2024-05-09T10:51:26.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f27737fe640 1 -- 172.21.15.98:0/3343055015 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276000bb20 con 0x7f2760004d40 2024-05-09T10:51:26.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f27737fe640 1 -- 172.21.15.98:0/3343055015 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f276000bd20 con 0x7f27740d0b80 2024-05-09T10:51:26.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f2773fff640 1 --2- 172.21.15.98:0/3343055015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2760003fd0 0x7f2760004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:26.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f27788a5640 1 --2- 172.21.15.98:0/3343055015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f27740d0b80 0x7f27600039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:26.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f27790a6640 1 --2- 172.21.15.98:0/3343055015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2760004d40 0x7f27600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:26.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f2773fff640 1 -- 172.21.15.98:0/3343055015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f27740d0b80 msgr2=0x7f27600039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:26.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f2773fff640 1 --2- 172.21.15.98:0/3343055015 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f27740d0b80 0x7f27600039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:26.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f2773fff640 1 -- 172.21.15.98:0/3343055015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2760004d40 msgr2=0x7f27600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:26.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f2773fff640 1 --2- 172.21.15.98:0/3343055015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2760004d40 0x7f27600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:26.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.528+0000 7f2773fff640 1 -- 172.21.15.98:0/3343055015 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2760003520 con 0x7f2760003fd0 2024-05-09T10:51:26.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.529+0000 7f2773fff640 1 --2- 172.21.15.98:0/3343055015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2760003fd0 0x7f2760004800 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f275c00c210 tx=0x7f2760005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8a7e235c96ff8b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:26.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.529+0000 7f27737fe640 1 -- 172.21.15.98:0/3343055015 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275c007d30 con 0x7f2760003fd0 2024-05-09T10:51:26.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.529+0000 7f27737fe640 1 -- 172.21.15.98:0/3343055015 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275c005af0 con 0x7f2760003fd0 2024-05-09T10:51:26.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.529+0000 7f27737fe640 1 -- 172.21.15.98:0/3343055015 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275c007070 con 0x7f2760003fd0 2024-05-09T10:51:26.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.530+0000 7f277ab09640 1 -- 172.21.15.98:0/3343055015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2760003fd0 msgr2=0x7f2760004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:26.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.530+0000 7f277ab09640 1 --2- 172.21.15.98:0/3343055015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2760003fd0 0x7f2760004800 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f275c00c210 tx=0x7f2760005830 comp rx=0 tx=0).stop 2024-05-09T10:51:26.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.530+0000 7f27790a6640 1 -- 172.21.15.98:0/3343055015 reap_dead start 2024-05-09T10:51:26.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.530+0000 7f277ab09640 1 -- 172.21.15.98:0/3343055015 shutdown_connections 2024-05-09T10:51:26.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.530+0000 7f277ab09640 1 -- 172.21.15.98:0/3343055015 >> 172.21.15.98:0/3343055015 conn(0x7f277409ce90 msgr2=0x7f27740d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:26.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.530+0000 7f277ab09640 1 -- 172.21.15.98:0/3343055015 shutdown_connections 2024-05-09T10:51:26.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.530+0000 7f277ab09640 1 -- 172.21.15.98:0/3343055015 wait complete. 2024-05-09T10:51:26.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.531+0000 7f277ab09640 1 Processor -- start 2024-05-09T10:51:26.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.531+0000 7f277ab09640 1 -- start start 2024-05-09T10:51:26.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.532+0000 7f277ab09640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2760003fd0 0x7f27741560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:26.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.532+0000 7f277ab09640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2774156600 0x7f27741581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:26.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.532+0000 7f277ab09640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2774158710 0x7f2774158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:26.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.532+0000 7f277ab09640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27740d2280 con 0x7f2760003fd0 2024-05-09T10:51:26.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.532+0000 7f277ab09640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27740d1fa0 con 0x7f2774158710 2024-05-09T10:51:26.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.532+0000 7f277ab09640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27740cf960 con 0x7f2774156600 2024-05-09T10:51:26.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.532+0000 7f27788a5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2760003fd0 0x7f27741560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:26.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.532+0000 7f27788a5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2760003fd0 0x7f27741560c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60128/0 (socket says 172.21.15.98:60128) 2024-05-09T10:51:26.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.532+0000 7f27788a5640 1 -- 172.21.15.98:0/3428049699 learned_addr learned my addr 172.21.15.98:0/3428049699 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:26.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.532+0000 7f2773fff640 1 --2- 172.21.15.98:0/3428049699 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2774156600 0x7f27741581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:26.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.532+0000 7f27788a5640 1 -- 172.21.15.98:0/3428049699 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2774156600 msgr2=0x7f27741581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:26.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.532+0000 7f27788a5640 1 --2- 172.21.15.98:0/3428049699 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2774156600 0x7f27741581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:26.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.532+0000 7f27788a5640 1 -- 172.21.15.98:0/3428049699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2774158710 msgr2=0x7f2774158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:26.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.532+0000 7f27790a6640 1 --2- 172.21.15.98:0/3428049699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2774158710 0x7f2774158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:26.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.532+0000 7f27788a5640 1 --2- 172.21.15.98:0/3428049699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2774158710 0x7f2774158b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:26.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.532+0000 7f27788a5640 1 -- 172.21.15.98:0/3428049699 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27741590d0 con 0x7f2760003fd0 2024-05-09T10:51:26.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.532+0000 7f27788a5640 1 --2- 172.21.15.98:0/3428049699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2760003fd0 0x7f27741560c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f276400fb90 tx=0x7f276400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:26.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.533+0000 7f27717fa640 1 -- 172.21.15.98:0/3428049699 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276400fe90 con 0x7f2760003fd0 2024-05-09T10:51:26.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.533+0000 7f27717fa640 1 -- 172.21.15.98:0/3428049699 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2764018070 con 0x7f2760003fd0 2024-05-09T10:51:26.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.533+0000 7f277ab09640 1 -- 172.21.15.98:0/3428049699 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27741524a0 con 0x7f2760003fd0 2024-05-09T10:51:26.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.533+0000 7f27717fa640 1 -- 172.21.15.98:0/3428049699 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276400de10 con 0x7f2760003fd0 2024-05-09T10:51:26.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.533+0000 7f277ab09640 1 -- 172.21.15.98:0/3428049699 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27740cd240 con 0x7f2760003fd0 2024-05-09T10:51:26.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.534+0000 7f277ab09640 1 -- 172.21.15.98:0/3428049699 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2774069cb0 con 0x7f2760003fd0 2024-05-09T10:51:26.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.534+0000 7f27717fa640 1 -- 172.21.15.98:0/3428049699 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f2764003970 con 0x7f2760003fd0 2024-05-09T10:51:26.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.535+0000 7f27717fa640 1 --2- 172.21.15.98:0/3428049699 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f275406b850 0x7f275406dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:26.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.535+0000 7f27717fa640 1 -- 172.21.15.98:0/3428049699 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f276408a680 con 0x7f2760003fd0 2024-05-09T10:51:26.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.535+0000 7f2773fff640 1 --2- 172.21.15.98:0/3428049699 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f275406b850 0x7f275406dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:26.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.535+0000 7f2773fff640 1 --2- 172.21.15.98:0/3428049699 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f275406b850 0x7f275406dd10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f275c032000 tx=0x7f275c009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:26.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.539+0000 7f27717fa640 1 -- 172.21.15.98:0/3428049699 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f276401a310 con 0x7f2760003fd0 2024-05-09T10:51:26.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.744+0000 7f277ab09640 1 -- 172.21.15.98:0/3428049699 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27740cf230 con 0x7f2760003fd0 2024-05-09T10:51:26.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.745+0000 7f27717fa640 1 -- 172.21.15.98:0/3428049699 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f2764059280 con 0x7f2760003fd0 2024-05-09T10:51:26.738 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:26.738 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2332373437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2332373437},{"type":"v1","addr":"172.21.15.105:6834","nonce":2332373437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4849,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/4092257628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":4092257628},{"type":"v1","addr":"172.21.15.98:6836","nonce":4092257628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4855,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3957569958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3957569958},{"type":"v1","addr":"172.21.15.105:6835","nonce":3957569958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T10:51:06.745057+0000","modified":"2024-05-09T10:51:08.738161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5127":{"gid":5127,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/164638690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":164638690},{"type":"v1","addr":"172.21.15.98:6837","nonce":164638690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-09T10:51:26.739 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 14 2024-05-09T10:51:26.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.747+0000 7f277ab09640 1 -- 172.21.15.98:0/3428049699 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f275406b850 msgr2=0x7f275406dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:26.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.747+0000 7f277ab09640 1 --2- 172.21.15.98:0/3428049699 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f275406b850 0x7f275406dd10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f275c032000 tx=0x7f275c009030 comp rx=0 tx=0).stop 2024-05-09T10:51:26.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.747+0000 7f277ab09640 1 -- 172.21.15.98:0/3428049699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2760003fd0 msgr2=0x7f27741560c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:26.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.747+0000 7f277ab09640 1 --2- 172.21.15.98:0/3428049699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2760003fd0 0x7f27741560c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f276400fb90 tx=0x7f276400d520 comp rx=0 tx=0).stop 2024-05-09T10:51:26.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.747+0000 7f277ab09640 1 -- 172.21.15.98:0/3428049699 shutdown_connections 2024-05-09T10:51:26.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.747+0000 7f277ab09640 1 --2- 172.21.15.98:0/3428049699 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f275406b850 0x7f275406dd10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:26.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.747+0000 7f277ab09640 1 --2- 172.21.15.98:0/3428049699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2774158710 0x7f2774158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:26.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.747+0000 7f277ab09640 1 --2- 172.21.15.98:0/3428049699 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2774156600 0x7f27741581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:26.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.747+0000 7f277ab09640 1 --2- 172.21.15.98:0/3428049699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2760003fd0 0x7f27741560c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:26.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.747+0000 7f277ab09640 1 -- 172.21.15.98:0/3428049699 >> 172.21.15.98:0/3428049699 conn(0x7f277409ce90 msgr2=0x7f277415d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:26.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.747+0000 7f277ab09640 1 -- 172.21.15.98:0/3428049699 shutdown_connections 2024-05-09T10:51:26.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.747+0000 7f277ab09640 1 -- 172.21.15.98:0/3428049699 wait complete. 2024-05-09T10:51:26.752 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:51:26.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.858+0000 7f1f4ff42640 1 Processor -- start 2024-05-09T10:51:26.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.858+0000 7f1f4ff42640 1 -- start start 2024-05-09T10:51:26.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.859+0000 7f1f4ff42640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1f4806be60 0x7f1f48072830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:26.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.859+0000 7f1f4ff42640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1f4806c240 0x7f1f480ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:26.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.859+0000 7f1f4ff42640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f480aee40 con 0x7f1f480ce800 2024-05-09T10:51:26.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.859+0000 7f1f4ff42640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f48071a70 con 0x7f1f4806be60 2024-05-09T10:51:26.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.859+0000 7f1f4ff42640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f48071d50 con 0x7f1f4806c240 2024-05-09T10:51:26.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.859+0000 7f1f4dcde640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1f4806be60 0x7f1f48072830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:26.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.859+0000 7f1f4dcde640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1f4806be60 0x7f1f48072830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60140/0 (socket says 172.21.15.98:60140) 2024-05-09T10:51:26.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.859+0000 7f1f4dcde640 1 -- 172.21.15.98:0/4055605127 learned_addr learned my addr 172.21.15.98:0/4055605127 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:26.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.859+0000 7f1f4e4df640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f1f480ce800 0x7f1f480ca670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46996/0 (socket says 172.21.15.98:46996) 2024-05-09T10:51:26.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.859+0000 7f1f4d4dd640 1 --2- 172.21.15.98:0/4055605127 >> v2:172.21.15.105:3300/0 conn(0x7f1f4806c240 0x7f1f480ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:26.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.859+0000 7f1f4dcde640 1 -- 172.21.15.98:0/4055605127 >> v2:172.21.15.105:3300/0 conn(0x7f1f4806c240 msgr2=0x7f1f480ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:26.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.859+0000 7f1f4dcde640 1 --2- 172.21.15.98:0/4055605127 >> v2:172.21.15.105:3300/0 conn(0x7f1f4806c240 0x7f1f480ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:26.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.859+0000 7f1f4dcde640 1 -- 172.21.15.98:0/4055605127 >> v1:172.21.15.98:6789/0 conn(0x7f1f480ce800 legacy=0x7f1f480ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:26.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.859+0000 7f1f4dcde640 1 -- 172.21.15.98:0/4055605127 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f48072030 con 0x7f1f4806be60 2024-05-09T10:51:26.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.860+0000 7f1f4dcde640 1 --2- 172.21.15.98:0/4055605127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f4806be60 0x7f1f48072830 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1f480684e0 tx=0x7f1f4402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c99608210b8a9bce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:26.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.860+0000 7f1f4ccdc640 1 -- 172.21.15.98:0/4055605127 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f44037070 con 0x7f1f4806be60 2024-05-09T10:51:26.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.860+0000 7f1f4ccdc640 1 -- 172.21.15.98:0/4055605127 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4402ae10 con 0x7f1f4806be60 2024-05-09T10:51:26.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.860+0000 7f1f4ccdc640 1 -- 172.21.15.98:0/4055605127 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f44032660 con 0x7f1f4806be60 2024-05-09T10:51:26.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.861+0000 7f1f4ff42640 1 -- 172.21.15.98:0/4055605127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f4806be60 msgr2=0x7f1f48072830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:26.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.861+0000 7f1f4ff42640 1 --2- 172.21.15.98:0/4055605127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f4806be60 0x7f1f48072830 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1f480684e0 tx=0x7f1f4402a180 comp rx=0 tx=0).stop 2024-05-09T10:51:26.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.861+0000 7f1f4ff42640 1 -- 172.21.15.98:0/4055605127 shutdown_connections 2024-05-09T10:51:26.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.861+0000 7f1f4ff42640 1 --2- 172.21.15.98:0/4055605127 >> v2:172.21.15.105:3300/0 conn(0x7f1f4806c240 0x7f1f480ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:26.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.861+0000 7f1f4ff42640 1 --2- 172.21.15.98:0/4055605127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f4806be60 0x7f1f48072830 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:26.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.861+0000 7f1f4ff42640 1 -- 172.21.15.98:0/4055605127 >> 172.21.15.98:0/4055605127 conn(0x7f1f480ccaa0 msgr2=0x7f1f480cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:26.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.861+0000 7f1f4ff42640 1 -- 172.21.15.98:0/4055605127 shutdown_connections 2024-05-09T10:51:26.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.861+0000 7f1f4ff42640 1 -- 172.21.15.98:0/4055605127 wait complete. 2024-05-09T10:51:26.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.862+0000 7f1f4ff42640 1 Processor -- start 2024-05-09T10:51:26.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.862+0000 7f1f4ff42640 1 -- start start 2024-05-09T10:51:26.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.863+0000 7f1f4ff42640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f4806be60 0x7f1f481d8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:26.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.863+0000 7f1f4ff42640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f4806c240 0x7f1f480b5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:26.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.863+0000 7f1f4ff42640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f480ce800 0x7f1f4809c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:26.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.863+0000 7f1f4ff42640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f480af100 con 0x7f1f4806be60 2024-05-09T10:51:26.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.863+0000 7f1f4ff42640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f480cec90 con 0x7f1f480ce800 2024-05-09T10:51:26.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.863+0000 7f1f4ff42640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f48072d70 con 0x7f1f4806c240 2024-05-09T10:51:26.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.863+0000 7f1f4dcde640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f4806be60 0x7f1f481d8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:26.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.863+0000 7f1f4dcde640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f4806be60 0x7f1f481d8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60150/0 (socket says 172.21.15.98:60150) 2024-05-09T10:51:26.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.863+0000 7f1f4dcde640 1 -- 172.21.15.98:0/748578053 learned_addr learned my addr 172.21.15.98:0/748578053 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:26.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.863+0000 7f1f4dcde640 1 -- 172.21.15.98:0/748578053 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f4806c240 msgr2=0x7f1f480b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:26.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.863+0000 7f1f4dcde640 1 --2- 172.21.15.98:0/748578053 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f4806c240 0x7f1f480b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:26.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.863+0000 7f1f4dcde640 1 -- 172.21.15.98:0/748578053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f480ce800 msgr2=0x7f1f4809c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:26.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.863+0000 7f1f4dcde640 1 --2- 172.21.15.98:0/748578053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f480ce800 0x7f1f4809c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:26.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.863+0000 7f1f4dcde640 1 -- 172.21.15.98:0/748578053 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f481f6f10 con 0x7f1f4806be60 2024-05-09T10:51:26.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.863+0000 7f1f4dcde640 1 --2- 172.21.15.98:0/748578053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f4806be60 0x7f1f481d8b40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1f44000c00 tx=0x7f1f440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:26.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.864+0000 7f1f3e7fc640 1 -- 172.21.15.98:0/748578053 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f44037040 con 0x7f1f4806be60 2024-05-09T10:51:26.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.864+0000 7f1f3e7fc640 1 -- 172.21.15.98:0/748578053 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f44004030 con 0x7f1f4806be60 2024-05-09T10:51:26.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.864+0000 7f1f4ff42640 1 -- 172.21.15.98:0/748578053 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f4809b4c0 con 0x7f1f4806be60 2024-05-09T10:51:26.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.864+0000 7f1f3e7fc640 1 -- 172.21.15.98:0/748578053 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f44004240 con 0x7f1f4806be60 2024-05-09T10:51:26.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.864+0000 7f1f4ff42640 1 -- 172.21.15.98:0/748578053 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f4809ba80 con 0x7f1f4806be60 2024-05-09T10:51:26.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.865+0000 7f1f4ff42640 1 -- 172.21.15.98:0/748578053 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f48069cb0 con 0x7f1f4806be60 2024-05-09T10:51:26.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.865+0000 7f1f3e7fc640 1 -- 172.21.15.98:0/748578053 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f1f4403a040 con 0x7f1f4806be60 2024-05-09T10:51:26.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.866+0000 7f1f3e7fc640 1 --2- 172.21.15.98:0/748578053 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1f2c06b680 0x7f1f2c06db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:26.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.866+0000 7f1f3e7fc640 1 -- 172.21.15.98:0/748578053 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f440aa930 con 0x7f1f4806be60 2024-05-09T10:51:26.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.866+0000 7f1f4d4dd640 1 --2- 172.21.15.98:0/748578053 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1f2c06b680 0x7f1f2c06db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:26.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.866+0000 7f1f4d4dd640 1 --2- 172.21.15.98:0/748578053 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1f2c06b680 0x7f1f2c06db40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1f38006fd0 tx=0x7f1f38008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:26.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:26.870+0000 7f1f3e7fc640 1 -- 172.21.15.98:0/748578053 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1f44042020 con 0x7f1f4806be60 2024-05-09T10:51:27.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.042+0000 7f1f4ff42640 1 -- 172.21.15.98:0/748578053 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f48066b10 con 0x7f1f4806be60 2024-05-09T10:51:27.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.043+0000 7f1f3e7fc640 1 -- 172.21.15.98:0/748578053 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+16669 (secure 0 0 0) 0x7f1f440795c0 con 0x7f1f4806be60 2024-05-09T10:51:27.036 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:27.036 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":31,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:51:08.400895+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:51:05.150410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-09T10:51:05.708391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:51:27.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.045+0000 7f1f4ff42640 1 -- 172.21.15.98:0/748578053 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1f2c06b680 msgr2=0x7f1f2c06db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:27.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.045+0000 7f1f4ff42640 1 --2- 172.21.15.98:0/748578053 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1f2c06b680 0x7f1f2c06db40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1f38006fd0 tx=0x7f1f38008040 comp rx=0 tx=0).stop 2024-05-09T10:51:27.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.045+0000 7f1f4ff42640 1 -- 172.21.15.98:0/748578053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f4806be60 msgr2=0x7f1f481d8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:27.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.045+0000 7f1f4ff42640 1 --2- 172.21.15.98:0/748578053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f4806be60 0x7f1f481d8b40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1f44000c00 tx=0x7f1f440046c0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.045+0000 7f1f4ff42640 1 -- 172.21.15.98:0/748578053 shutdown_connections 2024-05-09T10:51:27.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.045+0000 7f1f4ff42640 1 --2- 172.21.15.98:0/748578053 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f1f2c06b680 0x7f1f2c06db40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.045+0000 7f1f4ff42640 1 --2- 172.21.15.98:0/748578053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f480ce800 0x7f1f4809c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.045+0000 7f1f4ff42640 1 --2- 172.21.15.98:0/748578053 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f4806c240 0x7f1f480b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.045+0000 7f1f4ff42640 1 --2- 172.21.15.98:0/748578053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f4806be60 0x7f1f481d8b40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.045+0000 7f1f4ff42640 1 -- 172.21.15.98:0/748578053 >> 172.21.15.98:0/748578053 conn(0x7f1f480ccaa0 msgr2=0x7f1f4806cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:27.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.045+0000 7f1f4ff42640 1 -- 172.21.15.98:0/748578053 shutdown_connections 2024-05-09T10:51:27.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.045+0000 7f1f4ff42640 1 -- 172.21.15.98:0/748578053 wait complete. 2024-05-09T10:51:27.049 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T10:50:07.396423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T10:51:05.150410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T10:51:05.708391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T10:51:27.049 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T10:51:27.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.159+0000 7f0aff09a640 1 Processor -- start 2024-05-09T10:51:27.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.159+0000 7f0aff09a640 1 -- start start 2024-05-09T10:51:27.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.159+0000 7f0aff09a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0af80b20e0 0x7f0af80ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:27.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.159+0000 7f0aff09a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0af80cc060 0x7f0af80cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:27.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.159+0000 7f0aff09a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af80d1710 con 0x7f0af80b2780 2024-05-09T10:51:27.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.159+0000 7f0aff09a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af80b2e70 con 0x7f0af80cc060 2024-05-09T10:51:27.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.159+0000 7f0aff09a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80d05e0 con 0x7f0af80b20e0 2024-05-09T10:51:27.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.160+0000 7f0afd637640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0af80cc060 0x7f0af80cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:27.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.160+0000 7f0afd637640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0af80cc060 0x7f0af80cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60152/0 (socket says 172.21.15.98:60152) 2024-05-09T10:51:27.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.160+0000 7f0afd637640 1 -- 172.21.15.98:0/328725149 learned_addr learned my addr 172.21.15.98:0/328725149 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:27.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.160+0000 7f0afce36640 1 --2- 172.21.15.98:0/328725149 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0af80b20e0 0x7f0af80ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:27.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.160+0000 7f0afd637640 1 -- 172.21.15.98:0/328725149 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0af80b20e0 msgr2=0x7f0af80ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:27.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.160+0000 7f0afd637640 1 --2- 172.21.15.98:0/328725149 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0af80b20e0 0x7f0af80ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.160+0000 7f0afd637640 1 -- 172.21.15.98:0/328725149 >> v1:172.21.15.105:6789/0 conn(0x7f0af80b2780 legacy=0x7f0af80ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:27.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.160+0000 7f0afd637640 1 -- 172.21.15.98:0/328725149 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af80d08c0 con 0x7f0af80cc060 2024-05-09T10:51:27.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.161+0000 7f0afd637640 1 --2- 172.21.15.98:0/328725149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0af80cc060 0x7f0af80cc4f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009e50 tx=0x7f0af002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14c3ece8433894 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:27.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.161+0000 7f0af77fe640 1 -- 172.21.15.98:0/328725149 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0036070 con 0x7f0af80cc060 2024-05-09T10:51:27.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.161+0000 7f0af77fe640 1 -- 172.21.15.98:0/328725149 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af0031600 con 0x7f0af80cc060 2024-05-09T10:51:27.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.161+0000 7f0af77fe640 1 -- 172.21.15.98:0/328725149 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af00319d0 con 0x7f0af80cc060 2024-05-09T10:51:27.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.161+0000 7f0aff09a640 1 -- 172.21.15.98:0/328725149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0af80cc060 msgr2=0x7f0af80cc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:27.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.161+0000 7f0aff09a640 1 --2- 172.21.15.98:0/328725149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0af80cc060 0x7f0af80cc4f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009e50 tx=0x7f0af002a6c0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.161+0000 7f0aff09a640 1 -- 172.21.15.98:0/328725149 shutdown_connections 2024-05-09T10:51:27.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.161+0000 7f0aff09a640 1 --2- 172.21.15.98:0/328725149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0af80cc060 0x7f0af80cc4f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.161+0000 7f0aff09a640 1 --2- 172.21.15.98:0/328725149 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0af80b20e0 0x7f0af80ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.161+0000 7f0aff09a640 1 -- 172.21.15.98:0/328725149 >> 172.21.15.98:0/328725149 conn(0x7f0af80b34e0 msgr2=0x7f0af80b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:27.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.161+0000 7f0aff09a640 1 -- 172.21.15.98:0/328725149 shutdown_connections 2024-05-09T10:51:27.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.161+0000 7f0aff09a640 1 -- 172.21.15.98:0/328725149 wait complete. 2024-05-09T10:51:27.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.162+0000 7f0aff09a640 1 Processor -- start 2024-05-09T10:51:27.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.163+0000 7f0aff09a640 1 -- start start 2024-05-09T10:51:27.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.163+0000 7f0aff09a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0af80b20e0 0x7f0af80b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:27.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.163+0000 7f0aff09a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0af80b2780 0x7f0af81f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:27.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.163+0000 7f0aff09a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0af80cc060 0x7f0af815aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:27.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.163+0000 7f0aff09a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80cede0 con 0x7f0af80cc060 2024-05-09T10:51:27.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.163+0000 7f0aff09a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80b3150 con 0x7f0af80b20e0 2024-05-09T10:51:27.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.163+0000 7f0aff09a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80cef60 con 0x7f0af80b2780 2024-05-09T10:51:27.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.164+0000 7f0afd637640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0af80cc060 0x7f0af815aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:27.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.164+0000 7f0afd637640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0af80cc060 0x7f0af815aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60162/0 (socket says 172.21.15.98:60162) 2024-05-09T10:51:27.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.164+0000 7f0afd637640 1 -- 172.21.15.98:0/371431637 learned_addr learned my addr 172.21.15.98:0/371431637 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:27.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.164+0000 7f0afce36640 1 --2- 172.21.15.98:0/371431637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0af80b20e0 0x7f0af80b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:27.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.164+0000 7f0af7fff640 1 --2- 172.21.15.98:0/371431637 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0af80b2780 0x7f0af81f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:27.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.164+0000 7f0afd637640 1 -- 172.21.15.98:0/371431637 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0af80b2780 msgr2=0x7f0af81f3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:27.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.164+0000 7f0afd637640 1 --2- 172.21.15.98:0/371431637 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0af80b2780 0x7f0af81f3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.164+0000 7f0afd637640 1 -- 172.21.15.98:0/371431637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0af80b20e0 msgr2=0x7f0af80b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:27.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.164+0000 7f0afd637640 1 --2- 172.21.15.98:0/371431637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0af80b20e0 0x7f0af80b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.164+0000 7f0afd637640 1 -- 172.21.15.98:0/371431637 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af81f3f40 con 0x7f0af80cc060 2024-05-09T10:51:27.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.164+0000 7f0afce36640 1 --2- 172.21.15.98:0/371431637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0af80b20e0 0x7f0af80b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:27.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.164+0000 7f0afd637640 1 --2- 172.21.15.98:0/371431637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0af80cc060 0x7f0af815aeb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009e50 tx=0x7f0af0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:27.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.164+0000 7f0af57fa640 1 -- 172.21.15.98:0/371431637 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0041020 con 0x7f0af80cc060 2024-05-09T10:51:27.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.164+0000 7f0af57fa640 1 -- 172.21.15.98:0/371431637 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af0007780 con 0x7f0af80cc060 2024-05-09T10:51:27.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.164+0000 7f0aff09a640 1 -- 172.21.15.98:0/371431637 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af81f4210 con 0x7f0af80cc060 2024-05-09T10:51:27.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.164+0000 7f0af57fa640 1 -- 172.21.15.98:0/371431637 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0036040 con 0x7f0af80cc060 2024-05-09T10:51:27.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.164+0000 7f0aff09a640 1 -- 172.21.15.98:0/371431637 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af81f7020 con 0x7f0af80cc060 2024-05-09T10:51:27.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.165+0000 7f0aff09a640 1 -- 172.21.15.98:0/371431637 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af809fb40 con 0x7f0af80cc060 2024-05-09T10:51:27.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.166+0000 7f0af57fa640 1 -- 172.21.15.98:0/371431637 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f0af0044020 con 0x7f0af80cc060 2024-05-09T10:51:27.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.166+0000 7f0af57fa640 1 --2- 172.21.15.98:0/371431637 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0adc06b680 0x7f0adc06db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:27.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.166+0000 7f0af57fa640 1 -- 172.21.15.98:0/371431637 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0af00ab2a0 con 0x7f0af80cc060 2024-05-09T10:51:27.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.166+0000 7f0afce36640 1 --2- 172.21.15.98:0/371431637 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0adc06b680 0x7f0adc06db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:27.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.167+0000 7f0afce36640 1 --2- 172.21.15.98:0/371431637 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0adc06b680 0x7f0adc06db40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0aec006fd0 tx=0x7f0aec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:27.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.170+0000 7f0af57fa640 1 -- 172.21.15.98:0/371431637 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0af004b030 con 0x7f0af80cc060 2024-05-09T10:51:27.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.341+0000 7f0aff09a640 1 -- 172.21.15.98:0/371431637 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0af804f000 con 0x7f0af80cc060 2024-05-09T10:51:27.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.341+0000 7f0af57fa640 1 -- 172.21.15.98:0/371431637 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0af0079ee0 con 0x7f0af80cc060 2024-05-09T10:51:27.335 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-05-09T10:51:27.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.343+0000 7f0aff09a640 1 -- 172.21.15.98:0/371431637 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0adc06b680 msgr2=0x7f0adc06db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:27.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.343+0000 7f0aff09a640 1 --2- 172.21.15.98:0/371431637 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0adc06b680 0x7f0adc06db40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0aec006fd0 tx=0x7f0aec008040 comp rx=0 tx=0).stop 2024-05-09T10:51:27.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.343+0000 7f0aff09a640 1 -- 172.21.15.98:0/371431637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0af80cc060 msgr2=0x7f0af815aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:27.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.343+0000 7f0aff09a640 1 --2- 172.21.15.98:0/371431637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0af80cc060 0x7f0af815aeb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009e50 tx=0x7f0af0004a40 comp rx=0 tx=0).stop 2024-05-09T10:51:27.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.343+0000 7f0aff09a640 1 -- 172.21.15.98:0/371431637 shutdown_connections 2024-05-09T10:51:27.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.343+0000 7f0aff09a640 1 --2- 172.21.15.98:0/371431637 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f0adc06b680 0x7f0adc06db40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.343+0000 7f0aff09a640 1 --2- 172.21.15.98:0/371431637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0af80cc060 0x7f0af815aeb0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.343+0000 7f0aff09a640 1 --2- 172.21.15.98:0/371431637 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0af80b2780 0x7f0af81f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.343+0000 7f0aff09a640 1 --2- 172.21.15.98:0/371431637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0af80b20e0 0x7f0af80b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.343+0000 7f0aff09a640 1 -- 172.21.15.98:0/371431637 >> 172.21.15.98:0/371431637 conn(0x7f0af80b34e0 msgr2=0x7f0af80cd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:27.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.344+0000 7f0aff09a640 1 -- 172.21.15.98:0/371431637 shutdown_connections 2024-05-09T10:51:27.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.344+0000 7f0aff09a640 1 -- 172.21.15.98:0/371431637 wait complete. 2024-05-09T10:51:27.348 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T10:51:27.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.457+0000 7f783056a640 1 Processor -- start 2024-05-09T10:51:27.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.458+0000 7f783056a640 1 -- start start 2024-05-09T10:51:27.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.459+0000 7f783056a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f78280d2820 0x7f78280d4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:27.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.459+0000 7f783056a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f78280b2a30 0x7f78280b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:27.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.459+0000 7f783056a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7828161830 con 0x7f7828161450 2024-05-09T10:51:27.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.459+0000 7f783056a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78280b0760 con 0x7f78280d2820 2024-05-09T10:51:27.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.459+0000 7f783056a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78280cf2f0 con 0x7f78280b2a30 2024-05-09T10:51:27.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.459+0000 7f782db05640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f78280d2820 0x7f78280d4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:27.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.459+0000 7f782db05640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f78280d2820 0x7f78280d4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60164/0 (socket says 172.21.15.98:60164) 2024-05-09T10:51:27.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.459+0000 7f782e306640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f7828161450 0x7f78280d2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:47002/0 (socket says 172.21.15.98:47002) 2024-05-09T10:51:27.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.459+0000 7f782db05640 1 -- 172.21.15.98:0/2091045600 learned_addr learned my addr 172.21.15.98:0/2091045600 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:27.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.459+0000 7f782eb07640 1 --2- 172.21.15.98:0/2091045600 >> v2:172.21.15.105:3300/0 conn(0x7f78280b2a30 0x7f78280b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:27.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.459+0000 7f782db05640 1 -- 172.21.15.98:0/2091045600 >> v2:172.21.15.105:3300/0 conn(0x7f78280b2a30 msgr2=0x7f78280b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:27.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.459+0000 7f782db05640 1 --2- 172.21.15.98:0/2091045600 >> v2:172.21.15.105:3300/0 conn(0x7f78280b2a30 0x7f78280b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.459+0000 7f782db05640 1 -- 172.21.15.98:0/2091045600 >> v1:172.21.15.98:6789/0 conn(0x7f7828161450 legacy=0x7f78280d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:27.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.459+0000 7f782db05640 1 -- 172.21.15.98:0/2091045600 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78280cf5d0 con 0x7f78280d2820 2024-05-09T10:51:27.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.459+0000 7f782eb07640 1 --2- 172.21.15.98:0/2091045600 >> v2:172.21.15.105:3300/0 conn(0x7f78280b2a30 0x7f78280b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:27.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.460+0000 7f782db05640 1 --2- 172.21.15.98:0/2091045600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78280d2820 0x7f78280d4650 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f782808cea0 tx=0x7f781802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6bb578ff00b5fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:27.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.460+0000 7f782d304640 1 -- 172.21.15.98:0/2091045600 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7818037070 con 0x7f78280d2820 2024-05-09T10:51:27.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.460+0000 7f782d304640 1 -- 172.21.15.98:0/2091045600 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7818032440 con 0x7f78280d2820 2024-05-09T10:51:27.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.460+0000 7f782d304640 1 -- 172.21.15.98:0/2091045600 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7818032810 con 0x7f78280d2820 2024-05-09T10:51:27.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.460+0000 7f783056a640 1 -- 172.21.15.98:0/2091045600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78280d2820 msgr2=0x7f78280d4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:27.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.460+0000 7f783056a640 1 --2- 172.21.15.98:0/2091045600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78280d2820 0x7f78280d4650 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f782808cea0 tx=0x7f781802a1e0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.460+0000 7f783056a640 1 -- 172.21.15.98:0/2091045600 shutdown_connections 2024-05-09T10:51:27.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.460+0000 7f783056a640 1 --2- 172.21.15.98:0/2091045600 >> v2:172.21.15.105:3300/0 conn(0x7f78280b2a30 0x7f78280b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.460+0000 7f783056a640 1 --2- 172.21.15.98:0/2091045600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78280d2820 0x7f78280d4650 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.461+0000 7f783056a640 1 -- 172.21.15.98:0/2091045600 >> 172.21.15.98:0/2091045600 conn(0x7f78280d3b30 msgr2=0x7f78280d3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:27.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.461+0000 7f783056a640 1 -- 172.21.15.98:0/2091045600 shutdown_connections 2024-05-09T10:51:27.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.461+0000 7f783056a640 1 -- 172.21.15.98:0/2091045600 wait complete. 2024-05-09T10:51:27.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.462+0000 7f783056a640 1 Processor -- start 2024-05-09T10:51:27.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.462+0000 7f783056a640 1 -- start start 2024-05-09T10:51:27.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.462+0000 7f783056a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78280b2a30 0x7f78280b7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:27.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.462+0000 7f783056a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78280d2820 0x7f7828155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:27.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.462+0000 7f783056a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7828161450 0x7f7828156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:27.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.462+0000 7f783056a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78280b1410 con 0x7f78280b2a30 2024-05-09T10:51:27.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.462+0000 7f783056a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78280b09b0 con 0x7f7828161450 2024-05-09T10:51:27.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.462+0000 7f783056a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78280b1590 con 0x7f78280d2820 2024-05-09T10:51:27.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.463+0000 7f782e306640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78280b2a30 0x7f78280b7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:27.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.463+0000 7f782e306640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78280b2a30 0x7f78280b7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60174/0 (socket says 172.21.15.98:60174) 2024-05-09T10:51:27.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.463+0000 7f782e306640 1 -- 172.21.15.98:0/1672620564 learned_addr learned my addr 172.21.15.98:0/1672620564 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:27.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.463+0000 7f782db05640 1 --2- 172.21.15.98:0/1672620564 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78280d2820 0x7f7828155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:27.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.463+0000 7f782eb07640 1 --2- 172.21.15.98:0/1672620564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7828161450 0x7f7828156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:27.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.463+0000 7f782e306640 1 -- 172.21.15.98:0/1672620564 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78280d2820 msgr2=0x7f7828155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:27.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.463+0000 7f782e306640 1 --2- 172.21.15.98:0/1672620564 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78280d2820 0x7f7828155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.463+0000 7f782e306640 1 -- 172.21.15.98:0/1672620564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7828161450 msgr2=0x7f7828156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:27.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.463+0000 7f782e306640 1 --2- 172.21.15.98:0/1672620564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7828161450 0x7f7828156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.463+0000 7f782e306640 1 -- 172.21.15.98:0/1672620564 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78280b91a0 con 0x7f78280b2a30 2024-05-09T10:51:27.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.463+0000 7f782db05640 1 --2- 172.21.15.98:0/1672620564 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78280d2820 0x7f7828155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:27.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.463+0000 7f782eb07640 1 --2- 172.21.15.98:0/1672620564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7828161450 0x7f7828156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:27.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.463+0000 7f782e306640 1 --2- 172.21.15.98:0/1672620564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78280b2a30 0x7f78280b7b00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f782400c3f0 tx=0x7f782400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:27.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.463+0000 7f781effd640 1 -- 172.21.15.98:0/1672620564 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78240074e0 con 0x7f78280b2a30 2024-05-09T10:51:27.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.463+0000 7f783056a640 1 -- 172.21.15.98:0/1672620564 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78280cad30 con 0x7f78280b2a30 2024-05-09T10:51:27.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.463+0000 7f781effd640 1 -- 172.21.15.98:0/1672620564 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7824007680 con 0x7f78280b2a30 2024-05-09T10:51:27.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.463+0000 7f781effd640 1 -- 172.21.15.98:0/1672620564 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7824004a00 con 0x7f78280b2a30 2024-05-09T10:51:27.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.463+0000 7f783056a640 1 -- 172.21.15.98:0/1672620564 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78280cb2b0 con 0x7f78280b2a30 2024-05-09T10:51:27.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.464+0000 7f783056a640 1 -- 172.21.15.98:0/1672620564 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78280a1ae0 con 0x7f78280b2a30 2024-05-09T10:51:27.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.465+0000 7f781effd640 1 -- 172.21.15.98:0/1672620564 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f782401a020 con 0x7f78280b2a30 2024-05-09T10:51:27.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.465+0000 7f781effd640 1 --2- 172.21.15.98:0/1672620564 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f780c06b680 0x7f780c06db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:27.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.465+0000 7f782db05640 1 --2- 172.21.15.98:0/1672620564 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f780c06b680 0x7f780c06db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:27.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.465+0000 7f781effd640 1 -- 172.21.15.98:0/1672620564 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7824089be0 con 0x7f78280b2a30 2024-05-09T10:51:27.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.466+0000 7f782db05640 1 --2- 172.21.15.98:0/1672620564 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f780c06b680 0x7f780c06db40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7818034000 tx=0x7f781802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:27.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.469+0000 7f781effd640 1 -- 172.21.15.98:0/1672620564 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7828161830 con 0x7f78280b2a30 2024-05-09T10:51:27.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.638+0000 7f783056a640 1 -- 172.21.15.98:0/1672620564 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78280b27f0 con 0x7f78280b2a30 2024-05-09T10:51:27.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.638+0000 7f781effd640 1 -- 172.21.15.98:0/1672620564 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v31) v1 ==== 104+0+11 (secure 0 0 0) 0x7f78240587f0 con 0x7f78280b2a30 2024-05-09T10:51:27.632 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2024-05-09T10:51:27.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.640+0000 7f783056a640 1 -- 172.21.15.98:0/1672620564 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f780c06b680 msgr2=0x7f780c06db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:27.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.640+0000 7f783056a640 1 --2- 172.21.15.98:0/1672620564 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f780c06b680 0x7f780c06db40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7818034000 tx=0x7f781802e000 comp rx=0 tx=0).stop 2024-05-09T10:51:27.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.640+0000 7f783056a640 1 -- 172.21.15.98:0/1672620564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78280b2a30 msgr2=0x7f78280b7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:27.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.640+0000 7f783056a640 1 --2- 172.21.15.98:0/1672620564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78280b2a30 0x7f78280b7b00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f782400c3f0 tx=0x7f782400c8b0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.640+0000 7f783056a640 1 -- 172.21.15.98:0/1672620564 shutdown_connections 2024-05-09T10:51:27.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.640+0000 7f783056a640 1 --2- 172.21.15.98:0/1672620564 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f780c06b680 0x7f780c06db40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.640+0000 7f783056a640 1 --2- 172.21.15.98:0/1672620564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7828161450 0x7f7828156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.640+0000 7f783056a640 1 --2- 172.21.15.98:0/1672620564 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78280d2820 0x7f7828155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.640+0000 7f783056a640 1 --2- 172.21.15.98:0/1672620564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78280b2a30 0x7f78280b7b00 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.640+0000 7f783056a640 1 -- 172.21.15.98:0/1672620564 >> 172.21.15.98:0/1672620564 conn(0x7f78280d3b30 msgr2=0x7f78280d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:27.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.640+0000 7f783056a640 1 -- 172.21.15.98:0/1672620564 shutdown_connections 2024-05-09T10:51:27.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.640+0000 7f783056a640 1 -- 172.21.15.98:0/1672620564 wait complete. 2024-05-09T10:51:27.644 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T10:51:27.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.753+0000 7fc1f0963640 1 Processor -- start 2024-05-09T10:51:27.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.753+0000 7fc1f0963640 1 -- start start 2024-05-09T10:51:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.754+0000 7fc1f0963640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc1ec0ad880 0x7fc1ec0b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.754+0000 7fc1f0963640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc1ec0ae370 0x7fc1ec167820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.754+0000 7fc1f0963640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1ec0afa70 con 0x7fc1ec0b5080 2024-05-09T10:51:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.754+0000 7fc1f0963640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1ec15e200 con 0x7fc1ec0ae370 2024-05-09T10:51:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.754+0000 7fc1f0963640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1ec15e4c0 con 0x7fc1ec0ad880 2024-05-09T10:51:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.755+0000 7fc1ead9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc1ec0ae370 0x7fc1ec167820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.755+0000 7fc1ead9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc1ec0ae370 0x7fc1ec167820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60190/0 (socket says 172.21.15.98:60190) 2024-05-09T10:51:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.755+0000 7fc1ead9d640 1 -- 172.21.15.98:0/1205816169 learned_addr learned my addr 172.21.15.98:0/1205816169 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.755+0000 7fc1ea59c640 1 --2- 172.21.15.98:0/1205816169 >> v2:172.21.15.105:3300/0 conn(0x7fc1ec0ad880 0x7fc1ec0b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.755+0000 7fc1ead9d640 1 -- 172.21.15.98:0/1205816169 >> v2:172.21.15.105:3300/0 conn(0x7fc1ec0ad880 msgr2=0x7fc1ec0b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.755+0000 7fc1ead9d640 1 --2- 172.21.15.98:0/1205816169 >> v2:172.21.15.105:3300/0 conn(0x7fc1ec0ad880 0x7fc1ec0b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.755+0000 7fc1ead9d640 1 -- 172.21.15.98:0/1205816169 >> v1:172.21.15.105:6789/0 conn(0x7fc1ec0b5080 legacy=0x7fc1ec0adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:27.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.755+0000 7fc1ead9d640 1 -- 172.21.15.98:0/1205816169 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ec15e7a0 con 0x7fc1ec0ae370 2024-05-09T10:51:27.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.755+0000 7fc1ea59c640 1 --2- 172.21.15.98:0/1205816169 >> v2:172.21.15.105:3300/0 conn(0x7fc1ec0ad880 0x7fc1ec0b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:27.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.756+0000 7fc1ead9d640 1 --2- 172.21.15.98:0/1205816169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1ec0ae370 0x7fc1ec167820 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009e50 tx=0x7fc1e402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a403d23f5dd822c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:27.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.756+0000 7fc1e959a640 1 -- 172.21.15.98:0/1205816169 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4036070 con 0x7fc1ec0ae370 2024-05-09T10:51:27.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.756+0000 7fc1e959a640 1 -- 172.21.15.98:0/1205816169 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e4031600 con 0x7fc1ec0ae370 2024-05-09T10:51:27.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.756+0000 7fc1e959a640 1 -- 172.21.15.98:0/1205816169 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e40319d0 con 0x7fc1ec0ae370 2024-05-09T10:51:27.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.756+0000 7fc1f0963640 1 -- 172.21.15.98:0/1205816169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1ec0ae370 msgr2=0x7fc1ec167820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:27.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.756+0000 7fc1f0963640 1 --2- 172.21.15.98:0/1205816169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1ec0ae370 0x7fc1ec167820 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009e50 tx=0x7fc1e402a5f0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.756+0000 7fc1f0963640 1 -- 172.21.15.98:0/1205816169 shutdown_connections 2024-05-09T10:51:27.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.756+0000 7fc1f0963640 1 --2- 172.21.15.98:0/1205816169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1ec0ae370 0x7fc1ec167820 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.756+0000 7fc1f0963640 1 --2- 172.21.15.98:0/1205816169 >> v2:172.21.15.105:3300/0 conn(0x7fc1ec0ad880 0x7fc1ec0b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.756+0000 7fc1f0963640 1 -- 172.21.15.98:0/1205816169 >> 172.21.15.98:0/1205816169 conn(0x7fc1ec06cbf0 msgr2=0x7fc1ec06d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:27.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.756+0000 7fc1f0963640 1 -- 172.21.15.98:0/1205816169 shutdown_connections 2024-05-09T10:51:27.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.756+0000 7fc1f0963640 1 -- 172.21.15.98:0/1205816169 wait complete. 2024-05-09T10:51:27.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.758+0000 7fc1f0963640 1 Processor -- start 2024-05-09T10:51:27.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.758+0000 7fc1f0963640 1 -- start start 2024-05-09T10:51:27.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.758+0000 7fc1f0963640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1ec0ad880 0x7fc1ec157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:27.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.758+0000 7fc1f0963640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec0ae370 0x7fc1ec14fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:27.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.758+0000 7fc1f0963640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1ec0b5080 0x7fc1ec0cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:27.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.758+0000 7fc1f0963640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec0b3520 con 0x7fc1ec0b5080 2024-05-09T10:51:27.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.758+0000 7fc1f0963640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec15ed60 con 0x7fc1ec0ad880 2024-05-09T10:51:27.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.758+0000 7fc1f0963640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec0b36a0 con 0x7fc1ec0ae370 2024-05-09T10:51:27.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.759+0000 7fc1ead9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1ec0b5080 0x7fc1ec0cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:27.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.759+0000 7fc1ead9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1ec0b5080 0x7fc1ec0cdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60198/0 (socket says 172.21.15.98:60198) 2024-05-09T10:51:27.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.759+0000 7fc1ead9d640 1 -- 172.21.15.98:0/3988699607 learned_addr learned my addr 172.21.15.98:0/3988699607 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:27.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.759+0000 7fc1ea59c640 1 --2- 172.21.15.98:0/3988699607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1ec0ad880 0x7fc1ec157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:27.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.759+0000 7fc1e9d9b640 1 --2- 172.21.15.98:0/3988699607 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec0ae370 0x7fc1ec14fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:27.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.759+0000 7fc1ead9d640 1 -- 172.21.15.98:0/3988699607 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec0ae370 msgr2=0x7fc1ec14fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:27.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.759+0000 7fc1ead9d640 1 --2- 172.21.15.98:0/3988699607 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec0ae370 0x7fc1ec14fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.759+0000 7fc1ead9d640 1 -- 172.21.15.98:0/3988699607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1ec0ad880 msgr2=0x7fc1ec157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:27.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.759+0000 7fc1ead9d640 1 --2- 172.21.15.98:0/3988699607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1ec0ad880 0x7fc1ec157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.759+0000 7fc1ead9d640 1 -- 172.21.15.98:0/3988699607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ec1583a0 con 0x7fc1ec0b5080 2024-05-09T10:51:27.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.759+0000 7fc1ea59c640 1 --2- 172.21.15.98:0/3988699607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1ec0ad880 0x7fc1ec157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:27.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.759+0000 7fc1e9d9b640 1 --2- 172.21.15.98:0/3988699607 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec0ae370 0x7fc1ec14fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:27.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.759+0000 7fc1ead9d640 1 --2- 172.21.15.98:0/3988699607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1ec0b5080 0x7fc1ec0cdd00 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009e50 tx=0x7fc1e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:27.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.759+0000 7fc1daffd640 1 -- 172.21.15.98:0/3988699607 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4036050 con 0x7fc1ec0b5080 2024-05-09T10:51:27.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.759+0000 7fc1daffd640 1 -- 172.21.15.98:0/3988699607 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e4042070 con 0x7fc1ec0b5080 2024-05-09T10:51:27.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.759+0000 7fc1f0963640 1 -- 172.21.15.98:0/3988699607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1ec0cf120 con 0x7fc1ec0b5080 2024-05-09T10:51:27.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.759+0000 7fc1daffd640 1 -- 172.21.15.98:0/3988699607 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4009510 con 0x7fc1ec0b5080 2024-05-09T10:51:27.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.759+0000 7fc1f0963640 1 -- 172.21.15.98:0/3988699607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ec0cf6e0 con 0x7fc1ec0b5080 2024-05-09T10:51:27.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.760+0000 7fc1f0963640 1 -- 172.21.15.98:0/3988699607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1ec06dd40 con 0x7fc1ec0b5080 2024-05-09T10:51:27.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.761+0000 7fc1daffd640 1 -- 172.21.15.98:0/3988699607 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7fc1e4007bb0 con 0x7fc1ec0b5080 2024-05-09T10:51:27.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.761+0000 7fc1daffd640 1 --2- 172.21.15.98:0/3988699607 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc1cc06b680 0x7fc1cc06db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:27.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.761+0000 7fc1daffd640 1 -- 172.21.15.98:0/3988699607 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1e40ab520 con 0x7fc1ec0b5080 2024-05-09T10:51:27.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.761+0000 7fc1ea59c640 1 --2- 172.21.15.98:0/3988699607 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc1cc06b680 0x7fc1cc06db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:27.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.762+0000 7fc1ea59c640 1 --2- 172.21.15.98:0/3988699607 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc1cc06b680 0x7fc1cc06db40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00a990 tx=0x7fc1dc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:27.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.765+0000 7fc1daffd640 1 -- 172.21.15.98:0/3988699607 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc1ec0afa70 con 0x7fc1ec0b5080 2024-05-09T10:51:27.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.936+0000 7fc1f0963640 1 -- 172.21.15.98:0/3988699607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc1ec072de0 con 0x7fc1ec0b5080 2024-05-09T10:51:27.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.937+0000 7fc1daffd640 1 -- 172.21.15.98:0/3988699607 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v31) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc1e4045360 con 0x7fc1ec0b5080 2024-05-09T10:51:27.930 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2024-05-09T10:51:27.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.939+0000 7fc1f0963640 1 -- 172.21.15.98:0/3988699607 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc1cc06b680 msgr2=0x7fc1cc06db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:27.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.939+0000 7fc1f0963640 1 --2- 172.21.15.98:0/3988699607 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc1cc06b680 0x7fc1cc06db40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00a990 tx=0x7fc1dc008040 comp rx=0 tx=0).stop 2024-05-09T10:51:27.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.939+0000 7fc1f0963640 1 -- 172.21.15.98:0/3988699607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1ec0b5080 msgr2=0x7fc1ec0cdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.939+0000 7fc1f0963640 1 --2- 172.21.15.98:0/3988699607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1ec0b5080 0x7fc1ec0cdd00 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009e50 tx=0x7fc1e4004a40 comp rx=0 tx=0).stop 2024-05-09T10:51:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.939+0000 7fc1f0963640 1 -- 172.21.15.98:0/3988699607 shutdown_connections 2024-05-09T10:51:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.939+0000 7fc1f0963640 1 --2- 172.21.15.98:0/3988699607 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fc1cc06b680 0x7fc1cc06db40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.939+0000 7fc1f0963640 1 --2- 172.21.15.98:0/3988699607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1ec0b5080 0x7fc1ec0cdd00 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.939+0000 7fc1f0963640 1 --2- 172.21.15.98:0/3988699607 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc1ec0ae370 0x7fc1ec14fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.939+0000 7fc1f0963640 1 --2- 172.21.15.98:0/3988699607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1ec0ad880 0x7fc1ec157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.939+0000 7fc1f0963640 1 -- 172.21.15.98:0/3988699607 >> 172.21.15.98:0/3988699607 conn(0x7fc1ec06cbf0 msgr2=0x7fc1ec1544c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.939+0000 7fc1f0963640 1 -- 172.21.15.98:0/3988699607 shutdown_connections 2024-05-09T10:51:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:27.939+0000 7fc1f0963640 1 -- 172.21.15.98:0/3988699607 wait complete. 2024-05-09T10:51:27.943 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T10:51:28.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.049+0000 7fa312cf5640 1 Processor -- start 2024-05-09T10:51:28.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.049+0000 7fa312cf5640 1 -- start start 2024-05-09T10:51:28.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.050+0000 7fa312cf5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa30c15dce0 0x7fa30c06bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:28.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.050+0000 7fa312cf5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa30c0d0c70 con 0x7fa30c06cac0 2024-05-09T10:51:28.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.050+0000 7fa312cf5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa30c0d0f50 con 0x7fa30c15e0c0 2024-05-09T10:51:28.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.050+0000 7fa312cf5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c0b4350 con 0x7fa30c15dce0 2024-05-09T10:51:28.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.050+0000 7fa311292640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa30c06cac0 0x7fa30c0b1ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:47012/0 (socket says 172.21.15.98:47012) 2024-05-09T10:51:28.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.050+0000 7fa311292640 1 -- 172.21.15.98:0/788273966 learned_addr learned my addr 172.21.15.98:0/788273966 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:28.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.050+0000 7fa310a91640 1 --2- 172.21.15.98:0/788273966 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa30c15dce0 0x7fa30c06bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:28.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.050+0000 7fa3037fe640 1 -- 172.21.15.98:0/788273966 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1699282296 0 0) 0x7fa30c0d0c70 con 0x7fa30c06cac0 2024-05-09T10:51:28.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.050+0000 7fa3037fe640 1 -- 172.21.15.98:0/788273966 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2f4003520 con 0x7fa30c06cac0 2024-05-09T10:51:28.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.050+0000 7fa310a91640 1 -- 172.21.15.98:0/788273966 >> v1:172.21.15.105:6789/0 conn(0x7fa30c15e0c0 legacy=0x7fa30c06c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:28.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.050+0000 7fa3037fe640 1 -- 172.21.15.98:0/788273966 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1010168285 0 0) 0x7fa30c0d0f50 con 0x7fa30c15e0c0 2024-05-09T10:51:28.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.050+0000 7fa310a91640 1 -- 172.21.15.98:0/788273966 >> v1:172.21.15.98:6789/0 conn(0x7fa30c06cac0 legacy=0x7fa30c0b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:28.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.050+0000 7fa310a91640 1 -- 172.21.15.98:0/788273966 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30c0b0fd0 con 0x7fa30c15dce0 2024-05-09T10:51:28.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.051+0000 7fa310a91640 1 --2- 172.21.15.98:0/788273966 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa30c15dce0 0x7fa30c06bcf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa308009dc0 tx=0x7fa30802a760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8820cffa23eb6371 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:28.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.051+0000 7fa3037fe640 1 -- 172.21.15.98:0/788273966 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa308007740 con 0x7fa30c15dce0 2024-05-09T10:51:28.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.051+0000 7fa3037fe640 1 -- 172.21.15.98:0/788273966 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3080078e0 con 0x7fa30c15dce0 2024-05-09T10:51:28.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.051+0000 7fa3037fe640 1 -- 172.21.15.98:0/788273966 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa308007cb0 con 0x7fa30c15dce0 2024-05-09T10:51:28.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.052+0000 7fa312cf5640 1 -- 172.21.15.98:0/788273966 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa30c15dce0 msgr2=0x7fa30c06bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:28.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.052+0000 7fa312cf5640 1 --2- 172.21.15.98:0/788273966 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa30c15dce0 0x7fa30c06bcf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa308009dc0 tx=0x7fa30802a760 comp rx=0 tx=0).stop 2024-05-09T10:51:28.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.052+0000 7fa312cf5640 1 -- 172.21.15.98:0/788273966 shutdown_connections 2024-05-09T10:51:28.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.052+0000 7fa312cf5640 1 --2- 172.21.15.98:0/788273966 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa30c15dce0 0x7fa30c06bcf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.052+0000 7fa312cf5640 1 -- 172.21.15.98:0/788273966 >> 172.21.15.98:0/788273966 conn(0x7fa30c0b34e0 msgr2=0x7fa30c0b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:28.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.052+0000 7fa312cf5640 1 -- 172.21.15.98:0/788273966 shutdown_connections 2024-05-09T10:51:28.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.052+0000 7fa312cf5640 1 -- 172.21.15.98:0/788273966 wait complete. 2024-05-09T10:51:28.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.053+0000 7fa312cf5640 1 Processor -- start 2024-05-09T10:51:28.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.053+0000 7fa312cf5640 1 -- start start 2024-05-09T10:51:28.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.054+0000 7fa312cf5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa30c06cac0 0x7fa30c0c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:28.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.054+0000 7fa312cf5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa30c15dce0 0x7fa30c1583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:28.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.054+0000 7fa312cf5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa30c15e0c0 0x7fa30c1f34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:28.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.054+0000 7fa312cf5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c0d0700 con 0x7fa30c15dce0 2024-05-09T10:51:28.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.054+0000 7fa312cf5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c0b1590 con 0x7fa30c06cac0 2024-05-09T10:51:28.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.054+0000 7fa312cf5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c0d0880 con 0x7fa30c15e0c0 2024-05-09T10:51:28.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.054+0000 7fa303fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa30c15dce0 0x7fa30c1583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:28.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.054+0000 7fa303fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa30c15dce0 0x7fa30c1583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60204/0 (socket says 172.21.15.98:60204) 2024-05-09T10:51:28.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.054+0000 7fa303fff640 1 -- 172.21.15.98:0/551771808 learned_addr learned my addr 172.21.15.98:0/551771808 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:28.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.054+0000 7fa311292640 1 --2- 172.21.15.98:0/551771808 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa30c15e0c0 0x7fa30c1f34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:28.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.054+0000 7fa310a91640 1 --2- 172.21.15.98:0/551771808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa30c06cac0 0x7fa30c0c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:28.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.054+0000 7fa303fff640 1 -- 172.21.15.98:0/551771808 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa30c15e0c0 msgr2=0x7fa30c1f34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:28.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.054+0000 7fa303fff640 1 --2- 172.21.15.98:0/551771808 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa30c15e0c0 0x7fa30c1f34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.054+0000 7fa303fff640 1 -- 172.21.15.98:0/551771808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa30c06cac0 msgr2=0x7fa30c0c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:28.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.054+0000 7fa303fff640 1 --2- 172.21.15.98:0/551771808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa30c06cac0 0x7fa30c0c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.054+0000 7fa303fff640 1 -- 172.21.15.98:0/551771808 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30c1588f0 con 0x7fa30c15dce0 2024-05-09T10:51:28.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.054+0000 7fa311292640 1 --2- 172.21.15.98:0/551771808 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa30c15e0c0 0x7fa30c1f34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:28.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.054+0000 7fa310a91640 1 --2- 172.21.15.98:0/551771808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa30c06cac0 0x7fa30c0c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:28.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.054+0000 7fa303fff640 1 --2- 172.21.15.98:0/551771808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa30c15dce0 0x7fa30c1583b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00c4e0 tx=0x7fa2fc00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:28.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.055+0000 7fa3017fa640 1 -- 172.21.15.98:0/551771808 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc017070 con 0x7fa30c15dce0 2024-05-09T10:51:28.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.055+0000 7fa3017fa640 1 -- 172.21.15.98:0/551771808 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2fc0035f0 con 0x7fa30c15dce0 2024-05-09T10:51:28.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.055+0000 7fa3017fa640 1 -- 172.21.15.98:0/551771808 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc0039e0 con 0x7fa30c15dce0 2024-05-09T10:51:28.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.055+0000 7fa312cf5640 1 -- 172.21.15.98:0/551771808 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa30c0c6090 con 0x7fa30c15dce0 2024-05-09T10:51:28.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.055+0000 7fa312cf5640 1 -- 172.21.15.98:0/551771808 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa30c0c6650 con 0x7fa30c15dce0 2024-05-09T10:51:28.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.056+0000 7fa312cf5640 1 -- 172.21.15.98:0/551771808 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa30c069cb0 con 0x7fa30c15dce0 2024-05-09T10:51:28.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.056+0000 7fa3017fa640 1 -- 172.21.15.98:0/551771808 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7fa2fc01a070 con 0x7fa30c15dce0 2024-05-09T10:51:28.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.057+0000 7fa3017fa640 1 --2- 172.21.15.98:0/551771808 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa2f806b680 0x7fa2f806db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:28.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.057+0000 7fa3017fa640 1 -- 172.21.15.98:0/551771808 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2fc089890 con 0x7fa30c15dce0 2024-05-09T10:51:28.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.057+0000 7fa310a91640 1 --2- 172.21.15.98:0/551771808 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa2f806b680 0x7fa2f806db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:28.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.057+0000 7fa310a91640 1 --2- 172.21.15.98:0/551771808 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa2f806b680 0x7fa2f806db40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa308034000 tx=0x7fa30802f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:28.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.061+0000 7fa3017fa640 1 -- 172.21.15.98:0/551771808 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa30c1588f0 con 0x7fa30c15dce0 2024-05-09T10:51:28.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.266+0000 7fa312cf5640 1 -- 172.21.15.98:0/551771808 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa30c0d14f0 con 0x7fa30c15dce0 2024-05-09T10:51:28.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.266+0000 7fa3017fa640 1 -- 172.21.15.98:0/551771808 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fa2fc057da0 con 0x7fa30c15dce0 2024-05-09T10:51:28.260 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:28.260 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2332373437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2332373437},{"type":"v1","addr":"172.21.15.105:6834","nonce":2332373437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4849,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/4092257628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":4092257628},{"type":"v1","addr":"172.21.15.98:6836","nonce":4092257628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4855,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3957569958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3957569958},{"type":"v1","addr":"172.21.15.105:6835","nonce":3957569958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T10:51:06.745057+0000","modified":"2024-05-09T10:51:08.738161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5127":{"gid":5127,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/164638690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":164638690},{"type":"v1","addr":"172.21.15.98:6837","nonce":164638690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-09T10:51:28.260 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 14 2024-05-09T10:51:28.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.268+0000 7fa312cf5640 1 -- 172.21.15.98:0/551771808 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa2f806b680 msgr2=0x7fa2f806db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:28.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.268+0000 7fa312cf5640 1 --2- 172.21.15.98:0/551771808 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa2f806b680 0x7fa2f806db40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa308034000 tx=0x7fa30802f000 comp rx=0 tx=0).stop 2024-05-09T10:51:28.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.268+0000 7fa312cf5640 1 -- 172.21.15.98:0/551771808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa30c15dce0 msgr2=0x7fa30c1583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:28.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.268+0000 7fa312cf5640 1 --2- 172.21.15.98:0/551771808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa30c15dce0 0x7fa30c1583b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00c4e0 tx=0x7fa2fc00c9a0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.268+0000 7fa312cf5640 1 -- 172.21.15.98:0/551771808 shutdown_connections 2024-05-09T10:51:28.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.268+0000 7fa312cf5640 1 --2- 172.21.15.98:0/551771808 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fa2f806b680 0x7fa2f806db40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.268+0000 7fa312cf5640 1 --2- 172.21.15.98:0/551771808 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa30c15e0c0 0x7fa30c1f34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.268+0000 7fa312cf5640 1 --2- 172.21.15.98:0/551771808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa30c15dce0 0x7fa30c1583b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.268+0000 7fa312cf5640 1 --2- 172.21.15.98:0/551771808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa30c06cac0 0x7fa30c0c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.268+0000 7fa312cf5640 1 -- 172.21.15.98:0/551771808 >> 172.21.15.98:0/551771808 conn(0x7fa30c0b34e0 msgr2=0x7fa30c15cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:28.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.268+0000 7fa312cf5640 1 -- 172.21.15.98:0/551771808 shutdown_connections 2024-05-09T10:51:28.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.268+0000 7fa312cf5640 1 -- 172.21.15.98:0/551771808 wait complete. 2024-05-09T10:51:28.272 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:51:28.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.378+0000 7f9a747be640 1 Processor -- start 2024-05-09T10:51:28.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.379+0000 7f9a747be640 1 -- start start 2024-05-09T10:51:28.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.379+0000 7f9a747be640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9a6c15f1c0 0x7f9a6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:28.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.379+0000 7f9a747be640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a6c06c300 0x7f9a6c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:28.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.380+0000 7f9a747be640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a6c15fb70 con 0x7f9a6c0cf430 2024-05-09T10:51:28.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.380+0000 7f9a747be640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a6c160930 con 0x7f9a6c15f1c0 2024-05-09T10:51:28.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.380+0000 7f9a747be640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c15fe50 con 0x7f9a6c06c300 2024-05-09T10:51:28.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.380+0000 7f9a7255a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9a6c15f1c0 0x7f9a6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:28.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.380+0000 7f9a71d59640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f9a6c0cf430 0x7f9a6c06bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:47014/0 (socket says 172.21.15.98:47014) 2024-05-09T10:51:28.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.380+0000 7f9a71d59640 1 -- 172.21.15.98:0/2708416068 learned_addr learned my addr 172.21.15.98:0/2708416068 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:28.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.380+0000 7f9a7255a640 1 -- 172.21.15.98:0/2708416068 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a6c06c300 msgr2=0x7f9a6c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:28.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.380+0000 7f9a72d5b640 1 --2- 172.21.15.98:0/2708416068 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a6c06c300 0x7f9a6c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:28.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.380+0000 7f9a71558640 1 -- 172.21.15.98:0/2708416068 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1016820365 0 0) 0x7f9a6c15fb70 con 0x7f9a6c0cf430 2024-05-09T10:51:28.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.380+0000 7f9a7255a640 1 --2- 172.21.15.98:0/2708416068 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a6c06c300 0x7f9a6c06d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.380+0000 7f9a7255a640 1 -- 172.21.15.98:0/2708416068 >> v1:172.21.15.98:6789/0 conn(0x7f9a6c0cf430 legacy=0x7f9a6c06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:28.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.380+0000 7f9a7255a640 1 -- 172.21.15.98:0/2708416068 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6c160130 con 0x7f9a6c15f1c0 2024-05-09T10:51:28.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.380+0000 7f9a72d5b640 1 --2- 172.21.15.98:0/2708416068 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a6c06c300 0x7f9a6c06d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:28.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.381+0000 7f9a7255a640 1 --2- 172.21.15.98:0/2708416068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a6c15f1c0 0x7f9a6c06edb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9a680097a0 tx=0x7f9a6802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605df403ef07fe9a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:28.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.381+0000 7f9a71558640 1 -- 172.21.15.98:0/2708416068 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a68037070 con 0x7f9a6c15f1c0 2024-05-09T10:51:28.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.381+0000 7f9a71558640 1 -- 172.21.15.98:0/2708416068 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a68032440 con 0x7f9a6c15f1c0 2024-05-09T10:51:28.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.381+0000 7f9a71558640 1 -- 172.21.15.98:0/2708416068 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a68032810 con 0x7f9a6c15f1c0 2024-05-09T10:51:28.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.381+0000 7f9a747be640 1 -- 172.21.15.98:0/2708416068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a6c15f1c0 msgr2=0x7f9a6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:28.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.381+0000 7f9a747be640 1 --2- 172.21.15.98:0/2708416068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a6c15f1c0 0x7f9a6c06edb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9a680097a0 tx=0x7f9a6802a180 comp rx=0 tx=0).stop 2024-05-09T10:51:28.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.381+0000 7f9a747be640 1 -- 172.21.15.98:0/2708416068 shutdown_connections 2024-05-09T10:51:28.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.381+0000 7f9a747be640 1 --2- 172.21.15.98:0/2708416068 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a6c06c300 0x7f9a6c06d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.381+0000 7f9a747be640 1 --2- 172.21.15.98:0/2708416068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a6c15f1c0 0x7f9a6c06edb0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.381+0000 7f9a747be640 1 -- 172.21.15.98:0/2708416068 >> 172.21.15.98:0/2708416068 conn(0x7f9a6c0ccaa0 msgr2=0x7f9a6c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:28.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.381+0000 7f9a747be640 1 -- 172.21.15.98:0/2708416068 shutdown_connections 2024-05-09T10:51:28.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.381+0000 7f9a747be640 1 -- 172.21.15.98:0/2708416068 wait complete. 2024-05-09T10:51:28.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.383+0000 7f9a747be640 1 Processor -- start 2024-05-09T10:51:28.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.383+0000 7f9a747be640 1 -- start start 2024-05-09T10:51:28.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.383+0000 7f9a747be640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a6c06c300 0x7f9a6c0cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:28.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.383+0000 7f9a747be640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a6c0cf430 0x7f9a6c0c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:28.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.383+0000 7f9a747be640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a6c15f1c0 0x7f9a6c09aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:28.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.383+0000 7f9a747be640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c15f5a0 con 0x7f9a6c0cf430 2024-05-09T10:51:28.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.383+0000 7f9a747be640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c06f2f0 con 0x7f9a6c15f1c0 2024-05-09T10:51:28.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.383+0000 7f9a747be640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c15f720 con 0x7f9a6c06c300 2024-05-09T10:51:28.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.383+0000 7f9a71d59640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a6c0cf430 0x7f9a6c0c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:28.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.383+0000 7f9a71d59640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a6c0cf430 0x7f9a6c0c8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60234/0 (socket says 172.21.15.98:60234) 2024-05-09T10:51:28.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.383+0000 7f9a71d59640 1 -- 172.21.15.98:0/3977888185 learned_addr learned my addr 172.21.15.98:0/3977888185 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:28.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.384+0000 7f9a7255a640 1 --2- 172.21.15.98:0/3977888185 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a6c06c300 0x7f9a6c0cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:28.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.384+0000 7f9a72d5b640 1 --2- 172.21.15.98:0/3977888185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a6c15f1c0 0x7f9a6c09aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:28.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.384+0000 7f9a71d59640 1 -- 172.21.15.98:0/3977888185 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a6c06c300 msgr2=0x7f9a6c0cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:28.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.384+0000 7f9a71d59640 1 --2- 172.21.15.98:0/3977888185 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a6c06c300 0x7f9a6c0cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.384+0000 7f9a71d59640 1 -- 172.21.15.98:0/3977888185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a6c15f1c0 msgr2=0x7f9a6c09aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:28.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.384+0000 7f9a71d59640 1 --2- 172.21.15.98:0/3977888185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a6c15f1c0 0x7f9a6c09aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.384+0000 7f9a71d59640 1 -- 172.21.15.98:0/3977888185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6c0c93f0 con 0x7f9a6c0cf430 2024-05-09T10:51:28.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.384+0000 7f9a71d59640 1 --2- 172.21.15.98:0/3977888185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a6c0cf430 0x7f9a6c0c8eb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00c4d0 tx=0x7f9a5c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:28.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.384+0000 7f9a62ffd640 1 -- 172.21.15.98:0/3977888185 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c018070 con 0x7f9a6c0cf430 2024-05-09T10:51:28.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.384+0000 7f9a62ffd640 1 -- 172.21.15.98:0/3977888185 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5c007500 con 0x7f9a6c0cf430 2024-05-09T10:51:28.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.384+0000 7f9a62ffd640 1 -- 172.21.15.98:0/3977888185 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c004a20 con 0x7f9a6c0cf430 2024-05-09T10:51:28.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.384+0000 7f9a747be640 1 -- 172.21.15.98:0/3977888185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a6c09af50 con 0x7f9a6c0cf430 2024-05-09T10:51:28.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.384+0000 7f9a747be640 1 -- 172.21.15.98:0/3977888185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a6c0c4ba0 con 0x7f9a6c0cf430 2024-05-09T10:51:28.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.385+0000 7f9a747be640 1 -- 172.21.15.98:0/3977888185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a6c15ed20 con 0x7f9a6c0cf430 2024-05-09T10:51:28.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.386+0000 7f9a62ffd640 1 -- 172.21.15.98:0/3977888185 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f9a5c01b070 con 0x7f9a6c0cf430 2024-05-09T10:51:28.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.386+0000 7f9a62ffd640 1 --2- 172.21.15.98:0/3977888185 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9a5006b680 0x7f9a5006db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:28.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.386+0000 7f9a62ffd640 1 -- 172.21.15.98:0/3977888185 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a5c0897c0 con 0x7f9a6c0cf430 2024-05-09T10:51:28.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.386+0000 7f9a7255a640 1 --2- 172.21.15.98:0/3977888185 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9a5006b680 0x7f9a5006db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:28.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.387+0000 7f9a7255a640 1 --2- 172.21.15.98:0/3977888185 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9a5006b680 0x7f9a5006db40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9a68034000 tx=0x7f9a6802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:28.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.390+0000 7f9a62ffd640 1 -- 172.21.15.98:0/3977888185 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9a6c15ed20 con 0x7f9a6c0cf430 2024-05-09T10:51:28.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.564+0000 7f9a747be640 1 -- 172.21.15.98:0/3977888185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a6c15f9c0 con 0x7f9a6c0cf430 2024-05-09T10:51:28.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.565+0000 7f9a62ffd640 1 -- 172.21.15.98:0/3977888185 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+16669 (secure 0 0 0) 0x7f9a5c0583c0 con 0x7f9a6c0cf430 2024-05-09T10:51:28.559 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:28.559 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":31,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:51:08.400895+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:51:05.150410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-09T10:51:05.708391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:51:28.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.567+0000 7f9a747be640 1 -- 172.21.15.98:0/3977888185 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9a5006b680 msgr2=0x7f9a5006db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:28.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.567+0000 7f9a747be640 1 --2- 172.21.15.98:0/3977888185 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9a5006b680 0x7f9a5006db40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9a68034000 tx=0x7f9a6802e000 comp rx=0 tx=0).stop 2024-05-09T10:51:28.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.567+0000 7f9a747be640 1 -- 172.21.15.98:0/3977888185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a6c0cf430 msgr2=0x7f9a6c0c8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:28.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.567+0000 7f9a747be640 1 --2- 172.21.15.98:0/3977888185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a6c0cf430 0x7f9a6c0c8eb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00c4d0 tx=0x7f9a5c00c990 comp rx=0 tx=0).stop 2024-05-09T10:51:28.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.567+0000 7f9a747be640 1 -- 172.21.15.98:0/3977888185 shutdown_connections 2024-05-09T10:51:28.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.567+0000 7f9a747be640 1 --2- 172.21.15.98:0/3977888185 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9a5006b680 0x7f9a5006db40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.567+0000 7f9a747be640 1 --2- 172.21.15.98:0/3977888185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9a6c15f1c0 0x7f9a6c09aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.567+0000 7f9a747be640 1 --2- 172.21.15.98:0/3977888185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a6c0cf430 0x7f9a6c0c8eb0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.567+0000 7f9a747be640 1 --2- 172.21.15.98:0/3977888185 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9a6c06c300 0x7f9a6c0cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.567+0000 7f9a747be640 1 -- 172.21.15.98:0/3977888185 >> 172.21.15.98:0/3977888185 conn(0x7f9a6c0ccaa0 msgr2=0x7f9a6c09e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:28.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.567+0000 7f9a747be640 1 -- 172.21.15.98:0/3977888185 shutdown_connections 2024-05-09T10:51:28.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.567+0000 7f9a747be640 1 -- 172.21.15.98:0/3977888185 wait complete. 2024-05-09T10:51:28.571 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T10:51:28.571 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T10:51:28.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.678+0000 7f502d370640 1 Processor -- start 2024-05-09T10:51:28.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.678+0000 7f502d370640 1 -- start start 2024-05-09T10:51:28.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.679+0000 7f502d370640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50280d27a0 0x7f50280d0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:28.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.679+0000 7f502d370640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f50280d09b0 0x7f50280b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:28.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.679+0000 7f502d370640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f502815e220 con 0x7f502815de40 2024-05-09T10:51:28.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.679+0000 7f502d370640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50280d1790 con 0x7f50280d09b0 2024-05-09T10:51:28.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.679+0000 7f502d370640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50280d2c00 con 0x7f50280d27a0 2024-05-09T10:51:28.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.679+0000 7f5026ffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f502815de40 0x7f50280cf250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:47020/0 (socket says 172.21.15.98:47020) 2024-05-09T10:51:28.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.679+0000 7f5026ffd640 1 -- 172.21.15.98:0/3975480230 learned_addr learned my addr 172.21.15.98:0/3975480230 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:28.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.679+0000 7f50267fc640 1 --2- 172.21.15.98:0/3975480230 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50280d27a0 0x7f50280d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:28.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.679+0000 7f5025ffb640 1 -- 172.21.15.98:0/3975480230 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688465818 0 0) 0x7f502815e220 con 0x7f502815de40 2024-05-09T10:51:28.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.679+0000 7f50277fe640 1 --2- 172.21.15.98:0/3975480230 >> v2:172.21.15.105:3300/0 conn(0x7f50280d09b0 0x7f50280b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:28.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.679+0000 7f5025ffb640 1 -- 172.21.15.98:0/3975480230 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5010003520 con 0x7f502815de40 2024-05-09T10:51:28.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.680+0000 7f5025ffb640 1 -- 172.21.15.98:0/3975480230 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2616361526 0 0) 0x7f5010003520 con 0x7f502815de40 2024-05-09T10:51:28.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.680+0000 7f5025ffb640 1 -- 172.21.15.98:0/3975480230 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50280d27a0 msgr2=0x7f50280d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:28.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.680+0000 7f5025ffb640 1 --2- 172.21.15.98:0/3975480230 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50280d27a0 0x7f50280d0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.680+0000 7f5025ffb640 1 -- 172.21.15.98:0/3975480230 >> v2:172.21.15.105:3300/0 conn(0x7f50280d09b0 msgr2=0x7f50280b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:28.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.680+0000 7f5025ffb640 1 --2- 172.21.15.98:0/3975480230 >> v2:172.21.15.105:3300/0 conn(0x7f50280d09b0 0x7f50280b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.680+0000 7f5025ffb640 1 -- 172.21.15.98:0/3975480230 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50280d19e0 con 0x7f502815de40 2024-05-09T10:51:28.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.680+0000 7f5025ffb640 1 -- 172.21.15.98:0/3975480230 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f50180019f0 con 0x7f502815de40 2024-05-09T10:51:28.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.680+0000 7f5025ffb640 1 -- 172.21.15.98:0/3975480230 >> v1:172.21.15.98:6789/0 conn(0x7f502815de40 legacy=0x7f50280cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:28.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.680+0000 7f5025ffb640 1 --2- 172.21.15.98:0/3975480230 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50280d27a0 0x7f50100039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:28.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.680+0000 7f5025ffb640 1 --2- 172.21.15.98:0/3975480230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5010003f30 0x7f5010004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:28.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.680+0000 7f5025ffb640 1 --2- 172.21.15.98:0/3975480230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5010004d40 0x7f50100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:28.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.680+0000 7f5025ffb640 1 -- 172.21.15.98:0/3975480230 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50280d1790 con 0x7f5010003f30 2024-05-09T10:51:28.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.680+0000 7f5025ffb640 1 -- 172.21.15.98:0/3975480230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50280d2c00 con 0x7f5010004d40 2024-05-09T10:51:28.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.680+0000 7f5025ffb640 1 -- 172.21.15.98:0/3975480230 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f501000b9c0 con 0x7f50280d27a0 2024-05-09T10:51:28.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.680+0000 7f50267fc640 1 --2- 172.21.15.98:0/3975480230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5010003f30 0x7f5010004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:28.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.680+0000 7f5026ffd640 1 --2- 172.21.15.98:0/3975480230 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50280d27a0 0x7f50100039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:28.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.680+0000 7f50277fe640 1 --2- 172.21.15.98:0/3975480230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5010004d40 0x7f50100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:28.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.680+0000 7f50267fc640 1 -- 172.21.15.98:0/3975480230 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50280d27a0 msgr2=0x7f50100039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:28.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.680+0000 7f50267fc640 1 --2- 172.21.15.98:0/3975480230 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50280d27a0 0x7f50100039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.680+0000 7f50267fc640 1 -- 172.21.15.98:0/3975480230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5010004d40 msgr2=0x7f50100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:28.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.680+0000 7f50267fc640 1 --2- 172.21.15.98:0/3975480230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5010004d40 0x7f50100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.680+0000 7f50267fc640 1 -- 172.21.15.98:0/3975480230 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5010003520 con 0x7f5010003f30 2024-05-09T10:51:28.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.680+0000 7f50277fe640 1 --2- 172.21.15.98:0/3975480230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5010004d40 0x7f50100051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:28.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.681+0000 7f50267fc640 1 --2- 172.21.15.98:0/3975480230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5010003f30 0x7f5010004800 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f501c009ac0 tx=0x7f5010005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d83a0d0437cee0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:28.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.681+0000 7f5025ffb640 1 -- 172.21.15.98:0/3975480230 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c036070 con 0x7f5010003f30 2024-05-09T10:51:28.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.681+0000 7f5025ffb640 1 -- 172.21.15.98:0/3975480230 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501c031400 con 0x7f5010003f30 2024-05-09T10:51:28.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.681+0000 7f5025ffb640 1 -- 172.21.15.98:0/3975480230 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c009690 con 0x7f5010003f30 2024-05-09T10:51:28.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.681+0000 7f502d370640 1 -- 172.21.15.98:0/3975480230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5010003f30 msgr2=0x7f5010004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:28.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.681+0000 7f502d370640 1 --2- 172.21.15.98:0/3975480230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5010003f30 0x7f5010004800 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f501c009ac0 tx=0x7f5010005830 comp rx=0 tx=0).stop 2024-05-09T10:51:28.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.681+0000 7f50277fe640 1 -- 172.21.15.98:0/3975480230 reap_dead start 2024-05-09T10:51:28.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.681+0000 7f502d370640 1 -- 172.21.15.98:0/3975480230 shutdown_connections 2024-05-09T10:51:28.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.681+0000 7f502d370640 1 -- 172.21.15.98:0/3975480230 >> 172.21.15.98:0/3975480230 conn(0x7f502809ce90 msgr2=0x7f50280d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:28.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.682+0000 7f502d370640 1 -- 172.21.15.98:0/3975480230 shutdown_connections 2024-05-09T10:51:28.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.682+0000 7f502d370640 1 -- 172.21.15.98:0/3975480230 wait complete. 2024-05-09T10:51:28.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.683+0000 7f502d370640 1 Processor -- start 2024-05-09T10:51:28.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.683+0000 7f502d370640 1 -- start start 2024-05-09T10:51:28.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.684+0000 7f502d370640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5010003f30 0x7f50280c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:28.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.684+0000 7f502d370640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50281f6bf0 0x7f50281f7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:28.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.684+0000 7f502d370640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50281571d0 0x7f5028157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:28.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.684+0000 7f502d370640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50280d2280 con 0x7f50281571d0 2024-05-09T10:51:28.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.684+0000 7f502d370640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50280d1fa0 con 0x7f5010003f30 2024-05-09T10:51:28.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.684+0000 7f502d370640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50280cf960 con 0x7f50281f6bf0 2024-05-09T10:51:28.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.684+0000 7f50277fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50281571d0 0x7f5028157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:28.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.684+0000 7f50277fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50281571d0 0x7f5028157660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60254/0 (socket says 172.21.15.98:60254) 2024-05-09T10:51:28.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.684+0000 7f50277fe640 1 -- 172.21.15.98:0/3822300883 learned_addr learned my addr 172.21.15.98:0/3822300883 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:28.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.684+0000 7f5026ffd640 1 --2- 172.21.15.98:0/3822300883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5010003f30 0x7f50280c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:28.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.684+0000 7f50267fc640 1 --2- 172.21.15.98:0/3822300883 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50281f6bf0 0x7f50281f7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:28.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.684+0000 7f50277fe640 1 -- 172.21.15.98:0/3822300883 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50281f6bf0 msgr2=0x7f50281f7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:28.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.684+0000 7f50277fe640 1 --2- 172.21.15.98:0/3822300883 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50281f6bf0 0x7f50281f7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.684+0000 7f50277fe640 1 -- 172.21.15.98:0/3822300883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5010003f30 msgr2=0x7f50280c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:28.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.684+0000 7f50277fe640 1 --2- 172.21.15.98:0/3822300883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5010003f30 0x7f50280c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.684+0000 7f50277fe640 1 -- 172.21.15.98:0/3822300883 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50280c6e70 con 0x7f50281571d0 2024-05-09T10:51:28.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.684+0000 7f50277fe640 1 --2- 172.21.15.98:0/3822300883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50281571d0 0x7f5028157660 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f502806d960 tx=0x7f5020011860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:28.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.684+0000 7f500f7fe640 1 -- 172.21.15.98:0/3822300883 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5020019020 con 0x7f50281571d0 2024-05-09T10:51:28.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.684+0000 7f500f7fe640 1 -- 172.21.15.98:0/3822300883 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5020004460 con 0x7f50281571d0 2024-05-09T10:51:28.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.684+0000 7f502d370640 1 -- 172.21.15.98:0/3822300883 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5028159230 con 0x7f50281571d0 2024-05-09T10:51:28.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.684+0000 7f500f7fe640 1 -- 172.21.15.98:0/3822300883 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5020003850 con 0x7f50281571d0 2024-05-09T10:51:28.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.684+0000 7f502d370640 1 -- 172.21.15.98:0/3822300883 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50281597b0 con 0x7f50281571d0 2024-05-09T10:51:28.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.685+0000 7f502d370640 1 -- 172.21.15.98:0/3822300883 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5028069cb0 con 0x7f50281571d0 2024-05-09T10:51:28.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.686+0000 7f500f7fe640 1 -- 172.21.15.98:0/3822300883 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f502001c020 con 0x7f50281571d0 2024-05-09T10:51:28.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.686+0000 7f500f7fe640 1 --2- 172.21.15.98:0/3822300883 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f501406b680 0x7f501406db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:28.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.686+0000 7f500f7fe640 1 -- 172.21.15.98:0/3822300883 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f502008a540 con 0x7f50281571d0 2024-05-09T10:51:28.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.686+0000 7f5026ffd640 1 --2- 172.21.15.98:0/3822300883 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f501406b680 0x7f501406db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:28.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.687+0000 7f5026ffd640 1 --2- 172.21.15.98:0/3822300883 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f501406b680 0x7f501406db40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f50280cac10 tx=0x7f5018006040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:28.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.690+0000 7f500f7fe640 1 -- 172.21.15.98:0/3822300883 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5020023030 con 0x7f50281571d0 2024-05-09T10:51:28.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.895+0000 7f502d370640 1 -- 172.21.15.98:0/3822300883 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50280d42d0 con 0x7f50281571d0 2024-05-09T10:51:28.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.896+0000 7f500f7fe640 1 -- 172.21.15.98:0/3822300883 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f50200591d0 con 0x7f50281571d0 2024-05-09T10:51:28.889 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:28.889 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2332373437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2332373437},{"type":"v1","addr":"172.21.15.105:6834","nonce":2332373437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4849,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/4092257628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":4092257628},{"type":"v1","addr":"172.21.15.98:6836","nonce":4092257628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4855,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3957569958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3957569958},{"type":"v1","addr":"172.21.15.105:6835","nonce":3957569958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T10:51:06.745057+0000","modified":"2024-05-09T10:51:08.738161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5127":{"gid":5127,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/164638690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":164638690},{"type":"v1","addr":"172.21.15.98:6837","nonce":164638690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-09T10:51:28.890 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 14 2024-05-09T10:51:28.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.898+0000 7f502d370640 1 -- 172.21.15.98:0/3822300883 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f501406b680 msgr2=0x7f501406db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:28.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.898+0000 7f502d370640 1 --2- 172.21.15.98:0/3822300883 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f501406b680 0x7f501406db40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f50280cac10 tx=0x7f5018006040 comp rx=0 tx=0).stop 2024-05-09T10:51:28.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.898+0000 7f502d370640 1 -- 172.21.15.98:0/3822300883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50281571d0 msgr2=0x7f5028157660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:28.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.898+0000 7f502d370640 1 --2- 172.21.15.98:0/3822300883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50281571d0 0x7f5028157660 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f502806d960 tx=0x7f5020011860 comp rx=0 tx=0).stop 2024-05-09T10:51:28.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.898+0000 7f502d370640 1 -- 172.21.15.98:0/3822300883 shutdown_connections 2024-05-09T10:51:28.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.898+0000 7f502d370640 1 --2- 172.21.15.98:0/3822300883 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f501406b680 0x7f501406db40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.898+0000 7f502d370640 1 --2- 172.21.15.98:0/3822300883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50281571d0 0x7f5028157660 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.898+0000 7f502d370640 1 --2- 172.21.15.98:0/3822300883 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f50281f6bf0 0x7f50281f7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.898+0000 7f502d370640 1 --2- 172.21.15.98:0/3822300883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5010003f30 0x7f50280c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:28.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.898+0000 7f502d370640 1 -- 172.21.15.98:0/3822300883 >> 172.21.15.98:0/3822300883 conn(0x7f502809ce90 msgr2=0x7f502806b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:28.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.898+0000 7f502d370640 1 -- 172.21.15.98:0/3822300883 shutdown_connections 2024-05-09T10:51:28.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:28.898+0000 7f502d370640 1 -- 172.21.15.98:0/3822300883 wait complete. 2024-05-09T10:51:28.902 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T10:51:29.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.009+0000 7fd1535ce640 1 Processor -- start 2024-05-09T10:51:29.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.009+0000 7fd1535ce640 1 -- start start 2024-05-09T10:51:29.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.010+0000 7fd1535ce640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd14c15db90 0x7fd14c06d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.010+0000 7fd1535ce640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c0cdb30 con 0x7fd14c06abe0 2024-05-09T10:51:29.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.010+0000 7fd1535ce640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c071f40 con 0x7fd14c15df70 2024-05-09T10:51:29.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.010+0000 7fd1535ce640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c072220 con 0x7fd14c15db90 2024-05-09T10:51:29.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.010+0000 7fd151b6b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fd14c06abe0 0x7fd14c0cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:47030/0 (socket says 172.21.15.98:47030) 2024-05-09T10:51:29.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.010+0000 7fd151b6b640 1 -- 172.21.15.98:0/3759755956 learned_addr learned my addr 172.21.15.98:0/3759755956 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:29.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.010+0000 7fd13ffff640 1 -- 172.21.15.98:0/3759755956 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3875922407 0 0) 0x7fd144001490 con 0x7fd14c06abe0 2024-05-09T10:51:29.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.010+0000 7fd15136a640 1 --2- 172.21.15.98:0/3759755956 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd14c15db90 0x7fd14c06d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.010+0000 7fd13ffff640 1 -- 172.21.15.98:0/3759755956 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd138003520 con 0x7fd14c06abe0 2024-05-09T10:51:29.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.010+0000 7fd13ffff640 1 -- 172.21.15.98:0/3759755956 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 681149427 0 0) 0x7fd138003520 con 0x7fd14c06abe0 2024-05-09T10:51:29.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.010+0000 7fd13ffff640 1 -- 172.21.15.98:0/3759755956 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd14c15db90 msgr2=0x7fd14c06d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:29.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.010+0000 7fd13ffff640 1 --2- 172.21.15.98:0/3759755956 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd14c15db90 0x7fd14c06d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.010+0000 7fd13ffff640 1 -- 172.21.15.98:0/3759755956 >> v1:172.21.15.105:6789/0 conn(0x7fd14c15df70 legacy=0x7fd14c0713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.010+0000 7fd13ffff640 1 -- 172.21.15.98:0/3759755956 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c072500 con 0x7fd14c06abe0 2024-05-09T10:51:29.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.010+0000 7fd13ffff640 1 -- 172.21.15.98:0/3759755956 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7fd144001bc0 con 0x7fd14c06abe0 2024-05-09T10:51:29.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.010+0000 7fd13ffff640 1 -- 172.21.15.98:0/3759755956 >> v1:172.21.15.98:6789/0 conn(0x7fd14c06abe0 legacy=0x7fd14c0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.010+0000 7fd13ffff640 1 --2- 172.21.15.98:0/3759755956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1380039e0 0x7fd138003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.010+0000 7fd13ffff640 1 --2- 172.21.15.98:0/3759755956 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd138004800 0x7fd138004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.010+0000 7fd13ffff640 1 --2- 172.21.15.98:0/3759755956 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd14c15db90 0x7fd138005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.010+0000 7fd13ffff640 1 -- 172.21.15.98:0/3759755956 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c072220 con 0x7fd1380039e0 2024-05-09T10:51:29.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.010+0000 7fd13ffff640 1 -- 172.21.15.98:0/3759755956 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13800baf0 con 0x7fd138004800 2024-05-09T10:51:29.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.010+0000 7fd13ffff640 1 -- 172.21.15.98:0/3759755956 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd13800bcf0 con 0x7fd14c15db90 2024-05-09T10:51:29.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.011+0000 7fd15136a640 1 --2- 172.21.15.98:0/3759755956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1380039e0 0x7fd138003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.011+0000 7fd151b6b640 1 --2- 172.21.15.98:0/3759755956 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd14c15db90 0x7fd138005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.011+0000 7fd150b69640 1 --2- 172.21.15.98:0/3759755956 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd138004800 0x7fd138004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.011+0000 7fd15136a640 1 -- 172.21.15.98:0/3759755956 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd14c15db90 msgr2=0x7fd138005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:29.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.011+0000 7fd15136a640 1 --2- 172.21.15.98:0/3759755956 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd14c15db90 0x7fd138005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.011+0000 7fd15136a640 1 -- 172.21.15.98:0/3759755956 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd138004800 msgr2=0x7fd138004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:29.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.011+0000 7fd15136a640 1 --2- 172.21.15.98:0/3759755956 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd138004800 0x7fd138004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.011+0000 7fd15136a640 1 -- 172.21.15.98:0/3759755956 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd138003520 con 0x7fd1380039e0 2024-05-09T10:51:29.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.011+0000 7fd150b69640 1 --2- 172.21.15.98:0/3759755956 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd138004800 0x7fd138004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:29.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.011+0000 7fd15136a640 1 --2- 172.21.15.98:0/3759755956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1380039e0 0x7fd138003e00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd148009ac0 tx=0x7fd138005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8b234789471d41 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:29.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.012+0000 7fd13ffff640 1 -- 172.21.15.98:0/3759755956 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd148036070 con 0x7fd1380039e0 2024-05-09T10:51:29.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.012+0000 7fd13ffff640 1 -- 172.21.15.98:0/3759755956 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd148031400 con 0x7fd1380039e0 2024-05-09T10:51:29.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.012+0000 7fd13ffff640 1 -- 172.21.15.98:0/3759755956 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd148009690 con 0x7fd1380039e0 2024-05-09T10:51:29.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.012+0000 7fd1535ce640 1 -- 172.21.15.98:0/3759755956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1380039e0 msgr2=0x7fd138003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.012+0000 7fd1535ce640 1 --2- 172.21.15.98:0/3759755956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1380039e0 0x7fd138003e00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd148009ac0 tx=0x7fd138005830 comp rx=0 tx=0).stop 2024-05-09T10:51:29.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.012+0000 7fd151b6b640 1 -- 172.21.15.98:0/3759755956 reap_dead start 2024-05-09T10:51:29.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.012+0000 7fd1535ce640 1 -- 172.21.15.98:0/3759755956 shutdown_connections 2024-05-09T10:51:29.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.012+0000 7fd1535ce640 1 -- 172.21.15.98:0/3759755956 >> 172.21.15.98:0/3759755956 conn(0x7fd14c066f20 msgr2=0x7fd14c06bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:29.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.012+0000 7fd1535ce640 1 -- 172.21.15.98:0/3759755956 shutdown_connections 2024-05-09T10:51:29.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.012+0000 7fd1535ce640 1 -- 172.21.15.98:0/3759755956 wait complete. 2024-05-09T10:51:29.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.013+0000 7fd1535ce640 1 Processor -- start 2024-05-09T10:51:29.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.013+0000 7fd1535ce640 1 -- start start 2024-05-09T10:51:29.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.014+0000 7fd1535ce640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1380039e0 0x7fd14c155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.014+0000 7fd1535ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd14c1f7380 0x7fd14c147250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.014+0000 7fd1535ce640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd14c0b0330 0x7fd14c0aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.014+0000 7fd1535ce640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c071ae0 con 0x7fd14c0b0330 2024-05-09T10:51:29.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.014+0000 7fd1535ce640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c0cefd0 con 0x7fd14c1f7380 2024-05-09T10:51:29.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.014+0000 7fd1535ce640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c071c60 con 0x7fd1380039e0 2024-05-09T10:51:29.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.014+0000 7fd151b6b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd14c0b0330 0x7fd14c0aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.014+0000 7fd151b6b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd14c0b0330 0x7fd14c0aec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60272/0 (socket says 172.21.15.98:60272) 2024-05-09T10:51:29.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.014+0000 7fd151b6b640 1 -- 172.21.15.98:0/3638689598 learned_addr learned my addr 172.21.15.98:0/3638689598 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:29.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.014+0000 7fd150b69640 1 --2- 172.21.15.98:0/3638689598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd14c1f7380 0x7fd14c147250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.014+0000 7fd15136a640 1 --2- 172.21.15.98:0/3638689598 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1380039e0 0x7fd14c155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.015+0000 7fd151b6b640 1 -- 172.21.15.98:0/3638689598 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1380039e0 msgr2=0x7fd14c155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.015+0000 7fd151b6b640 1 --2- 172.21.15.98:0/3638689598 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1380039e0 0x7fd14c155210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.015+0000 7fd151b6b640 1 -- 172.21.15.98:0/3638689598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd14c1f7380 msgr2=0x7fd14c147250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.015+0000 7fd151b6b640 1 --2- 172.21.15.98:0/3638689598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd14c1f7380 0x7fd14c147250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.015+0000 7fd151b6b640 1 -- 172.21.15.98:0/3638689598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c1f3db0 con 0x7fd14c0b0330 2024-05-09T10:51:29.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.015+0000 7fd15136a640 1 --2- 172.21.15.98:0/3638689598 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1380039e0 0x7fd14c155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:29.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.015+0000 7fd151b6b640 1 --2- 172.21.15.98:0/3638689598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd14c0b0330 0x7fd14c0aec50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd14c067330 tx=0x7fd14400af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:29.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.015+0000 7fd13dffb640 1 -- 172.21.15.98:0/3638689598 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd144007840 con 0x7fd14c0b0330 2024-05-09T10:51:29.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.015+0000 7fd13dffb640 1 -- 172.21.15.98:0/3638689598 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd144019070 con 0x7fd14c0b0330 2024-05-09T10:51:29.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.015+0000 7fd1535ce640 1 -- 172.21.15.98:0/3638689598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14c1f7880 con 0x7fd14c0b0330 2024-05-09T10:51:29.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.015+0000 7fd13dffb640 1 -- 172.21.15.98:0/3638689598 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1440133f0 con 0x7fd14c0b0330 2024-05-09T10:51:29.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.015+0000 7fd1535ce640 1 -- 172.21.15.98:0/3638689598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14c1f7e40 con 0x7fd14c0b0330 2024-05-09T10:51:29.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.016+0000 7fd1535ce640 1 -- 172.21.15.98:0/3638689598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14c069cb0 con 0x7fd14c0b0330 2024-05-09T10:51:29.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.017+0000 7fd13dffb640 1 -- 172.21.15.98:0/3638689598 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7fd1440135b0 con 0x7fd14c0b0330 2024-05-09T10:51:29.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.017+0000 7fd13dffb640 1 --2- 172.21.15.98:0/3638689598 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd13006b680 0x7fd13006db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.017+0000 7fd13dffb640 1 -- 172.21.15.98:0/3638689598 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd14408aac0 con 0x7fd14c0b0330 2024-05-09T10:51:29.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.017+0000 7fd15136a640 1 --2- 172.21.15.98:0/3638689598 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd13006b680 0x7fd13006db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.018+0000 7fd15136a640 1 --2- 172.21.15.98:0/3638689598 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd13006b680 0x7fd13006db40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd148033000 tx=0x7fd148008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:29.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.021+0000 7fd13dffb640 1 -- 172.21.15.98:0/3638689598 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd14c1f3db0 con 0x7fd14c0b0330 2024-05-09T10:51:29.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.227+0000 7fd1535ce640 1 -- 172.21.15.98:0/3638689598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd14c0d05a0 con 0x7fd14c0b0330 2024-05-09T10:51:29.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.228+0000 7fd13dffb640 1 -- 172.21.15.98:0/3638689598 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fd14c0cdb30 con 0x7fd14c0b0330 2024-05-09T10:51:29.221 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:29.221 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2332373437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2332373437},{"type":"v1","addr":"172.21.15.105:6834","nonce":2332373437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4849,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/4092257628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":4092257628},{"type":"v1","addr":"172.21.15.98:6836","nonce":4092257628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4855,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3957569958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3957569958},{"type":"v1","addr":"172.21.15.105:6835","nonce":3957569958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T10:51:06.745057+0000","modified":"2024-05-09T10:51:08.738161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5127":{"gid":5127,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/164638690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":164638690},{"type":"v1","addr":"172.21.15.98:6837","nonce":164638690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-05-09T10:51:29.222 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 14 2024-05-09T10:51:29.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.230+0000 7fd1535ce640 1 -- 172.21.15.98:0/3638689598 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd13006b680 msgr2=0x7fd13006db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.230+0000 7fd1535ce640 1 --2- 172.21.15.98:0/3638689598 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd13006b680 0x7fd13006db40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd148033000 tx=0x7fd148008030 comp rx=0 tx=0).stop 2024-05-09T10:51:29.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.230+0000 7fd1535ce640 1 -- 172.21.15.98:0/3638689598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd14c0b0330 msgr2=0x7fd14c0aec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.230+0000 7fd1535ce640 1 --2- 172.21.15.98:0/3638689598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd14c0b0330 0x7fd14c0aec50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd14c067330 tx=0x7fd14400af40 comp rx=0 tx=0).stop 2024-05-09T10:51:29.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.230+0000 7fd1535ce640 1 -- 172.21.15.98:0/3638689598 shutdown_connections 2024-05-09T10:51:29.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.230+0000 7fd1535ce640 1 --2- 172.21.15.98:0/3638689598 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd13006b680 0x7fd13006db40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.230+0000 7fd1535ce640 1 --2- 172.21.15.98:0/3638689598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd14c0b0330 0x7fd14c0aec50 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.230+0000 7fd1535ce640 1 --2- 172.21.15.98:0/3638689598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd14c1f7380 0x7fd14c147250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.230+0000 7fd1535ce640 1 --2- 172.21.15.98:0/3638689598 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1380039e0 0x7fd14c155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.230+0000 7fd1535ce640 1 -- 172.21.15.98:0/3638689598 >> 172.21.15.98:0/3638689598 conn(0x7fd14c066f20 msgr2=0x7fd14c160d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:29.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.230+0000 7fd1535ce640 1 -- 172.21.15.98:0/3638689598 shutdown_connections 2024-05-09T10:51:29.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.230+0000 7fd1535ce640 1 -- 172.21.15.98:0/3638689598 wait complete. 2024-05-09T10:51:29.234 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:51:29.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.340+0000 7f633c513640 1 Processor -- start 2024-05-09T10:51:29.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.340+0000 7f633c513640 1 -- start start 2024-05-09T10:51:29.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.341+0000 7f633c513640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f633406abe0 0x7f633406b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.341+0000 7f633c513640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63340cdb30 con 0x7f633415df70 2024-05-09T10:51:29.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.341+0000 7f633c513640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6334071f40 con 0x7f633415db90 2024-05-09T10:51:29.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.341+0000 7f633c513640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6334072220 con 0x7f633406abe0 2024-05-09T10:51:29.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.341+0000 7f6339aae640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f633415df70 0x7f63340713d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:47044/0 (socket says 172.21.15.98:47044) 2024-05-09T10:51:29.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.341+0000 7f6339aae640 1 -- 172.21.15.98:0/4079645360 learned_addr learned my addr 172.21.15.98:0/4079645360 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:29.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.341+0000 7f633aab0640 1 --2- 172.21.15.98:0/4079645360 >> v2:172.21.15.105:3300/0 conn(0x7f633406abe0 0x7f633406b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.341+0000 7f63392ad640 1 -- 172.21.15.98:0/4079645360 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1329316490 0 0) 0x7f6324001110 con 0x7f633415df70 2024-05-09T10:51:29.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.341+0000 7f63392ad640 1 -- 172.21.15.98:0/4079645360 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6320003520 con 0x7f633415df70 2024-05-09T10:51:29.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.341+0000 7f63392ad640 1 -- 172.21.15.98:0/4079645360 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 610597829 0 0) 0x7f6334071f40 con 0x7f633415db90 2024-05-09T10:51:29.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.341+0000 7f63392ad640 1 -- 172.21.15.98:0/4079645360 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6324001110 con 0x7f633415db90 2024-05-09T10:51:29.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.341+0000 7f633aab0640 1 -- 172.21.15.98:0/4079645360 >> v1:172.21.15.105:6789/0 conn(0x7f633415db90 legacy=0x7f633406d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.341+0000 7f633aab0640 1 -- 172.21.15.98:0/4079645360 >> v1:172.21.15.98:6789/0 conn(0x7f633415df70 legacy=0x7f63340713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.341+0000 7f633aab0640 1 -- 172.21.15.98:0/4079645360 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6334072500 con 0x7f633406abe0 2024-05-09T10:51:29.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.342+0000 7f633aab0640 1 --2- 172.21.15.98:0/4079645360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f633406abe0 0x7f633406b070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f632c009d90 tx=0x7f632c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0ceea5cfdc11f07 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:29.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.342+0000 7f63392ad640 1 -- 172.21.15.98:0/4079645360 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c007740 con 0x7f633406abe0 2024-05-09T10:51:29.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.342+0000 7f63392ad640 1 -- 172.21.15.98:0/4079645360 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632c0078e0 con 0x7f633406abe0 2024-05-09T10:51:29.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.342+0000 7f63392ad640 1 -- 172.21.15.98:0/4079645360 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c007c90 con 0x7f633406abe0 2024-05-09T10:51:29.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.343+0000 7f633c513640 1 -- 172.21.15.98:0/4079645360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f633406abe0 msgr2=0x7f633406b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.343+0000 7f633c513640 1 --2- 172.21.15.98:0/4079645360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f633406abe0 0x7f633406b070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f632c009d90 tx=0x7f632c02a660 comp rx=0 tx=0).stop 2024-05-09T10:51:29.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.343+0000 7f633c513640 1 -- 172.21.15.98:0/4079645360 shutdown_connections 2024-05-09T10:51:29.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.343+0000 7f633c513640 1 --2- 172.21.15.98:0/4079645360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f633406abe0 0x7f633406b070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.343+0000 7f633c513640 1 -- 172.21.15.98:0/4079645360 >> 172.21.15.98:0/4079645360 conn(0x7f6334066f20 msgr2=0x7f633406bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:29.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.343+0000 7f633c513640 1 -- 172.21.15.98:0/4079645360 shutdown_connections 2024-05-09T10:51:29.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.343+0000 7f633c513640 1 -- 172.21.15.98:0/4079645360 wait complete. 2024-05-09T10:51:29.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.344+0000 7f633c513640 1 Processor -- start 2024-05-09T10:51:29.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.344+0000 7f633c513640 1 -- start start 2024-05-09T10:51:29.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.345+0000 7f633c513640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f633406abe0 0x7f633415e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.345+0000 7f633c513640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f633415db90 0x7f633415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.345+0000 7f633c513640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f633415df70 0x7f63341431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.345+0000 7f633c513640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6334071ae0 con 0x7f633406abe0 2024-05-09T10:51:29.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.345+0000 7f633c513640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63340cefd0 con 0x7f633415db90 2024-05-09T10:51:29.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.345+0000 7f633c513640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6334071c60 con 0x7f633415df70 2024-05-09T10:51:29.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.345+0000 7f633a2af640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f633406abe0 0x7f633415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.345+0000 7f633a2af640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f633406abe0 0x7f633415e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60274/0 (socket says 172.21.15.98:60274) 2024-05-09T10:51:29.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.345+0000 7f633a2af640 1 -- 172.21.15.98:0/3568797918 learned_addr learned my addr 172.21.15.98:0/3568797918 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:29.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.345+0000 7f633aab0640 1 --2- 172.21.15.98:0/3568797918 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f633415df70 0x7f63341431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.345+0000 7f6339aae640 1 --2- 172.21.15.98:0/3568797918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f633415db90 0x7f633415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.345+0000 7f633a2af640 1 -- 172.21.15.98:0/3568797918 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f633415df70 msgr2=0x7f63341431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.345+0000 7f633a2af640 1 --2- 172.21.15.98:0/3568797918 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f633415df70 0x7f63341431f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.345+0000 7f633a2af640 1 -- 172.21.15.98:0/3568797918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f633415db90 msgr2=0x7f633415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.345+0000 7f633a2af640 1 --2- 172.21.15.98:0/3568797918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f633415db90 0x7f633415ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.345+0000 7f633a2af640 1 -- 172.21.15.98:0/3568797918 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63340adbc0 con 0x7f633406abe0 2024-05-09T10:51:29.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.345+0000 7f6339aae640 1 --2- 172.21.15.98:0/3568797918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f633415db90 0x7f633415ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:29.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.345+0000 7f633a2af640 1 --2- 172.21.15.98:0/3568797918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f633406abe0 0x7f633415e480 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f633000c500 tx=0x7f633000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:29.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.346+0000 7f632affd640 1 -- 172.21.15.98:0/3568797918 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330017070 con 0x7f633406abe0 2024-05-09T10:51:29.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.346+0000 7f632affd640 1 -- 172.21.15.98:0/3568797918 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6330005310 con 0x7f633406abe0 2024-05-09T10:51:29.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.346+0000 7f633c513640 1 -- 172.21.15.98:0/3568797918 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63340adea0 con 0x7f633406abe0 2024-05-09T10:51:29.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.346+0000 7f632affd640 1 -- 172.21.15.98:0/3568797918 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330003830 con 0x7f633406abe0 2024-05-09T10:51:29.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.346+0000 7f633c513640 1 -- 172.21.15.98:0/3568797918 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63340ae460 con 0x7f633406abe0 2024-05-09T10:51:29.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.347+0000 7f633c513640 1 -- 172.21.15.98:0/3568797918 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6334069cb0 con 0x7f633406abe0 2024-05-09T10:51:29.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.347+0000 7f632affd640 1 -- 172.21.15.98:0/3568797918 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f633001a070 con 0x7f633406abe0 2024-05-09T10:51:29.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.347+0000 7f632affd640 1 --2- 172.21.15.98:0/3568797918 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f631806b680 0x7f631806db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.348+0000 7f632affd640 1 -- 172.21.15.98:0/3568797918 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63300897e0 con 0x7f633406abe0 2024-05-09T10:51:29.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.348+0000 7f6339aae640 1 --2- 172.21.15.98:0/3568797918 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f631806b680 0x7f631806db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.348+0000 7f6339aae640 1 --2- 172.21.15.98:0/3568797918 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f631806b680 0x7f631806db40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f63341f6fc0 tx=0x7f6324009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:29.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.352+0000 7f632affd640 1 -- 172.21.15.98:0/3568797918 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6330057da0 con 0x7f633406abe0 2024-05-09T10:51:29.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.525+0000 7f633c513640 1 -- 172.21.15.98:0/3568797918 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f633406e340 con 0x7f633406abe0 2024-05-09T10:51:29.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.526+0000 7f632affd640 1 -- 172.21.15.98:0/3568797918 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+16669 (secure 0 0 0) 0x7f633005be40 con 0x7f633406abe0 2024-05-09T10:51:29.519 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:29.519 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":31,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:51:08.400895+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-09T10:51:05.150410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-09T10:51:05.708391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:51:29.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.528+0000 7f633c513640 1 -- 172.21.15.98:0/3568797918 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f631806b680 msgr2=0x7f631806db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.528+0000 7f633c513640 1 --2- 172.21.15.98:0/3568797918 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f631806b680 0x7f631806db40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f63341f6fc0 tx=0x7f6324009040 comp rx=0 tx=0).stop 2024-05-09T10:51:29.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.528+0000 7f633c513640 1 -- 172.21.15.98:0/3568797918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f633406abe0 msgr2=0x7f633415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.528+0000 7f633c513640 1 --2- 172.21.15.98:0/3568797918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f633406abe0 0x7f633415e480 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f633000c500 tx=0x7f633000c9c0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.528+0000 7f633c513640 1 -- 172.21.15.98:0/3568797918 shutdown_connections 2024-05-09T10:51:29.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.528+0000 7f633c513640 1 --2- 172.21.15.98:0/3568797918 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f631806b680 0x7f631806db40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.528+0000 7f633c513640 1 --2- 172.21.15.98:0/3568797918 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f633415df70 0x7f63341431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.528+0000 7f633c513640 1 --2- 172.21.15.98:0/3568797918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f633415db90 0x7f633415ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.528+0000 7f633c513640 1 --2- 172.21.15.98:0/3568797918 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f633406abe0 0x7f633415e480 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.528+0000 7f633c513640 1 -- 172.21.15.98:0/3568797918 >> 172.21.15.98:0/3568797918 conn(0x7f6334066f20 msgr2=0x7f6334161ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:29.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.528+0000 7f633c513640 1 -- 172.21.15.98:0/3568797918 shutdown_connections 2024-05-09T10:51:29.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.528+0000 7f633c513640 1 -- 172.21.15.98:0/3568797918 wait complete. 2024-05-09T10:51:29.532 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T10:51:29.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.639+0000 7f7491238640 1 Processor -- start 2024-05-09T10:51:29.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.639+0000 7f7491238640 1 -- start start 2024-05-09T10:51:29.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.640+0000 7f7491238640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f748c06c300 0x7f748c166d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.640+0000 7f7491238640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748c15fb70 con 0x7f748c15f1c0 2024-05-09T10:51:29.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.640+0000 7f7491238640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748c15fe50 con 0x7f748c0cf430 2024-05-09T10:51:29.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.640+0000 7f7491238640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748c160930 con 0x7f748c06c300 2024-05-09T10:51:29.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.640+0000 7f748ad9d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f748c15f1c0 0x7f748c06edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:47058/0 (socket says 172.21.15.98:47058) 2024-05-09T10:51:29.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.640+0000 7f748ad9d640 1 -- 172.21.15.98:0/937506741 learned_addr learned my addr 172.21.15.98:0/937506741 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:29.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.640+0000 7f7489d9b640 1 -- 172.21.15.98:0/937506741 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 116180998 0 0) 0x7f748c15fb70 con 0x7f748c15f1c0 2024-05-09T10:51:29.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.640+0000 7f748b59e640 1 --2- 172.21.15.98:0/937506741 >> v2:172.21.15.105:3300/0 conn(0x7f748c06c300 0x7f748c166d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.640+0000 7f7489d9b640 1 -- 172.21.15.98:0/937506741 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7474003520 con 0x7f748c15f1c0 2024-05-09T10:51:29.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.640+0000 7f7489d9b640 1 -- 172.21.15.98:0/937506741 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2851932623 0 0) 0x7f7474003520 con 0x7f748c15f1c0 2024-05-09T10:51:29.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.640+0000 7f7489d9b640 1 -- 172.21.15.98:0/937506741 >> v2:172.21.15.105:3300/0 conn(0x7f748c06c300 msgr2=0x7f748c166d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:29.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.640+0000 7f7489d9b640 1 --2- 172.21.15.98:0/937506741 >> v2:172.21.15.105:3300/0 conn(0x7f748c06c300 0x7f748c166d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.640+0000 7f7489d9b640 1 -- 172.21.15.98:0/937506741 >> v1:172.21.15.105:6789/0 conn(0x7f748c0cf430 legacy=0x7f748c06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.640+0000 7f7489d9b640 1 -- 172.21.15.98:0/937506741 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c160130 con 0x7f748c15f1c0 2024-05-09T10:51:29.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.640+0000 7f7489d9b640 1 -- 172.21.15.98:0/937506741 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f747c001670 con 0x7f748c15f1c0 2024-05-09T10:51:29.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.640+0000 7f7489d9b640 1 -- 172.21.15.98:0/937506741 >> v1:172.21.15.98:6789/0 conn(0x7f748c15f1c0 legacy=0x7f748c06edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.641+0000 7f7489d9b640 1 --2- 172.21.15.98:0/937506741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74740039b0 0x7f7474003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.641+0000 7f7489d9b640 1 --2- 172.21.15.98:0/937506741 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7474004800 0x7f7474004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.641+0000 7f7489d9b640 1 --2- 172.21.15.98:0/937506741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74740051c0 0x7f747400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.641+0000 7f7489d9b640 1 -- 172.21.15.98:0/937506741 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747400c030 con 0x7f74740039b0 2024-05-09T10:51:29.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.641+0000 7f7489d9b640 1 -- 172.21.15.98:0/937506741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747400c1e0 con 0x7f74740051c0 2024-05-09T10:51:29.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.641+0000 7f7489d9b640 1 -- 172.21.15.98:0/937506741 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f747400c3e0 con 0x7f7474004800 2024-05-09T10:51:29.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.641+0000 7f748ad9d640 1 --2- 172.21.15.98:0/937506741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74740039b0 0x7f7474003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.641+0000 7f748a59c640 1 --2- 172.21.15.98:0/937506741 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7474004800 0x7f7474004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.641+0000 7f748b59e640 1 --2- 172.21.15.98:0/937506741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74740051c0 0x7f747400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.641+0000 7f748ad9d640 1 -- 172.21.15.98:0/937506741 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7474004800 msgr2=0x7f7474004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:29.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.641+0000 7f748ad9d640 1 --2- 172.21.15.98:0/937506741 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7474004800 0x7f7474004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.641+0000 7f748ad9d640 1 -- 172.21.15.98:0/937506741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74740051c0 msgr2=0x7f747400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:29.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.641+0000 7f748ad9d640 1 --2- 172.21.15.98:0/937506741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74740051c0 0x7f747400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.641+0000 7f748ad9d640 1 -- 172.21.15.98:0/937506741 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7474003520 con 0x7f74740039b0 2024-05-09T10:51:29.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.641+0000 7f748b59e640 1 -- 172.21.15.98:0/937506741 reap_dead start 2024-05-09T10:51:29.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.642+0000 7f748ad9d640 1 --2- 172.21.15.98:0/937506741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74740039b0 0x7f7474003dd0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f747c00c210 tx=0x7f7474005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9724d08789bfe2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:29.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.642+0000 7f7489d9b640 1 -- 172.21.15.98:0/937506741 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c007d30 con 0x7f74740039b0 2024-05-09T10:51:29.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.642+0000 7f7489d9b640 1 -- 172.21.15.98:0/937506741 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747c005ad0 con 0x7f74740039b0 2024-05-09T10:51:29.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.642+0000 7f7489d9b640 1 -- 172.21.15.98:0/937506741 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c007070 con 0x7f74740039b0 2024-05-09T10:51:29.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.642+0000 7f7491238640 1 -- 172.21.15.98:0/937506741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74740039b0 msgr2=0x7f7474003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.642+0000 7f7491238640 1 --2- 172.21.15.98:0/937506741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74740039b0 0x7f7474003dd0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f747c00c210 tx=0x7f7474005830 comp rx=0 tx=0).stop 2024-05-09T10:51:29.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.642+0000 7f7491238640 1 -- 172.21.15.98:0/937506741 shutdown_connections 2024-05-09T10:51:29.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.642+0000 7f7491238640 1 --2- 172.21.15.98:0/937506741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74740039b0 0x7f7474003dd0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.642+0000 7f7491238640 1 -- 172.21.15.98:0/937506741 >> 172.21.15.98:0/937506741 conn(0x7f748c0ccaa0 msgr2=0x7f748c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:29.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.642+0000 7f7491238640 1 -- 172.21.15.98:0/937506741 shutdown_connections 2024-05-09T10:51:29.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.642+0000 7f7491238640 1 -- 172.21.15.98:0/937506741 wait complete. 2024-05-09T10:51:29.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.644+0000 7f7491238640 1 Processor -- start 2024-05-09T10:51:29.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.644+0000 7f7491238640 1 -- start start 2024-05-09T10:51:29.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.645+0000 7f7491238640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74740039b0 0x7f748c0c2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.645+0000 7f7491238640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f748c09f4e0 0x7f748c09fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.645+0000 7f7491238640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f748c09ba60 0x7f748c09bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.645+0000 7f7491238640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c15f720 con 0x7f748c09f4e0 2024-05-09T10:51:29.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.645+0000 7f7491238640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c15f5a0 con 0x7f748c09ba60 2024-05-09T10:51:29.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.645+0000 7f7491238640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748c06bbf0 con 0x7f74740039b0 2024-05-09T10:51:29.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.645+0000 7f748a59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f748c09f4e0 0x7f748c09fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.645+0000 7f748a59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f748c09f4e0 0x7f748c09fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60290/0 (socket says 172.21.15.98:60290) 2024-05-09T10:51:29.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.645+0000 7f748a59c640 1 -- 172.21.15.98:0/1105623489 learned_addr learned my addr 172.21.15.98:0/1105623489 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:29.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.645+0000 7f748ad9d640 1 --2- 172.21.15.98:0/1105623489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74740039b0 0x7f748c0c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.645+0000 7f748b59e640 1 --2- 172.21.15.98:0/1105623489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f748c09ba60 0x7f748c09bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.645+0000 7f748a59c640 1 -- 172.21.15.98:0/1105623489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74740039b0 msgr2=0x7f748c0c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.645+0000 7f748a59c640 1 --2- 172.21.15.98:0/1105623489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74740039b0 0x7f748c0c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.645+0000 7f748a59c640 1 -- 172.21.15.98:0/1105623489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f748c09ba60 msgr2=0x7f748c09bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.645+0000 7f748a59c640 1 --2- 172.21.15.98:0/1105623489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f748c09ba60 0x7f748c09bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.645+0000 7f748a59c640 1 -- 172.21.15.98:0/1105623489 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c0c2c50 con 0x7f748c09f4e0 2024-05-09T10:51:29.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.645+0000 7f748a59c640 1 --2- 172.21.15.98:0/1105623489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f748c09f4e0 0x7f748c09fea0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f748000f5f0 tx=0x7f748000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:29.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.645+0000 7f74737fe640 1 -- 172.21.15.98:0/1105623489 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74800052e0 con 0x7f748c09f4e0 2024-05-09T10:51:29.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.645+0000 7f74737fe640 1 -- 172.21.15.98:0/1105623489 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7480018070 con 0x7f748c09f4e0 2024-05-09T10:51:29.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.645+0000 7f7491238640 1 -- 172.21.15.98:0/1105623489 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748c0a06d0 con 0x7f748c09f4e0 2024-05-09T10:51:29.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.645+0000 7f74737fe640 1 -- 172.21.15.98:0/1105623489 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7480003810 con 0x7f748c09f4e0 2024-05-09T10:51:29.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.645+0000 7f7491238640 1 -- 172.21.15.98:0/1105623489 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748c0a0c90 con 0x7f748c09f4e0 2024-05-09T10:51:29.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.646+0000 7f7491238640 1 -- 172.21.15.98:0/1105623489 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748c069cb0 con 0x7f748c09f4e0 2024-05-09T10:51:29.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.647+0000 7f74737fe640 1 -- 172.21.15.98:0/1105623489 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f74800039b0 con 0x7f748c09f4e0 2024-05-09T10:51:29.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.647+0000 7f74737fe640 1 --2- 172.21.15.98:0/1105623489 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f747806b6b0 0x7f747806db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.647+0000 7f74737fe640 1 -- 172.21.15.98:0/1105623489 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f748008a000 con 0x7f748c09f4e0 2024-05-09T10:51:29.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.647+0000 7f748ad9d640 1 --2- 172.21.15.98:0/1105623489 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f747806b6b0 0x7f747806db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.648+0000 7f748ad9d640 1 --2- 172.21.15.98:0/1105623489 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f747806b6b0 0x7f747806db70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f747c032000 tx=0x7f747c02d000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:29.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.651+0000 7f74737fe640 1 -- 172.21.15.98:0/1105623489 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f748c0c2c50 con 0x7f748c09f4e0 2024-05-09T10:51:29.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.861+0000 7f7491238640 1 -- 172.21.15.98:0/1105623489 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f748c0c34a0 con 0x7f748c09f4e0 2024-05-09T10:51:29.855 INFO:tasks.ceph.mon.a.smithi098.stderr:2024-05-09T10:51:29.862+0000 7fbec99f0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T10:51:29.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.863+0000 7f74737fe640 1 -- 172.21.15.98:0/1105623489 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v15) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7480058640 con 0x7f748c09f4e0 2024-05-09T10:51:29.856 INFO:teuthology.orchestra.run.smithi098.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T10:51:29.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.866+0000 7f7491238640 1 -- 172.21.15.98:0/1105623489 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f747806b6b0 msgr2=0x7f747806db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.866+0000 7f7491238640 1 --2- 172.21.15.98:0/1105623489 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f747806b6b0 0x7f747806db70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f747c032000 tx=0x7f747c02d000 comp rx=0 tx=0).stop 2024-05-09T10:51:29.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.866+0000 7f7491238640 1 -- 172.21.15.98:0/1105623489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f748c09f4e0 msgr2=0x7f748c09fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.866+0000 7f7491238640 1 --2- 172.21.15.98:0/1105623489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f748c09f4e0 0x7f748c09fea0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f748000f5f0 tx=0x7f748000fab0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.866+0000 7f7491238640 1 -- 172.21.15.98:0/1105623489 shutdown_connections 2024-05-09T10:51:29.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.866+0000 7f7491238640 1 --2- 172.21.15.98:0/1105623489 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f747806b6b0 0x7f747806db70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.866+0000 7f7491238640 1 --2- 172.21.15.98:0/1105623489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f748c09ba60 0x7f748c09bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.866+0000 7f7491238640 1 --2- 172.21.15.98:0/1105623489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f748c09f4e0 0x7f748c09fea0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.866+0000 7f7491238640 1 --2- 172.21.15.98:0/1105623489 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74740039b0 0x7f748c0c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.866+0000 7f7491238640 1 -- 172.21.15.98:0/1105623489 >> 172.21.15.98:0/1105623489 conn(0x7f748c0ccaa0 msgr2=0x7f748c0ce080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:29.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.866+0000 7f7491238640 1 -- 172.21.15.98:0/1105623489 shutdown_connections 2024-05-09T10:51:29.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.867+0000 7f7491238640 1 -- 172.21.15.98:0/1105623489 wait complete. 2024-05-09T10:51:29.871 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T10:51:29.896 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2024-05-09T10:51:29.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.980+0000 7fdf24e1f640 1 Processor -- start 2024-05-09T10:51:29.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.980+0000 7fdf24e1f640 1 -- start start 2024-05-09T10:51:29.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.981+0000 7fdf24e1f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf20162240 0x7fdf200b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.981+0000 7fdf24e1f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf200d0090 con 0x7fdf200cfbb0 2024-05-09T10:51:29.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.981+0000 7fdf24e1f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf200d0370 con 0x7fdf200b2d40 2024-05-09T10:51:29.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.981+0000 7fdf24e1f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf200d0650 con 0x7fdf20162240 2024-05-09T10:51:29.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.981+0000 7fdf1dd9b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fdf200cfbb0 0x7fdf20161b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:47074/0 (socket says 172.21.15.98:47074) 2024-05-09T10:51:29.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.981+0000 7fdf1dd9b640 1 -- 172.21.15.98:0/4061881821 learned_addr learned my addr 172.21.15.98:0/4061881821 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:29.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.981+0000 7fdf1ed9d640 1 --2- 172.21.15.98:0/4061881821 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf20162240 0x7fdf200b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.981+0000 7fdf1d59a640 1 -- 172.21.15.98:0/4061881821 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3923293220 0 0) 0x7fdf200d0090 con 0x7fdf200cfbb0 2024-05-09T10:51:29.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.981+0000 7fdf1d59a640 1 -- 172.21.15.98:0/4061881821 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf08003520 con 0x7fdf200cfbb0 2024-05-09T10:51:29.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.982+0000 7fdf1d59a640 1 -- 172.21.15.98:0/4061881821 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 831779746 0 0) 0x7fdf200d0370 con 0x7fdf200b2d40 2024-05-09T10:51:29.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.982+0000 7fdf1ed9d640 1 -- 172.21.15.98:0/4061881821 >> v1:172.21.15.105:6789/0 conn(0x7fdf200b2d40 legacy=0x7fdf2006c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.982+0000 7fdf1ed9d640 1 -- 172.21.15.98:0/4061881821 >> v1:172.21.15.98:6789/0 conn(0x7fdf200cfbb0 legacy=0x7fdf20161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.982+0000 7fdf1ed9d640 1 -- 172.21.15.98:0/4061881821 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf200d0930 con 0x7fdf20162240 2024-05-09T10:51:29.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.982+0000 7fdf1ed9d640 1 --2- 172.21.15.98:0/4061881821 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf20162240 0x7fdf200b12d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009dc0 tx=0x7fdf1802a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14725788ebd8bf3a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:29.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.983+0000 7fdf1d59a640 1 -- 172.21.15.98:0/4061881821 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1802ae60 con 0x7fdf20162240 2024-05-09T10:51:29.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.983+0000 7fdf1d59a640 1 -- 172.21.15.98:0/4061881821 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf18007740 con 0x7fdf20162240 2024-05-09T10:51:29.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.983+0000 7fdf1d59a640 1 -- 172.21.15.98:0/4061881821 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf18007af0 con 0x7fdf20162240 2024-05-09T10:51:29.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.983+0000 7fdf24e1f640 1 -- 172.21.15.98:0/4061881821 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf20162240 msgr2=0x7fdf200b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.983+0000 7fdf24e1f640 1 --2- 172.21.15.98:0/4061881821 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf20162240 0x7fdf200b12d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009dc0 tx=0x7fdf1802a640 comp rx=0 tx=0).stop 2024-05-09T10:51:29.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.983+0000 7fdf24e1f640 1 -- 172.21.15.98:0/4061881821 shutdown_connections 2024-05-09T10:51:29.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.983+0000 7fdf24e1f640 1 --2- 172.21.15.98:0/4061881821 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf20162240 0x7fdf200b12d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.983+0000 7fdf24e1f640 1 -- 172.21.15.98:0/4061881821 >> 172.21.15.98:0/4061881821 conn(0x7fdf200d1970 msgr2=0x7fdf200d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:29.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.983+0000 7fdf24e1f640 1 -- 172.21.15.98:0/4061881821 shutdown_connections 2024-05-09T10:51:29.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.983+0000 7fdf24e1f640 1 -- 172.21.15.98:0/4061881821 wait complete. 2024-05-09T10:51:29.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.985+0000 7fdf24e1f640 1 Processor -- start 2024-05-09T10:51:29.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.985+0000 7fdf24e1f640 1 -- start start 2024-05-09T10:51:29.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.985+0000 7fdf24e1f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf200b2d40 0x7fdf20145cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.985+0000 7fdf24e1f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf200cfbb0 0x7fdf200b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.985+0000 7fdf24e1f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf20162240 0x7fdf200b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.985+0000 7fdf24e1f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf200b3120 con 0x7fdf20162240 2024-05-09T10:51:29.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.985+0000 7fdf24e1f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2006d020 con 0x7fdf200cfbb0 2024-05-09T10:51:29.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.985+0000 7fdf24e1f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf200b32a0 con 0x7fdf200b2d40 2024-05-09T10:51:29.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.985+0000 7fdf1ed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf20162240 0x7fdf200b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.985+0000 7fdf1ed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf20162240 0x7fdf200b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60308/0 (socket says 172.21.15.98:60308) 2024-05-09T10:51:29.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.985+0000 7fdf1ed9d640 1 -- 172.21.15.98:0/1588086882 learned_addr learned my addr 172.21.15.98:0/1588086882 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:29.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.986+0000 7fdf1e59c640 1 --2- 172.21.15.98:0/1588086882 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf200b2d40 0x7fdf20145cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.986+0000 7fdf1dd9b640 1 --2- 172.21.15.98:0/1588086882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf200cfbb0 0x7fdf200b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.986+0000 7fdf1ed9d640 1 -- 172.21.15.98:0/1588086882 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf200b2d40 msgr2=0x7fdf20145cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.986+0000 7fdf1ed9d640 1 --2- 172.21.15.98:0/1588086882 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf200b2d40 0x7fdf20145cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.986+0000 7fdf1ed9d640 1 -- 172.21.15.98:0/1588086882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf200cfbb0 msgr2=0x7fdf200b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:29.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.986+0000 7fdf1ed9d640 1 --2- 172.21.15.98:0/1588086882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf200cfbb0 0x7fdf200b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:29.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.986+0000 7fdf1ed9d640 1 -- 172.21.15.98:0/1588086882 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf20158ef0 con 0x7fdf20162240 2024-05-09T10:51:29.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.986+0000 7fdf1ed9d640 1 --2- 172.21.15.98:0/1588086882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf20162240 0x7fdf200b1490 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fdf1802ac30 tx=0x7fdf18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:29.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.986+0000 7fdf0effd640 1 -- 172.21.15.98:0/1588086882 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf18040070 con 0x7fdf20162240 2024-05-09T10:51:29.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.986+0000 7fdf0effd640 1 -- 172.21.15.98:0/1588086882 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf18007e20 con 0x7fdf20162240 2024-05-09T10:51:29.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.986+0000 7fdf0effd640 1 -- 172.21.15.98:0/1588086882 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf18009450 con 0x7fdf20162240 2024-05-09T10:51:29.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.986+0000 7fdf24e1f640 1 -- 172.21.15.98:0/1588086882 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf201591d0 con 0x7fdf20162240 2024-05-09T10:51:29.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.986+0000 7fdf24e1f640 1 -- 172.21.15.98:0/1588086882 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf20159790 con 0x7fdf20162240 2024-05-09T10:51:29.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.987+0000 7fdf24e1f640 1 -- 172.21.15.98:0/1588086882 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2009fa40 con 0x7fdf20162240 2024-05-09T10:51:29.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.988+0000 7fdf0effd640 1 -- 172.21.15.98:0/1588086882 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7fdf18039040 con 0x7fdf20162240 2024-05-09T10:51:29.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.988+0000 7fdf0effd640 1 --2- 172.21.15.98:0/1588086882 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fdf0006b6b0 0x7fdf0006db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:29.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.988+0000 7fdf0effd640 1 -- 172.21.15.98:0/1588086882 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fdf180abc40 con 0x7fdf20162240 2024-05-09T10:51:29.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.988+0000 7fdf1e59c640 1 --2- 172.21.15.98:0/1588086882 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fdf0006b6b0 0x7fdf0006db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:29.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.989+0000 7fdf1e59c640 1 --2- 172.21.15.98:0/1588086882 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fdf0006b6b0 0x7fdf0006db70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdf10000f60 tx=0x7fdf10009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:29.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:29.992+0000 7fdf0effd640 1 -- 172.21.15.98:0/1588086882 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf18079ff0 con 0x7fdf20162240 2024-05-09T10:51:30.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:30.201+0000 7fdf24e1f640 1 -- 172.21.15.98:0/1588086882 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdf200727e0 con 0x7fdf20162240 2024-05-09T10:51:30.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:30.864+0000 7fdf0effd640 1 -- 172.21.15.98:0/1588086882 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v16) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdf1807e090 con 0x7fdf20162240 2024-05-09T10:51:30.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:30.867+0000 7fdf24e1f640 1 -- 172.21.15.98:0/1588086882 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fdf0006b6b0 msgr2=0x7fdf0006db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:30.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:30.867+0000 7fdf24e1f640 1 --2- 172.21.15.98:0/1588086882 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fdf0006b6b0 0x7fdf0006db70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdf10000f60 tx=0x7fdf10009040 comp rx=0 tx=0).stop 2024-05-09T10:51:30.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:30.867+0000 7fdf24e1f640 1 -- 172.21.15.98:0/1588086882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf20162240 msgr2=0x7fdf200b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:30.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:30.867+0000 7fdf24e1f640 1 --2- 172.21.15.98:0/1588086882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf20162240 0x7fdf200b1490 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fdf1802ac30 tx=0x7fdf18004a40 comp rx=0 tx=0).stop 2024-05-09T10:51:30.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:30.867+0000 7fdf24e1f640 1 -- 172.21.15.98:0/1588086882 shutdown_connections 2024-05-09T10:51:30.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:30.867+0000 7fdf24e1f640 1 --2- 172.21.15.98:0/1588086882 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fdf0006b6b0 0x7fdf0006db70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:30.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:30.867+0000 7fdf24e1f640 1 --2- 172.21.15.98:0/1588086882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf20162240 0x7fdf200b1490 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:30.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:30.867+0000 7fdf24e1f640 1 --2- 172.21.15.98:0/1588086882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf200cfbb0 0x7fdf200b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:30.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:30.867+0000 7fdf24e1f640 1 --2- 172.21.15.98:0/1588086882 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdf200b2d40 0x7fdf20145cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:30.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:30.867+0000 7fdf24e1f640 1 -- 172.21.15.98:0/1588086882 >> 172.21.15.98:0/1588086882 conn(0x7fdf200d1970 msgr2=0x7fdf200c54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:30.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:30.867+0000 7fdf24e1f640 1 -- 172.21.15.98:0/1588086882 shutdown_connections 2024-05-09T10:51:30.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:30.867+0000 7fdf24e1f640 1 -- 172.21.15.98:0/1588086882 wait complete. 2024-05-09T10:51:30.872 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T10:51:31.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.022+0000 7f9eb8b32640 1 Processor -- start 2024-05-09T10:51:31.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.022+0000 7f9eb8b32640 1 -- start start 2024-05-09T10:51:31.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.023+0000 7f9eb8b32640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9eb40ca530 0x7f9eb415f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:31.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.023+0000 7f9eb8b32640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9eb40cee80 0x7f9eb40c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:31.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.023+0000 7f9eb8b32640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb40ae960 con 0x7f9eb415a8f0 2024-05-09T10:51:31.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.023+0000 7f9eb8b32640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb415f5d0 con 0x7f9eb40cee80 2024-05-09T10:51:31.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.023+0000 7f9eb8b32640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb40cb540 con 0x7f9eb40ca530 2024-05-09T10:51:31.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.023+0000 7f9eb259c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f9eb415a8f0 0x7f9eb40ce770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:47088/0 (socket says 172.21.15.98:47088) 2024-05-09T10:51:31.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.023+0000 7f9eb259c640 1 -- 172.21.15.98:0/1565040675 learned_addr learned my addr 172.21.15.98:0/1565040675 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:31.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb159a640 1 -- 172.21.15.98:0/1565040675 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1731479764 0 0) 0x7f9eb40ae960 con 0x7f9eb415a8f0 2024-05-09T10:51:31.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb159a640 1 -- 172.21.15.98:0/1565040675 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e9c003520 con 0x7f9eb415a8f0 2024-05-09T10:51:31.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb1d9b640 1 --2- 172.21.15.98:0/1565040675 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9eb40ca530 0x7f9eb415f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:31.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb2d9d640 1 --2- 172.21.15.98:0/1565040675 >> v2:172.21.15.105:3300/0 conn(0x7f9eb40cee80 0x7f9eb40c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:31.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb159a640 1 -- 172.21.15.98:0/1565040675 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 118997839 0 0) 0x7f9e9c003520 con 0x7f9eb415a8f0 2024-05-09T10:51:31.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb159a640 1 -- 172.21.15.98:0/1565040675 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9eb40ca530 msgr2=0x7f9eb415f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:31.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb159a640 1 --2- 172.21.15.98:0/1565040675 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9eb40ca530 0x7f9eb415f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:31.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb159a640 1 -- 172.21.15.98:0/1565040675 >> v2:172.21.15.105:3300/0 conn(0x7f9eb40cee80 msgr2=0x7f9eb40c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:31.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb159a640 1 --2- 172.21.15.98:0/1565040675 >> v2:172.21.15.105:3300/0 conn(0x7f9eb40cee80 0x7f9eb40c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:31.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb159a640 1 -- 172.21.15.98:0/1565040675 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb40ca9c0 con 0x7f9eb415a8f0 2024-05-09T10:51:31.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb159a640 1 -- 172.21.15.98:0/1565040675 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f9ea40019f0 con 0x7f9eb415a8f0 2024-05-09T10:51:31.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb159a640 1 -- 172.21.15.98:0/1565040675 >> v1:172.21.15.98:6789/0 conn(0x7f9eb415a8f0 legacy=0x7f9eb40ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:31.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb159a640 1 --2- 172.21.15.98:0/1565040675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e9c0039b0 0x7f9e9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:31.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb159a640 1 --2- 172.21.15.98:0/1565040675 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e9c004800 0x7f9e9c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:31.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb159a640 1 --2- 172.21.15.98:0/1565040675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e9c005180 0x7f9e9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:31.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb159a640 1 -- 172.21.15.98:0/1565040675 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb415f5d0 con 0x7f9e9c005180 2024-05-09T10:51:31.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb159a640 1 -- 172.21.15.98:0/1565040675 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb40cb540 con 0x7f9e9c0039b0 2024-05-09T10:51:31.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb159a640 1 -- 172.21.15.98:0/1565040675 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c004600 con 0x7f9e9c004800 2024-05-09T10:51:31.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb2d9d640 1 --2- 172.21.15.98:0/1565040675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e9c005180 0x7f9e9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:31.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb259c640 1 --2- 172.21.15.98:0/1565040675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e9c0039b0 0x7f9e9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:31.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb1d9b640 1 --2- 172.21.15.98:0/1565040675 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e9c004800 0x7f9e9c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:31.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb2d9d640 1 -- 172.21.15.98:0/1565040675 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e9c004800 msgr2=0x7f9e9c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:31.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb2d9d640 1 --2- 172.21.15.98:0/1565040675 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e9c004800 0x7f9e9c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:31.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb2d9d640 1 -- 172.21.15.98:0/1565040675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e9c0039b0 msgr2=0x7f9e9c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:31.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb2d9d640 1 --2- 172.21.15.98:0/1565040675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e9c0039b0 0x7f9e9c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:31.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb2d9d640 1 -- 172.21.15.98:0/1565040675 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9c003520 con 0x7f9e9c005180 2024-05-09T10:51:31.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb1d9b640 1 --2- 172.21.15.98:0/1565040675 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9e9c004800 0x7f9e9c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:31.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.024+0000 7f9eb259c640 1 --2- 172.21.15.98:0/1565040675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e9c0039b0 0x7f9e9c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:31.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.025+0000 7f9eb2d9d640 1 --2- 172.21.15.98:0/1565040675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e9c005180 0x7f9e9c00b9c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009a90 tx=0x7f9e9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deda124a0281bf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:31.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.025+0000 7f9eb159a640 1 -- 172.21.15.98:0/1565040675 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac036070 con 0x7f9e9c005180 2024-05-09T10:51:31.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.025+0000 7f9eb159a640 1 -- 172.21.15.98:0/1565040675 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eac031400 con 0x7f9e9c005180 2024-05-09T10:51:31.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.025+0000 7f9eb159a640 1 -- 172.21.15.98:0/1565040675 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac009690 con 0x7f9e9c005180 2024-05-09T10:51:31.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.026+0000 7f9eb8b32640 1 -- 172.21.15.98:0/1565040675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e9c005180 msgr2=0x7f9e9c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:31.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.026+0000 7f9eb8b32640 1 --2- 172.21.15.98:0/1565040675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e9c005180 0x7f9e9c00b9c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009a90 tx=0x7f9e9c005830 comp rx=0 tx=0).stop 2024-05-09T10:51:31.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.026+0000 7f9eb2d9d640 1 -- 172.21.15.98:0/1565040675 reap_dead start 2024-05-09T10:51:31.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.026+0000 7f9eb8b32640 1 -- 172.21.15.98:0/1565040675 shutdown_connections 2024-05-09T10:51:31.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.026+0000 7f9eb8b32640 1 -- 172.21.15.98:0/1565040675 >> 172.21.15.98:0/1565040675 conn(0x7f9eb40cf760 msgr2=0x7f9eb40cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:31.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.026+0000 7f9eb8b32640 1 -- 172.21.15.98:0/1565040675 shutdown_connections 2024-05-09T10:51:31.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.026+0000 7f9eb8b32640 1 -- 172.21.15.98:0/1565040675 wait complete. 2024-05-09T10:51:31.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.027+0000 7f9eb8b32640 1 Processor -- start 2024-05-09T10:51:31.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.027+0000 7f9eb8b32640 1 -- start start 2024-05-09T10:51:31.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.028+0000 7f9eb8b32640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e9c005180 0x7f9eb41e4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:31.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.028+0000 7f9eb8b32640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9eb40afe60 0x7f9eb40b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:31.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.028+0000 7f9eb8b32640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9eb415b3e0 0x7f9eb415b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:31.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.028+0000 7f9eb8b32640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb40cb260 con 0x7f9eb40afe60 2024-05-09T10:51:31.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.028+0000 7f9eb8b32640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb40caf80 con 0x7f9e9c005180 2024-05-09T10:51:31.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.028+0000 7f9eb8b32640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb415acd0 con 0x7f9eb415b3e0 2024-05-09T10:51:31.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.028+0000 7f9eb1d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9eb40afe60 0x7f9eb40b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:31.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.028+0000 7f9eb1d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9eb40afe60 0x7f9eb40b02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60324/0 (socket says 172.21.15.98:60324) 2024-05-09T10:51:31.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.028+0000 7f9eb1d9b640 1 -- 172.21.15.98:0/2608959251 learned_addr learned my addr 172.21.15.98:0/2608959251 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:31.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.028+0000 7f9eb2d9d640 1 --2- 172.21.15.98:0/2608959251 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9eb415b3e0 0x7f9eb415b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:31.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.028+0000 7f9eb259c640 1 --2- 172.21.15.98:0/2608959251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e9c005180 0x7f9eb41e4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:31.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.028+0000 7f9eb1d9b640 1 -- 172.21.15.98:0/2608959251 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9eb415b3e0 msgr2=0x7f9eb415b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:31.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.028+0000 7f9eb1d9b640 1 --2- 172.21.15.98:0/2608959251 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9eb415b3e0 0x7f9eb415b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:31.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.028+0000 7f9eb1d9b640 1 -- 172.21.15.98:0/2608959251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e9c005180 msgr2=0x7f9eb41e4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:31.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.028+0000 7f9eb1d9b640 1 --2- 172.21.15.98:0/2608959251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e9c005180 0x7f9eb41e4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:31.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.028+0000 7f9eb1d9b640 1 -- 172.21.15.98:0/2608959251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb415c110 con 0x7f9eb40afe60 2024-05-09T10:51:31.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.028+0000 7f9eb1d9b640 1 --2- 172.21.15.98:0/2608959251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9eb40afe60 0x7f9eb40b02c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8011630 tx=0x7f9ea8011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:31.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.028+0000 7f9ea2ffd640 1 -- 172.21.15.98:0/2608959251 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea800f920 con 0x7f9eb40afe60 2024-05-09T10:51:31.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.028+0000 7f9ea2ffd640 1 -- 172.21.15.98:0/2608959251 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea800fac0 con 0x7f9eb40afe60 2024-05-09T10:51:31.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.028+0000 7f9eb8b32640 1 -- 172.21.15.98:0/2608959251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb415c3f0 con 0x7f9eb40afe60 2024-05-09T10:51:31.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.028+0000 7f9ea2ffd640 1 -- 172.21.15.98:0/2608959251 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea800fe90 con 0x7f9eb40afe60 2024-05-09T10:51:31.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.029+0000 7f9eb8b32640 1 -- 172.21.15.98:0/2608959251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb415c9b0 con 0x7f9eb40afe60 2024-05-09T10:51:31.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.029+0000 7f9eb8b32640 1 -- 172.21.15.98:0/2608959251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb4072c80 con 0x7f9eb40afe60 2024-05-09T10:51:31.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.030+0000 7f9ea2ffd640 1 -- 172.21.15.98:0/2608959251 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f9ea8015460 con 0x7f9eb40afe60 2024-05-09T10:51:31.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.031+0000 7f9ea2ffd640 1 --2- 172.21.15.98:0/2608959251 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9e9406b820 0x7f9e9406dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:31.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.031+0000 7f9ea2ffd640 1 -- 172.21.15.98:0/2608959251 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9ea808b2b0 con 0x7f9eb40afe60 2024-05-09T10:51:31.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.031+0000 7f9eb259c640 1 --2- 172.21.15.98:0/2608959251 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9e9406b820 0x7f9e9406dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:31.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.031+0000 7f9eb259c640 1 --2- 172.21.15.98:0/2608959251 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9e9406b820 0x7f9e9406dce0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4005e30 tx=0x7f9ea4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:31.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.034+0000 7f9ea2ffd640 1 -- 172.21.15.98:0/2608959251 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9ea8019030 con 0x7f9eb40afe60 2024-05-09T10:51:31.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.203+0000 7f9eb8b32640 1 -- 172.21.15.98:0/2608959251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9eb404f000 con 0x7f9eb40afe60 2024-05-09T10:51:31.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.869+0000 7f9ea2ffd640 1 -- 172.21.15.98:0/2608959251 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v33) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9ea8059f30 con 0x7f9eb40afe60 2024-05-09T10:51:31.863 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' removed 2024-05-09T10:51:31.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.872+0000 7f9eb8b32640 1 -- 172.21.15.98:0/2608959251 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9e9406b820 msgr2=0x7f9e9406dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:31.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.872+0000 7f9eb8b32640 1 --2- 172.21.15.98:0/2608959251 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9e9406b820 0x7f9e9406dce0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4005e30 tx=0x7f9ea4009040 comp rx=0 tx=0).stop 2024-05-09T10:51:31.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.872+0000 7f9eb8b32640 1 -- 172.21.15.98:0/2608959251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9eb40afe60 msgr2=0x7f9eb40b02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:31.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.872+0000 7f9eb8b32640 1 --2- 172.21.15.98:0/2608959251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9eb40afe60 0x7f9eb40b02c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8011630 tx=0x7f9ea8011af0 comp rx=0 tx=0).stop 2024-05-09T10:51:31.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.872+0000 7f9eb8b32640 1 -- 172.21.15.98:0/2608959251 shutdown_connections 2024-05-09T10:51:31.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.872+0000 7f9eb8b32640 1 --2- 172.21.15.98:0/2608959251 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9e9406b820 0x7f9e9406dce0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:31.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.872+0000 7f9eb8b32640 1 --2- 172.21.15.98:0/2608959251 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9eb415b3e0 0x7f9eb415b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:31.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.872+0000 7f9eb8b32640 1 --2- 172.21.15.98:0/2608959251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9eb40afe60 0x7f9eb40b02c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:31.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.872+0000 7f9eb8b32640 1 --2- 172.21.15.98:0/2608959251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e9c005180 0x7f9eb41e4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:31.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.872+0000 7f9eb8b32640 1 -- 172.21.15.98:0/2608959251 >> 172.21.15.98:0/2608959251 conn(0x7f9eb40cf760 msgr2=0x7f9eb40701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:31.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.872+0000 7f9eb8b32640 1 -- 172.21.15.98:0/2608959251 shutdown_connections 2024-05-09T10:51:31.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.872+0000 7f9eb8b32640 1 -- 172.21.15.98:0/2608959251 wait complete. 2024-05-09T10:51:31.880 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T10:51:31.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.991+0000 7fa01001f640 1 Processor -- start 2024-05-09T10:51:31.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.991+0000 7fa01001f640 1 -- start start 2024-05-09T10:51:31.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.992+0000 7fa01001f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa008159740 0x7fa0080c6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:31.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.992+0000 7fa01001f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa0080aede0 0x7fa0080c4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:31.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.992+0000 7fa01001f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa00806af30 con 0x7fa00815c9a0 2024-05-09T10:51:31.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.992+0000 7fa01001f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa00806b210 con 0x7fa0080aede0 2024-05-09T10:51:31.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.992+0000 7fa01001f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00806b4f0 con 0x7fa008159740 2024-05-09T10:51:31.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.992+0000 7fa00e5bc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa0080aede0 0x7fa0080c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:31.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.992+0000 7fa00e5bc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa0080aede0 0x7fa0080c4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60326/0 (socket says 172.21.15.98:60326) 2024-05-09T10:51:31.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.992+0000 7fa00e5bc640 1 -- 172.21.15.98:0/2980689928 learned_addr learned my addr 172.21.15.98:0/2980689928 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:31.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.992+0000 7fa00ddbb640 1 --2- 172.21.15.98:0/2980689928 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa008159740 0x7fa0080c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:31.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.992+0000 7fa00e5bc640 1 -- 172.21.15.98:0/2980689928 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa008159740 msgr2=0x7fa0080c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:31.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.992+0000 7fa00e5bc640 1 --2- 172.21.15.98:0/2980689928 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa008159740 0x7fa0080c6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:31.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.992+0000 7fa00e5bc640 1 -- 172.21.15.98:0/2980689928 >> v1:172.21.15.105:6789/0 conn(0x7fa00815c9a0 legacy=0x7fa0080ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:31.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.992+0000 7fa00e5bc640 1 -- 172.21.15.98:0/2980689928 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00815c480 con 0x7fa0080aede0 2024-05-09T10:51:31.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.993+0000 7fa00e5bc640 1 --2- 172.21.15.98:0/2980689928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0080aede0 0x7fa0080c4620 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa000009e50 tx=0x7fa00002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc98b90feadc2164 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:31.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.993+0000 7fa00cdb9640 1 -- 172.21.15.98:0/2980689928 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa000036070 con 0x7fa0080aede0 2024-05-09T10:51:31.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.993+0000 7fa00cdb9640 1 -- 172.21.15.98:0/2980689928 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa000031600 con 0x7fa0080aede0 2024-05-09T10:51:31.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.993+0000 7fa00cdb9640 1 -- 172.21.15.98:0/2980689928 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0000319d0 con 0x7fa0080aede0 2024-05-09T10:51:31.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.994+0000 7fa01001f640 1 -- 172.21.15.98:0/2980689928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0080aede0 msgr2=0x7fa0080c4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:31.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.994+0000 7fa01001f640 1 --2- 172.21.15.98:0/2980689928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0080aede0 0x7fa0080c4620 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa000009e50 tx=0x7fa00002a6c0 comp rx=0 tx=0).stop 2024-05-09T10:51:31.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.994+0000 7fa01001f640 1 -- 172.21.15.98:0/2980689928 shutdown_connections 2024-05-09T10:51:31.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.994+0000 7fa01001f640 1 --2- 172.21.15.98:0/2980689928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0080aede0 0x7fa0080c4620 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:31.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.994+0000 7fa01001f640 1 --2- 172.21.15.98:0/2980689928 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa008159740 0x7fa0080c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:31.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.994+0000 7fa01001f640 1 -- 172.21.15.98:0/2980689928 >> 172.21.15.98:0/2980689928 conn(0x7fa0080cf760 msgr2=0x7fa0080cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:31.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.994+0000 7fa01001f640 1 -- 172.21.15.98:0/2980689928 shutdown_connections 2024-05-09T10:51:31.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.994+0000 7fa01001f640 1 -- 172.21.15.98:0/2980689928 wait complete. 2024-05-09T10:51:31.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.995+0000 7fa01001f640 1 Processor -- start 2024-05-09T10:51:31.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.995+0000 7fa01001f640 1 -- start start 2024-05-09T10:51:31.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.996+0000 7fa01001f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0080aede0 0x7fa00814e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:31.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.996+0000 7fa01001f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa008159740 0x7fa0080ad880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:31.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.996+0000 7fa01001f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa00815c9a0 0x7fa0080addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:31.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.996+0000 7fa01001f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00815c170 con 0x7fa00815c9a0 2024-05-09T10:51:31.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.996+0000 7fa01001f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa008159b20 con 0x7fa008159740 2024-05-09T10:51:31.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.996+0000 7fa01001f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00806abe0 con 0x7fa0080aede0 2024-05-09T10:51:31.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.996+0000 7fa00e5bc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa00815c9a0 0x7fa0080addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:31.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.996+0000 7fa00e5bc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa00815c9a0 0x7fa0080addc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60342/0 (socket says 172.21.15.98:60342) 2024-05-09T10:51:31.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.996+0000 7fa00e5bc640 1 -- 172.21.15.98:0/1985425668 learned_addr learned my addr 172.21.15.98:0/1985425668 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:31.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.996+0000 7fa00ddbb640 1 --2- 172.21.15.98:0/1985425668 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0080aede0 0x7fa00814e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:31.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.996+0000 7fa00d5ba640 1 --2- 172.21.15.98:0/1985425668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa008159740 0x7fa0080ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:31.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.996+0000 7fa00e5bc640 1 -- 172.21.15.98:0/1985425668 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0080aede0 msgr2=0x7fa00814e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:31.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.996+0000 7fa00e5bc640 1 --2- 172.21.15.98:0/1985425668 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0080aede0 0x7fa00814e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:31.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.996+0000 7fa00e5bc640 1 -- 172.21.15.98:0/1985425668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa008159740 msgr2=0x7fa0080ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:31.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.996+0000 7fa00e5bc640 1 --2- 172.21.15.98:0/1985425668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa008159740 0x7fa0080ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:31.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.996+0000 7fa00e5bc640 1 -- 172.21.15.98:0/1985425668 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00815e840 con 0x7fa00815c9a0 2024-05-09T10:51:31.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.996+0000 7fa00e5bc640 1 --2- 172.21.15.98:0/1985425668 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa00815c9a0 0x7fa0080addc0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa000009e50 tx=0x7fa000004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:31.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.996+0000 7f9ffe7fc640 1 -- 172.21.15.98:0/1985425668 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa000036050 con 0x7fa00815c9a0 2024-05-09T10:51:31.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.996+0000 7f9ffe7fc640 1 -- 172.21.15.98:0/1985425668 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa000042070 con 0x7fa00815c9a0 2024-05-09T10:51:31.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.996+0000 7fa01001f640 1 -- 172.21.15.98:0/1985425668 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa00815eb20 con 0x7fa00815c9a0 2024-05-09T10:51:31.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.996+0000 7f9ffe7fc640 1 -- 172.21.15.98:0/1985425668 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa000009510 con 0x7fa00815c9a0 2024-05-09T10:51:31.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.996+0000 7fa01001f640 1 -- 172.21.15.98:0/1985425668 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa008157720 con 0x7fa00815c9a0 2024-05-09T10:51:31.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.997+0000 7fa01001f640 1 -- 172.21.15.98:0/1985425668 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa00815ecf0 con 0x7fa00815c9a0 2024-05-09T10:51:31.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.998+0000 7f9ffe7fc640 1 -- 172.21.15.98:0/1985425668 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7fa000007bd0 con 0x7fa00815c9a0 2024-05-09T10:51:31.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.998+0000 7f9ffe7fc640 1 --2- 172.21.15.98:0/1985425668 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9fec06b680 0x7f9fec06db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:31.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.998+0000 7f9ffe7fc640 1 -- 172.21.15.98:0/1985425668 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fa0000ab310 con 0x7fa00815c9a0 2024-05-09T10:51:31.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.998+0000 7fa00ddbb640 1 --2- 172.21.15.98:0/1985425668 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9fec06b680 0x7f9fec06db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:31.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:31.999+0000 7fa00ddbb640 1 --2- 172.21.15.98:0/1985425668 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9fec06b680 0x7f9fec06db40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa00400aa20 tx=0x7fa004008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:31.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.003+0000 7f9ffe7fc640 1 -- 172.21.15.98:0/1985425668 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa000079910 con 0x7fa00815c9a0 2024-05-09T10:51:32.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.173+0000 7fa01001f640 1 -- 172.21.15.98:0/1985425668 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa008072c80 con 0x7fa00815c9a0 2024-05-09T10:51:32.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.874+0000 7f9ffe7fc640 1 -- 172.21.15.98:0/1985425668 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v34) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa00007d9b0 con 0x7fa00815c9a0 2024-05-09T10:51:32.867 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' removed 2024-05-09T10:51:32.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.879+0000 7fa01001f640 1 -- 172.21.15.98:0/1985425668 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9fec06b680 msgr2=0x7f9fec06db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:32.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.879+0000 7fa01001f640 1 --2- 172.21.15.98:0/1985425668 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9fec06b680 0x7f9fec06db40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa00400aa20 tx=0x7fa004008040 comp rx=0 tx=0).stop 2024-05-09T10:51:32.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.879+0000 7fa01001f640 1 -- 172.21.15.98:0/1985425668 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa00815c9a0 msgr2=0x7fa0080addc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:32.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.879+0000 7fa01001f640 1 --2- 172.21.15.98:0/1985425668 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa00815c9a0 0x7fa0080addc0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa000009e50 tx=0x7fa000004a40 comp rx=0 tx=0).stop 2024-05-09T10:51:32.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.879+0000 7fa01001f640 1 -- 172.21.15.98:0/1985425668 shutdown_connections 2024-05-09T10:51:32.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.879+0000 7fa01001f640 1 --2- 172.21.15.98:0/1985425668 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f9fec06b680 0x7f9fec06db40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:32.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.879+0000 7fa01001f640 1 --2- 172.21.15.98:0/1985425668 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa00815c9a0 0x7fa0080addc0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:32.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.879+0000 7fa01001f640 1 --2- 172.21.15.98:0/1985425668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa008159740 0x7fa0080ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:32.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.879+0000 7fa01001f640 1 --2- 172.21.15.98:0/1985425668 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0080aede0 0x7fa00814e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:32.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.879+0000 7fa01001f640 1 -- 172.21.15.98:0/1985425668 >> 172.21.15.98:0/1985425668 conn(0x7fa0080cf760 msgr2=0x7fa008155d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:32.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.880+0000 7fa01001f640 1 -- 172.21.15.98:0/1985425668 shutdown_connections 2024-05-09T10:51:32.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.880+0000 7fa01001f640 1 -- 172.21.15.98:0/1985425668 wait complete. 2024-05-09T10:51:32.886 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-09T10:51:32.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.996+0000 7f244069a640 1 Processor -- start 2024-05-09T10:51:32.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.996+0000 7f244069a640 1 -- start start 2024-05-09T10:51:32.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.997+0000 7f244069a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24380d2840 0x7f24380d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:32.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.997+0000 7f244069a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f24380d4670 0x7f24380b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:32.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.997+0000 7f244069a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24380d1e60 con 0x7f243815def0 2024-05-09T10:51:32.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.997+0000 7f244069a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24380d1930 con 0x7f24380d4670 2024-05-09T10:51:32.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.997+0000 7f244069a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24380d1b80 con 0x7f24380d2840 2024-05-09T10:51:32.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.997+0000 7f243ec37640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f24380d4670 0x7f24380b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:32.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.997+0000 7f243e436640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f243815def0 0x7f24380d2130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:47102/0 (socket says 172.21.15.98:47102) 2024-05-09T10:51:32.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.997+0000 7f243e436640 1 -- 172.21.15.98:0/4262185438 learned_addr learned my addr 172.21.15.98:0/4262185438 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:32.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.997+0000 7f243dc35640 1 --2- 172.21.15.98:0/4262185438 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24380d2840 0x7f24380d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:32.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.997+0000 7f243ec37640 1 -- 172.21.15.98:0/4262185438 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24380d2840 msgr2=0x7f24380d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:32.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.997+0000 7f243ec37640 1 --2- 172.21.15.98:0/4262185438 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24380d2840 0x7f24380d0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:32.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.997+0000 7f243ec37640 1 -- 172.21.15.98:0/4262185438 >> v1:172.21.15.98:6789/0 conn(0x7f243815def0 legacy=0x7f24380d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:32.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.997+0000 7f243ec37640 1 -- 172.21.15.98:0/4262185438 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24380b3550 con 0x7f24380d4670 2024-05-09T10:51:32.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.998+0000 7f243ec37640 1 --2- 172.21.15.98:0/4262185438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24380d4670 0x7f24380b3d50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2430009d90 tx=0x7f243002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84707191d2da908 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:32.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.998+0000 7f243d434640 1 -- 172.21.15.98:0/4262185438 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430007740 con 0x7f24380d4670 2024-05-09T10:51:32.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.998+0000 7f243d434640 1 -- 172.21.15.98:0/4262185438 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24300078e0 con 0x7f24380d4670 2024-05-09T10:51:32.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.998+0000 7f243d434640 1 -- 172.21.15.98:0/4262185438 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430007cb0 con 0x7f24380d4670 2024-05-09T10:51:32.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.999+0000 7f244069a640 1 -- 172.21.15.98:0/4262185438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24380d4670 msgr2=0x7f24380b3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:32.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.999+0000 7f244069a640 1 --2- 172.21.15.98:0/4262185438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24380d4670 0x7f24380b3d50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2430009d90 tx=0x7f243002a6f0 comp rx=0 tx=0).stop 2024-05-09T10:51:32.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.999+0000 7f244069a640 1 -- 172.21.15.98:0/4262185438 shutdown_connections 2024-05-09T10:51:32.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.999+0000 7f244069a640 1 --2- 172.21.15.98:0/4262185438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24380d4670 0x7f24380b3d50 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:32.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.999+0000 7f244069a640 1 --2- 172.21.15.98:0/4262185438 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f24380d2840 0x7f24380d0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:32.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.999+0000 7f244069a640 1 -- 172.21.15.98:0/4262185438 >> 172.21.15.98:0/4262185438 conn(0x7f24380d3b50 msgr2=0x7f24380d3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:32.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.999+0000 7f244069a640 1 -- 172.21.15.98:0/4262185438 shutdown_connections 2024-05-09T10:51:32.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:32.999+0000 7f244069a640 1 -- 172.21.15.98:0/4262185438 wait complete. 2024-05-09T10:51:32.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.000+0000 7f244069a640 1 Processor -- start 2024-05-09T10:51:32.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.000+0000 7f244069a640 1 -- start start 2024-05-09T10:51:32.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.001+0000 7f244069a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24380d2840 0x7f24381f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:32.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.001+0000 7f244069a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24380d4670 0x7f243815a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:32.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.001+0000 7f244069a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f243815def0 0x7f243815ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:32.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.001+0000 7f244069a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243815e2d0 con 0x7f24380d2840 2024-05-09T10:51:32.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.001+0000 7f244069a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24380cfb00 con 0x7f24380d4670 2024-05-09T10:51:32.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.001+0000 7f244069a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24380b4290 con 0x7f243815def0 2024-05-09T10:51:32.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.001+0000 7f243e436640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24380d2840 0x7f24381f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:32.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.001+0000 7f243e436640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24380d2840 0x7f24381f43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60358/0 (socket says 172.21.15.98:60358) 2024-05-09T10:51:32.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.001+0000 7f243e436640 1 -- 172.21.15.98:0/1816194708 learned_addr learned my addr 172.21.15.98:0/1816194708 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:32.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.001+0000 7f243dc35640 1 --2- 172.21.15.98:0/1816194708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24380d4670 0x7f243815a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:32.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.001+0000 7f243ec37640 1 --2- 172.21.15.98:0/1816194708 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f243815def0 0x7f243815ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:32.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.001+0000 7f243e436640 1 -- 172.21.15.98:0/1816194708 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f243815def0 msgr2=0x7f243815ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:32.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.001+0000 7f243e436640 1 --2- 172.21.15.98:0/1816194708 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f243815def0 0x7f243815ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:32.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.001+0000 7f243e436640 1 -- 172.21.15.98:0/1816194708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24380d4670 msgr2=0x7f243815a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:32.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.001+0000 7f243e436640 1 --2- 172.21.15.98:0/1816194708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24380d4670 0x7f243815a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:32.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.001+0000 7f243e436640 1 -- 172.21.15.98:0/1816194708 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24381f4930 con 0x7f24380d2840 2024-05-09T10:51:32.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.001+0000 7f243dc35640 1 --2- 172.21.15.98:0/1816194708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24380d4670 0x7f243815a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:32.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.001+0000 7f243e436640 1 --2- 172.21.15.98:0/1816194708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24380d2840 0x7f24381f43f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f243400c3f0 tx=0x7f243400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:32.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.001+0000 7f242effd640 1 -- 172.21.15.98:0/1816194708 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434017070 con 0x7f24380d2840 2024-05-09T10:51:32.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.001+0000 7f242effd640 1 -- 172.21.15.98:0/1816194708 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2434005ce0 con 0x7f24380d2840 2024-05-09T10:51:32.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.001+0000 7f244069a640 1 -- 172.21.15.98:0/1816194708 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2438158ca0 con 0x7f24380d2840 2024-05-09T10:51:32.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.001+0000 7f242effd640 1 -- 172.21.15.98:0/1816194708 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24340072f0 con 0x7f24380d2840 2024-05-09T10:51:32.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.001+0000 7f244069a640 1 -- 172.21.15.98:0/1816194708 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2438159260 con 0x7f24380d2840 2024-05-09T10:51:32.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.002+0000 7f244069a640 1 -- 172.21.15.98:0/1816194708 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24380a1b60 con 0x7f24380d2840 2024-05-09T10:51:32.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.003+0000 7f242effd640 1 -- 172.21.15.98:0/1816194708 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f243401a070 con 0x7f24380d2840 2024-05-09T10:51:32.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.003+0000 7f242effd640 1 --2- 172.21.15.98:0/1816194708 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f241c06b6b0 0x7f241c06db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:32.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.003+0000 7f242effd640 1 -- 172.21.15.98:0/1816194708 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f24340895b0 con 0x7f24380d2840 2024-05-09T10:51:32.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.003+0000 7f243dc35640 1 --2- 172.21.15.98:0/1816194708 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f241c06b6b0 0x7f241c06db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:32.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.004+0000 7f243dc35640 1 --2- 172.21.15.98:0/1816194708 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f241c06b6b0 0x7f241c06db70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f24380b0ae0 tx=0x7f2428008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:33.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.007+0000 7f242effd640 1 -- 172.21.15.98:0/1816194708 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f24340584b0 con 0x7f24380d2840 2024-05-09T10:51:33.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.176+0000 7f244069a640 1 -- 172.21.15.98:0/1816194708 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f243815a6a0 con 0x7f24380d2840 2024-05-09T10:51:33.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.179+0000 7f242effd640 1 -- 172.21.15.98:0/1816194708 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f243405ce20 con 0x7f24380d2840 2024-05-09T10:51:33.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.181+0000 7f244069a640 1 -- 172.21.15.98:0/1816194708 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f241c06b6b0 msgr2=0x7f241c06db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:33.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.181+0000 7f244069a640 1 --2- 172.21.15.98:0/1816194708 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f241c06b6b0 0x7f241c06db70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f24380b0ae0 tx=0x7f2428008040 comp rx=0 tx=0).stop 2024-05-09T10:51:33.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.181+0000 7f244069a640 1 -- 172.21.15.98:0/1816194708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24380d2840 msgr2=0x7f24381f43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:33.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.181+0000 7f244069a640 1 --2- 172.21.15.98:0/1816194708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24380d2840 0x7f24381f43f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f243400c3f0 tx=0x7f243400c8b0 comp rx=0 tx=0).stop 2024-05-09T10:51:33.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.181+0000 7f244069a640 1 -- 172.21.15.98:0/1816194708 shutdown_connections 2024-05-09T10:51:33.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.181+0000 7f244069a640 1 --2- 172.21.15.98:0/1816194708 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f241c06b6b0 0x7f241c06db70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:33.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.181+0000 7f244069a640 1 --2- 172.21.15.98:0/1816194708 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f243815def0 0x7f243815ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:33.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.181+0000 7f244069a640 1 --2- 172.21.15.98:0/1816194708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24380d4670 0x7f243815a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:33.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.181+0000 7f244069a640 1 --2- 172.21.15.98:0/1816194708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24380d2840 0x7f24381f43f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:33.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.181+0000 7f244069a640 1 -- 172.21.15.98:0/1816194708 >> 172.21.15.98:0/1816194708 conn(0x7f24380d3b50 msgr2=0x7f24380d2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:33.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.181+0000 7f244069a640 1 -- 172.21.15.98:0/1816194708 shutdown_connections 2024-05-09T10:51:33.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.181+0000 7f244069a640 1 -- 172.21.15.98:0/1816194708 wait complete. 2024-05-09T10:51:33.185 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread' 2024-05-09T10:51:33.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.292+0000 7f817609a640 1 Processor -- start 2024-05-09T10:51:33.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.292+0000 7f817609a640 1 -- start start 2024-05-09T10:51:33.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.293+0000 7f817609a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8170156180 0x7f817006ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:33.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.293+0000 7f817609a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f817015d090 con 0x7f817006abe0 2024-05-09T10:51:33.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.293+0000 7f817609a640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8170156560 con 0x7f817006b060 2024-05-09T10:51:33.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.293+0000 7f817609a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817006d940 con 0x7f8170156180 2024-05-09T10:51:33.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.293+0000 7f816effd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f817006abe0 0x7f81700cd4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:47114/0 (socket says 172.21.15.98:47114) 2024-05-09T10:51:33.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.293+0000 7f816effd640 1 -- 172.21.15.98:0/2530428973 learned_addr learned my addr 172.21.15.98:0/2530428973 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:33.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.293+0000 7f816f7fe640 1 --2- 172.21.15.98:0/2530428973 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8170156180 0x7f817006ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:33.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.293+0000 7f816e7fc640 1 -- 172.21.15.98:0/2530428973 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135647693 0 0) 0x7f817015d090 con 0x7f817006abe0 2024-05-09T10:51:33.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.294+0000 7f816e7fc640 1 -- 172.21.15.98:0/2530428973 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8158003520 con 0x7f817006abe0 2024-05-09T10:51:33.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.294+0000 7f816e7fc640 1 -- 172.21.15.98:0/2530428973 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3660214866 0 0) 0x7f8170156560 con 0x7f817006b060 2024-05-09T10:51:33.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.294+0000 7f816f7fe640 1 -- 172.21.15.98:0/2530428973 >> v1:172.21.15.105:6789/0 conn(0x7f817006b060 legacy=0x7f81700cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:33.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.294+0000 7f816f7fe640 1 -- 172.21.15.98:0/2530428973 >> v1:172.21.15.98:6789/0 conn(0x7f817006abe0 legacy=0x7f81700cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:33.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.294+0000 7f816f7fe640 1 -- 172.21.15.98:0/2530428973 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817015c510 con 0x7f8170156180 2024-05-09T10:51:33.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.294+0000 7f816f7fe640 1 --2- 172.21.15.98:0/2530428973 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8170156180 0x7f817006ddf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f81600097d0 tx=0x7f816002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29b02afb32eb8b9d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:33.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.295+0000 7f816e7fc640 1 -- 172.21.15.98:0/2530428973 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8160037070 con 0x7f8170156180 2024-05-09T10:51:33.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.295+0000 7f816e7fc640 1 -- 172.21.15.98:0/2530428973 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816002ac10 con 0x7f8170156180 2024-05-09T10:51:33.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.295+0000 7f816e7fc640 1 -- 172.21.15.98:0/2530428973 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81640019f0 con 0x7f8170156180 2024-05-09T10:51:33.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.295+0000 7f817609a640 1 -- 172.21.15.98:0/2530428973 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8170156180 msgr2=0x7f817006ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:33.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.295+0000 7f817609a640 1 --2- 172.21.15.98:0/2530428973 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8170156180 0x7f817006ddf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f81600097d0 tx=0x7f816002a180 comp rx=0 tx=0).stop 2024-05-09T10:51:33.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.295+0000 7f817609a640 1 -- 172.21.15.98:0/2530428973 shutdown_connections 2024-05-09T10:51:33.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.295+0000 7f817609a640 1 --2- 172.21.15.98:0/2530428973 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8170156180 0x7f817006ddf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:33.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.295+0000 7f817609a640 1 -- 172.21.15.98:0/2530428973 >> 172.21.15.98:0/2530428973 conn(0x7f8170066f20 msgr2=0x7f817006bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:33.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.295+0000 7f817609a640 1 -- 172.21.15.98:0/2530428973 shutdown_connections 2024-05-09T10:51:33.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.295+0000 7f817609a640 1 -- 172.21.15.98:0/2530428973 wait complete. 2024-05-09T10:51:33.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.296+0000 7f817609a640 1 Processor -- start 2024-05-09T10:51:33.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.297+0000 7f817609a640 1 -- start start 2024-05-09T10:51:33.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.297+0000 7f817609a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f817006abe0 0x7f817014d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:33.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.297+0000 7f817609a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f817006b060 0x7f817014e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:33.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.297+0000 7f817609a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8170156180 0x7f817014e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:33.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.297+0000 7f817609a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817015cad0 con 0x7f8170156180 2024-05-09T10:51:33.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.297+0000 7f817609a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817015cdb0 con 0x7f817006abe0 2024-05-09T10:51:33.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.297+0000 7f817609a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817006b4d0 con 0x7f817006b060 2024-05-09T10:51:33.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.297+0000 7f816ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8170156180 0x7f817014e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:33.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.298+0000 7f816ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8170156180 0x7f817014e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:60370/0 (socket says 172.21.15.98:60370) 2024-05-09T10:51:33.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.298+0000 7f816ffff640 1 -- 172.21.15.98:0/4083826931 learned_addr learned my addr 172.21.15.98:0/4083826931 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:33.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.298+0000 7f816effd640 1 --2- 172.21.15.98:0/4083826931 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f817006b060 0x7f817014e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:33.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.298+0000 7f816f7fe640 1 --2- 172.21.15.98:0/4083826931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f817006abe0 0x7f817014d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:33.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.298+0000 7f816ffff640 1 -- 172.21.15.98:0/4083826931 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f817006b060 msgr2=0x7f817014e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:33.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.298+0000 7f816ffff640 1 --2- 172.21.15.98:0/4083826931 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f817006b060 0x7f817014e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:33.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.298+0000 7f816ffff640 1 -- 172.21.15.98:0/4083826931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f817006abe0 msgr2=0x7f817014d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:33.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.298+0000 7f816ffff640 1 --2- 172.21.15.98:0/4083826931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f817006abe0 0x7f817014d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:33.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.298+0000 7f816ffff640 1 -- 172.21.15.98:0/4083826931 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8170150370 con 0x7f8170156180 2024-05-09T10:51:33.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.298+0000 7f816effd640 1 --2- 172.21.15.98:0/4083826931 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f817006b060 0x7f817014e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:33.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.298+0000 7f816ffff640 1 --2- 172.21.15.98:0/4083826931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8170156180 0x7f817014e680 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f816800c480 tx=0x7f816800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:33.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.298+0000 7f8157fff640 1 -- 172.21.15.98:0/4083826931 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8168005480 con 0x7f8170156180 2024-05-09T10:51:33.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.298+0000 7f8157fff640 1 -- 172.21.15.98:0/4083826931 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81680035f0 con 0x7f8170156180 2024-05-09T10:51:33.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.298+0000 7f8157fff640 1 -- 172.21.15.98:0/4083826931 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8168003a00 con 0x7f8170156180 2024-05-09T10:51:33.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.298+0000 7f817609a640 1 -- 172.21.15.98:0/4083826931 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8170150650 con 0x7f8170156180 2024-05-09T10:51:33.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.298+0000 7f817609a640 1 -- 172.21.15.98:0/4083826931 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8170150c10 con 0x7f8170156180 2024-05-09T10:51:33.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.299+0000 7f817609a640 1 -- 172.21.15.98:0/4083826931 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8170069cb0 con 0x7f8170156180 2024-05-09T10:51:33.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.300+0000 7f8157fff640 1 -- 172.21.15.98:0/4083826931 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f8168003ba0 con 0x7f8170156180 2024-05-09T10:51:33.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.300+0000 7f8157fff640 1 --2- 172.21.15.98:0/4083826931 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f815c06b820 0x7f815c06dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:33.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.300+0000 7f8157fff640 1 -- 172.21.15.98:0/4083826931 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8168089da0 con 0x7f8170156180 2024-05-09T10:51:33.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.300+0000 7f816f7fe640 1 --2- 172.21.15.98:0/4083826931 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f815c06b820 0x7f815c06dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:33.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.301+0000 7f816f7fe640 1 --2- 172.21.15.98:0/4083826931 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f815c06b820 0x7f815c06dce0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8160034000 tx=0x7f816002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:33.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.304+0000 7f8157fff640 1 -- 172.21.15.98:0/4083826931 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8168058d30 con 0x7f8170156180 2024-05-09T10:51:33.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.541+0000 7f817609a640 1 -- 172.21.15.98:0/4083826931 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread"]} v 0) v1 -- 0x7f81700b1d80 con 0x7f8170156180 2024-05-09T10:51:33.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.875+0000 7f8157fff640 1 -- 172.21.15.98:0/4083826931 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread"]}]=0 v97) v1 ==== 167+0+0 (secure 0 0 0) 0x7f8168058630 con 0x7f8170156180 2024-05-09T10:51:33.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.877+0000 7f817609a640 1 -- 172.21.15.98:0/4083826931 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f815c06b820 msgr2=0x7f815c06dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:33.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.878+0000 7f817609a640 1 --2- 172.21.15.98:0/4083826931 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f815c06b820 0x7f815c06dce0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8160034000 tx=0x7f816002e000 comp rx=0 tx=0).stop 2024-05-09T10:51:33.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.878+0000 7f817609a640 1 -- 172.21.15.98:0/4083826931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8170156180 msgr2=0x7f817014e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:33.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.878+0000 7f817609a640 1 --2- 172.21.15.98:0/4083826931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8170156180 0x7f817014e680 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f816800c480 tx=0x7f816800c940 comp rx=0 tx=0).stop 2024-05-09T10:51:33.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.878+0000 7f817609a640 1 -- 172.21.15.98:0/4083826931 shutdown_connections 2024-05-09T10:51:33.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.878+0000 7f817609a640 1 --2- 172.21.15.98:0/4083826931 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f815c06b820 0x7f815c06dce0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:33.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.878+0000 7f817609a640 1 --2- 172.21.15.98:0/4083826931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8170156180 0x7f817014e680 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:33.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.878+0000 7f817609a640 1 --2- 172.21.15.98:0/4083826931 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f817006b060 0x7f817014e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:33.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.878+0000 7f817609a640 1 --2- 172.21.15.98:0/4083826931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f817006abe0 0x7f817014d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:33.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.878+0000 7f817609a640 1 -- 172.21.15.98:0/4083826931 >> 172.21.15.98:0/4083826931 conn(0x7f8170066f20 msgr2=0x7f8170071d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:33.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.878+0000 7f817609a640 1 -- 172.21.15.98:0/4083826931 shutdown_connections 2024-05-09T10:51:33.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:33.878+0000 7f817609a640 1 -- 172.21.15.98:0/4083826931 wait complete. 2024-05-09T10:51:33.882 INFO:tasks.cephfs_test_runner:test_volumes_module_finisher_thread (tasks.cephfs.test_volumes.TestPerModuleFinsherThread) ... ok 2024-05-09T10:51:33.883 INFO:tasks.cephfs_test_runner: 2024-05-09T10:51:33.883 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-09T10:51:33.883 INFO:tasks.cephfs_test_runner:Ran 1 test in 49.212s 2024-05-09T10:51:33.883 INFO:tasks.cephfs_test_runner: 2024-05-09T10:51:33.883 INFO:tasks.cephfs_test_runner:OK 2024-05-09T10:51:33.883 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-09T10:51:33.892 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-09T10:51:33.900 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:51:34.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.007+0000 7f7807d42640 1 Processor -- start 2024-05-09T10:51:34.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.008+0000 7f7807d42640 1 -- start start 2024-05-09T10:51:34.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.008+0000 7f7807d42640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f780006bbf0 0x7f780006bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:34.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.008+0000 7f7807d42640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7800072a20 0x7f78000ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:34.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.008+0000 7f7807d42640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78000cbf50 con 0x7f78000ce2c0 2024-05-09T10:51:34.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.008+0000 7f7807d42640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78000cc1c0 con 0x7f7800072a20 2024-05-09T10:51:34.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.008+0000 7f7807d42640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780006f2c0 con 0x7f780006bbf0 2024-05-09T10:51:34.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f78052dd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f78000ce2c0 0x7f7800072310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52708/0 (socket says 172.21.15.98:52708) 2024-05-09T10:51:34.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f78052dd640 1 -- 172.21.15.98:0/4125881148 learned_addr learned my addr 172.21.15.98:0/4125881148 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:34.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f7804adc640 1 -- 172.21.15.98:0/4125881148 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1032382351 0 0) 0x7f78000cbf50 con 0x7f78000ce2c0 2024-05-09T10:51:34.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f7805ade640 1 --2- 172.21.15.98:0/4125881148 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f780006bbf0 0x7f780006bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:34.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f78062df640 1 --2- 172.21.15.98:0/4125881148 >> v2:172.21.15.105:3300/0 conn(0x7f7800072a20 0x7f78000ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:34.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f7804adc640 1 -- 172.21.15.98:0/4125881148 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77e8003520 con 0x7f78000ce2c0 2024-05-09T10:51:34.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f7804adc640 1 -- 172.21.15.98:0/4125881148 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 514204091 0 0) 0x7f77e8003520 con 0x7f78000ce2c0 2024-05-09T10:51:34.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f7804adc640 1 -- 172.21.15.98:0/4125881148 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f780006bbf0 msgr2=0x7f780006bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:34.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f7804adc640 1 --2- 172.21.15.98:0/4125881148 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f780006bbf0 0x7f780006bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f7804adc640 1 -- 172.21.15.98:0/4125881148 >> v2:172.21.15.105:3300/0 conn(0x7f7800072a20 msgr2=0x7f78000ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:34.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f7804adc640 1 --2- 172.21.15.98:0/4125881148 >> v2:172.21.15.105:3300/0 conn(0x7f7800072a20 0x7f78000ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f7804adc640 1 -- 172.21.15.98:0/4125881148 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780006c510 con 0x7f78000ce2c0 2024-05-09T10:51:34.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f7804adc640 1 -- 172.21.15.98:0/4125881148 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f77f4001670 con 0x7f78000ce2c0 2024-05-09T10:51:34.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f7804adc640 1 -- 172.21.15.98:0/4125881148 >> v1:172.21.15.98:6789/0 conn(0x7f78000ce2c0 legacy=0x7f7800072310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:34.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f7804adc640 1 --2- 172.21.15.98:0/4125881148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77e80039b0 0x7f77e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:34.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f7804adc640 1 --2- 172.21.15.98:0/4125881148 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f780006bbf0 0x7f77e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:34.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f7804adc640 1 --2- 172.21.15.98:0/4125881148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77e8004d40 0x7f77e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:34.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f7804adc640 1 -- 172.21.15.98:0/4125881148 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78000cc1c0 con 0x7f77e80039b0 2024-05-09T10:51:34.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f7804adc640 1 -- 172.21.15.98:0/4125881148 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780006f2c0 con 0x7f77e8004d40 2024-05-09T10:51:34.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f7804adc640 1 -- 172.21.15.98:0/4125881148 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e800ba40 con 0x7f780006bbf0 2024-05-09T10:51:34.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f7805ade640 1 --2- 172.21.15.98:0/4125881148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77e80039b0 0x7f77e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:34.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f78052dd640 1 --2- 172.21.15.98:0/4125881148 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f780006bbf0 0x7f77e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:34.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f78062df640 1 --2- 172.21.15.98:0/4125881148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77e8004d40 0x7f77e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:34.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f7805ade640 1 -- 172.21.15.98:0/4125881148 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f780006bbf0 msgr2=0x7f77e8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:34.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.009+0000 7f7805ade640 1 --2- 172.21.15.98:0/4125881148 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f780006bbf0 0x7f77e8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.010+0000 7f7805ade640 1 -- 172.21.15.98:0/4125881148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77e8004d40 msgr2=0x7f77e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:34.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.010+0000 7f7805ade640 1 --2- 172.21.15.98:0/4125881148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77e8004d40 0x7f77e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.010+0000 7f7805ade640 1 -- 172.21.15.98:0/4125881148 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e8003520 con 0x7f77e80039b0 2024-05-09T10:51:34.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.010+0000 7f7805ade640 1 --2- 172.21.15.98:0/4125881148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77e80039b0 0x7f77e8003dd0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009ac0 tx=0x7f77e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a964e9b51ac5e844 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:34.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.010+0000 7f7804adc640 1 -- 172.21.15.98:0/4125881148 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f8036070 con 0x7f77e80039b0 2024-05-09T10:51:34.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.010+0000 7f7804adc640 1 -- 172.21.15.98:0/4125881148 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f8031400 con 0x7f77e80039b0 2024-05-09T10:51:34.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.010+0000 7f7804adc640 1 -- 172.21.15.98:0/4125881148 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f8009690 con 0x7f77e80039b0 2024-05-09T10:51:34.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.011+0000 7f7807d42640 1 -- 172.21.15.98:0/4125881148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77e80039b0 msgr2=0x7f77e8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:34.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.011+0000 7f7807d42640 1 --2- 172.21.15.98:0/4125881148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77e80039b0 0x7f77e8003dd0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009ac0 tx=0x7f77e8005830 comp rx=0 tx=0).stop 2024-05-09T10:51:34.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.011+0000 7f78062df640 1 -- 172.21.15.98:0/4125881148 reap_dead start 2024-05-09T10:51:34.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.011+0000 7f7807d42640 1 -- 172.21.15.98:0/4125881148 shutdown_connections 2024-05-09T10:51:34.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.011+0000 7f7807d42640 1 -- 172.21.15.98:0/4125881148 >> 172.21.15.98:0/4125881148 conn(0x7f78000ccaa0 msgr2=0x7f78000cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:34.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.011+0000 7f7807d42640 1 -- 172.21.15.98:0/4125881148 shutdown_connections 2024-05-09T10:51:34.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.011+0000 7f7807d42640 1 -- 172.21.15.98:0/4125881148 wait complete. 2024-05-09T10:51:34.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.012+0000 7f7807d42640 1 Processor -- start 2024-05-09T10:51:34.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.012+0000 7f7807d42640 1 -- start start 2024-05-09T10:51:34.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.013+0000 7f7807d42640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77e80039b0 0x7f78000b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:34.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.013+0000 7f7807d42640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78000b5240 0x7f78000b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:34.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.013+0000 7f7807d42640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78001f3d00 0x7f78001f4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:34.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.013+0000 7f7807d42640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78000c37d0 con 0x7f78001f3d00 2024-05-09T10:51:34.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.013+0000 7f7807d42640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78000cb820 con 0x7f77e80039b0 2024-05-09T10:51:34.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.013+0000 7f7807d42640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78000c3950 con 0x7f78000b5240 2024-05-09T10:51:34.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.013+0000 7f78062df640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78001f3d00 0x7f78001f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:34.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.013+0000 7f78062df640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78001f3d00 0x7f78001f4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54388/0 (socket says 172.21.15.98:54388) 2024-05-09T10:51:34.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.013+0000 7f78062df640 1 -- 172.21.15.98:0/4045249991 learned_addr learned my addr 172.21.15.98:0/4045249991 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:34.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.013+0000 7f78062df640 1 -- 172.21.15.98:0/4045249991 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78000b5240 msgr2=0x7f78000b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:34.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.013+0000 7f78062df640 1 --2- 172.21.15.98:0/4045249991 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78000b5240 0x7f78000b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.013+0000 7f78062df640 1 -- 172.21.15.98:0/4045249991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77e80039b0 msgr2=0x7f78000b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:34.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.013+0000 7f78062df640 1 --2- 172.21.15.98:0/4045249991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77e80039b0 0x7f78000b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.013+0000 7f78062df640 1 -- 172.21.15.98:0/4045249991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78000b3b20 con 0x7f78001f3d00 2024-05-09T10:51:34.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.013+0000 7f78062df640 1 --2- 172.21.15.98:0/4045249991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78001f3d00 0x7f78001f4180 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f78000ce040 tx=0x7f77f000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:34.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.014+0000 7f77fe7fc640 1 -- 172.21.15.98:0/4045249991 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0019070 con 0x7f78001f3d00 2024-05-09T10:51:34.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.014+0000 7f77fe7fc640 1 -- 172.21.15.98:0/4045249991 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f0011e90 con 0x7f78001f3d00 2024-05-09T10:51:34.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.014+0000 7f7807d42640 1 -- 172.21.15.98:0/4045249991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78000b3e00 con 0x7f78001f3d00 2024-05-09T10:51:34.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.014+0000 7f77fe7fc640 1 -- 172.21.15.98:0/4045249991 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0003bf0 con 0x7f78001f3d00 2024-05-09T10:51:34.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.014+0000 7f7807d42640 1 -- 172.21.15.98:0/4045249991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78000b43c0 con 0x7f78001f3d00 2024-05-09T10:51:34.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.014+0000 7f7807d42640 1 -- 172.21.15.98:0/4045249991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7800069cb0 con 0x7f78001f3d00 2024-05-09T10:51:34.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.015+0000 7f77fe7fc640 1 -- 172.21.15.98:0/4045249991 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f77f001c070 con 0x7f78001f3d00 2024-05-09T10:51:34.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.016+0000 7f77fe7fc640 1 --2- 172.21.15.98:0/4045249991 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f77ec06b6b0 0x7f77ec06db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:34.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.016+0000 7f77fe7fc640 1 -- 172.21.15.98:0/4045249991 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f77f008abe0 con 0x7f78001f3d00 2024-05-09T10:51:34.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.016+0000 7f7805ade640 1 --2- 172.21.15.98:0/4045249991 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f77ec06b6b0 0x7f77ec06db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:34.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.016+0000 7f7805ade640 1 --2- 172.21.15.98:0/4045249991 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f77ec06b6b0 0x7f77ec06db70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f77f8033000 tx=0x7f77f802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:34.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.020+0000 7f77fe7fc640 1 -- 172.21.15.98:0/4045249991 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f77f0010040 con 0x7f78001f3d00 2024-05-09T10:51:34.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.191+0000 7f7807d42640 1 -- 172.21.15.98:0/4045249991 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78000b3320 con 0x7f78001f3d00 2024-05-09T10:51:34.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.192+0000 7f77fe7fc640 1 -- 172.21.15.98:0/4045249991 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+13018 (secure 0 0 0) 0x7f77f0059a60 con 0x7f78001f3d00 2024-05-09T10:51:34.185 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:34.185 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":34,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:51:32.870839+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.98:6837/164638690":"2024-05-10T10:51:29.862716+0000","172.21.15.98:6835/164638690":"2024-05-10T10:51:29.862716+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:51:34.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.193+0000 7f7807d42640 1 -- 172.21.15.98:0/4045249991 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f77ec06b6b0 msgr2=0x7f77ec06db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:34.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.193+0000 7f7807d42640 1 --2- 172.21.15.98:0/4045249991 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f77ec06b6b0 0x7f77ec06db70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f77f8033000 tx=0x7f77f802e000 comp rx=0 tx=0).stop 2024-05-09T10:51:34.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.193+0000 7f7807d42640 1 -- 172.21.15.98:0/4045249991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78001f3d00 msgr2=0x7f78001f4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:34.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.194+0000 7f7807d42640 1 --2- 172.21.15.98:0/4045249991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78001f3d00 0x7f78001f4180 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f78000ce040 tx=0x7f77f000f580 comp rx=0 tx=0).stop 2024-05-09T10:51:34.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.194+0000 7f7807d42640 1 -- 172.21.15.98:0/4045249991 shutdown_connections 2024-05-09T10:51:34.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.194+0000 7f7807d42640 1 --2- 172.21.15.98:0/4045249991 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f77ec06b6b0 0x7f77ec06db70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.194+0000 7f7807d42640 1 --2- 172.21.15.98:0/4045249991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78001f3d00 0x7f78001f4180 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.194+0000 7f7807d42640 1 --2- 172.21.15.98:0/4045249991 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78000b5240 0x7f78000b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.194+0000 7f7807d42640 1 --2- 172.21.15.98:0/4045249991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77e80039b0 0x7f78000b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.194+0000 7f7807d42640 1 -- 172.21.15.98:0/4045249991 >> 172.21.15.98:0/4045249991 conn(0x7f78000ccaa0 msgr2=0x7f780009f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:34.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.194+0000 7f7807d42640 1 -- 172.21.15.98:0/4045249991 shutdown_connections 2024-05-09T10:51:34.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.194+0000 7f7807d42640 1 -- 172.21.15.98:0/4045249991 wait complete. 2024-05-09T10:51:34.198 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T10:50:07.396423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T10:51:34.198 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T10:51:34.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.306+0000 7efc390bd640 1 Processor -- start 2024-05-09T10:51:34.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.307+0000 7efc390bd640 1 -- start start 2024-05-09T10:51:34.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.307+0000 7efc390bd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efc340720d0 0x7efc340af720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:34.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.307+0000 7efc390bd640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc34072690 con 0x7efc3406abe0 2024-05-09T10:51:34.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.307+0000 7efc390bd640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc340b57b0 con 0x7efc3415db90 2024-05-09T10:51:34.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.307+0000 7efc390bd640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc340afc60 con 0x7efc340720d0 2024-05-09T10:51:34.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.307+0000 7efc3359e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7efc3406abe0 0x7efc3406afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52712/0 (socket says 172.21.15.98:52712) 2024-05-09T10:51:34.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc3359e640 1 -- 172.21.15.98:0/3716350183 learned_addr learned my addr 172.21.15.98:0/3716350183 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:34.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc3259c640 1 --2- 172.21.15.98:0/3716350183 >> v2:172.21.15.105:3300/0 conn(0x7efc340720d0 0x7efc340af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:34.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc31d9b640 1 -- 172.21.15.98:0/3716350183 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439349451 0 0) 0x7efc34072690 con 0x7efc3406abe0 2024-05-09T10:51:34.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc31d9b640 1 -- 172.21.15.98:0/3716350183 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc20003520 con 0x7efc3406abe0 2024-05-09T10:51:34.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc31d9b640 1 -- 172.21.15.98:0/3716350183 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 946725706 0 0) 0x7efc20003520 con 0x7efc3406abe0 2024-05-09T10:51:34.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc31d9b640 1 -- 172.21.15.98:0/3716350183 >> v2:172.21.15.105:3300/0 conn(0x7efc340720d0 msgr2=0x7efc340af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:34.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc31d9b640 1 --2- 172.21.15.98:0/3716350183 >> v2:172.21.15.105:3300/0 conn(0x7efc340720d0 0x7efc340af720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc31d9b640 1 -- 172.21.15.98:0/3716350183 >> v1:172.21.15.105:6789/0 conn(0x7efc3415db90 legacy=0x7efc3406d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:34.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc31d9b640 1 -- 172.21.15.98:0/3716350183 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc340b5a90 con 0x7efc3406abe0 2024-05-09T10:51:34.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc31d9b640 1 -- 172.21.15.98:0/3716350183 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7efc2c001a10 con 0x7efc3406abe0 2024-05-09T10:51:34.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc31d9b640 1 -- 172.21.15.98:0/3716350183 >> v1:172.21.15.98:6789/0 conn(0x7efc3406abe0 legacy=0x7efc3406afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:34.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc31d9b640 1 --2- 172.21.15.98:0/3716350183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc200039b0 0x7efc20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:34.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc31d9b640 1 --2- 172.21.15.98:0/3716350183 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc20004800 0x7efc20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:34.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc31d9b640 1 --2- 172.21.15.98:0/3716350183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc200052e0 0x7efc2000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:34.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc31d9b640 1 -- 172.21.15.98:0/3716350183 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc340afc60 con 0x7efc200039b0 2024-05-09T10:51:34.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc31d9b640 1 -- 172.21.15.98:0/3716350183 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2000c1b0 con 0x7efc200052e0 2024-05-09T10:51:34.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc31d9b640 1 -- 172.21.15.98:0/3716350183 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2000c3b0 con 0x7efc20004800 2024-05-09T10:51:34.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc32d9d640 1 --2- 172.21.15.98:0/3716350183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc200039b0 0x7efc20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:34.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc3359e640 1 --2- 172.21.15.98:0/3716350183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc200052e0 0x7efc2000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:34.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc3259c640 1 --2- 172.21.15.98:0/3716350183 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc20004800 0x7efc20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:34.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc32d9d640 1 -- 172.21.15.98:0/3716350183 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc20004800 msgr2=0x7efc20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:34.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc32d9d640 1 --2- 172.21.15.98:0/3716350183 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc20004800 0x7efc20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc32d9d640 1 -- 172.21.15.98:0/3716350183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc200052e0 msgr2=0x7efc2000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:34.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc32d9d640 1 --2- 172.21.15.98:0/3716350183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc200052e0 0x7efc2000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc3359e640 1 -- 172.21.15.98:0/3716350183 reap_dead start 2024-05-09T10:51:34.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.308+0000 7efc32d9d640 1 -- 172.21.15.98:0/3716350183 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc20003520 con 0x7efc200039b0 2024-05-09T10:51:34.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.309+0000 7efc32d9d640 1 --2- 172.21.15.98:0/3716350183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc200039b0 0x7efc20003dd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7efc1c001720 tx=0x7efc20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b7ed458597ddc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:34.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.309+0000 7efc31d9b640 1 -- 172.21.15.98:0/3716350183 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c001370 con 0x7efc200039b0 2024-05-09T10:51:34.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.309+0000 7efc31d9b640 1 -- 172.21.15.98:0/3716350183 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1c0052d0 con 0x7efc200039b0 2024-05-09T10:51:34.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.309+0000 7efc31d9b640 1 -- 172.21.15.98:0/3716350183 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c02ab90 con 0x7efc200039b0 2024-05-09T10:51:34.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.310+0000 7efc390bd640 1 -- 172.21.15.98:0/3716350183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc200039b0 msgr2=0x7efc20003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:34.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.310+0000 7efc390bd640 1 --2- 172.21.15.98:0/3716350183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc200039b0 0x7efc20003dd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7efc1c001720 tx=0x7efc20005830 comp rx=0 tx=0).stop 2024-05-09T10:51:34.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.310+0000 7efc390bd640 1 -- 172.21.15.98:0/3716350183 shutdown_connections 2024-05-09T10:51:34.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.310+0000 7efc390bd640 1 --2- 172.21.15.98:0/3716350183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc200039b0 0x7efc20003dd0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.310+0000 7efc390bd640 1 -- 172.21.15.98:0/3716350183 >> 172.21.15.98:0/3716350183 conn(0x7efc34066fd0 msgr2=0x7efc3406bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:34.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.310+0000 7efc390bd640 1 -- 172.21.15.98:0/3716350183 shutdown_connections 2024-05-09T10:51:34.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.310+0000 7efc390bd640 1 -- 172.21.15.98:0/3716350183 wait complete. 2024-05-09T10:51:34.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.311+0000 7efc390bd640 1 Processor -- start 2024-05-09T10:51:34.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.311+0000 7efc390bd640 1 -- start start 2024-05-09T10:51:34.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.312+0000 7efc390bd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc200039b0 0x7efc3415f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:34.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.312+0000 7efc390bd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc341484b0 0x7efc3414b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:34.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.312+0000 7efc390bd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc34148910 0x7efc3414b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:34.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.312+0000 7efc390bd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3415df70 con 0x7efc341484b0 2024-05-09T10:51:34.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.312+0000 7efc390bd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc340b5d70 con 0x7efc200039b0 2024-05-09T10:51:34.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.312+0000 7efc390bd640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3415e0f0 con 0x7efc34148910 2024-05-09T10:51:34.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.312+0000 7efc3259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc341484b0 0x7efc3414b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:34.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.312+0000 7efc3259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc341484b0 0x7efc3414b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54408/0 (socket says 172.21.15.98:54408) 2024-05-09T10:51:34.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.312+0000 7efc3259c640 1 -- 172.21.15.98:0/2894854845 learned_addr learned my addr 172.21.15.98:0/2894854845 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:34.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.312+0000 7efc32d9d640 1 --2- 172.21.15.98:0/2894854845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc200039b0 0x7efc3415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:34.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.312+0000 7efc3359e640 1 --2- 172.21.15.98:0/2894854845 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc34148910 0x7efc3414b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:34.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.312+0000 7efc3259c640 1 -- 172.21.15.98:0/2894854845 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc34148910 msgr2=0x7efc3414b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:34.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.312+0000 7efc3259c640 1 --2- 172.21.15.98:0/2894854845 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc34148910 0x7efc3414b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.312+0000 7efc3259c640 1 -- 172.21.15.98:0/2894854845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc200039b0 msgr2=0x7efc3415f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:34.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.312+0000 7efc3259c640 1 --2- 172.21.15.98:0/2894854845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc200039b0 0x7efc3415f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.312+0000 7efc3259c640 1 -- 172.21.15.98:0/2894854845 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3415fb50 con 0x7efc341484b0 2024-05-09T10:51:34.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.312+0000 7efc32d9d640 1 --2- 172.21.15.98:0/2894854845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc200039b0 0x7efc3415f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:34.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.313+0000 7efc3259c640 1 --2- 172.21.15.98:0/2894854845 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc341484b0 0x7efc3414b170 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7efc240114e0 tx=0x7efc240119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:34.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.313+0000 7efc2b7fe640 1 -- 172.21.15.98:0/2894854845 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc24015870 con 0x7efc341484b0 2024-05-09T10:51:34.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.313+0000 7efc2b7fe640 1 -- 172.21.15.98:0/2894854845 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc24015a10 con 0x7efc341484b0 2024-05-09T10:51:34.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.313+0000 7efc2b7fe640 1 -- 172.21.15.98:0/2894854845 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc24015dc0 con 0x7efc341484b0 2024-05-09T10:51:34.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.313+0000 7efc390bd640 1 -- 172.21.15.98:0/2894854845 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc3415fe30 con 0x7efc341484b0 2024-05-09T10:51:34.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.313+0000 7efc390bd640 1 -- 172.21.15.98:0/2894854845 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc341603f0 con 0x7efc341484b0 2024-05-09T10:51:34.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.314+0000 7efc390bd640 1 -- 172.21.15.98:0/2894854845 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc34072dd0 con 0x7efc341484b0 2024-05-09T10:51:34.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.314+0000 7efc2b7fe640 1 -- 172.21.15.98:0/2894854845 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7efc24013400 con 0x7efc341484b0 2024-05-09T10:51:34.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.315+0000 7efc2b7fe640 1 --2- 172.21.15.98:0/2894854845 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7efc1406b6b0 0x7efc1406db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:34.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.315+0000 7efc2b7fe640 1 -- 172.21.15.98:0/2894854845 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7efc2408aa20 con 0x7efc341484b0 2024-05-09T10:51:34.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.315+0000 7efc32d9d640 1 --2- 172.21.15.98:0/2894854845 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7efc1406b6b0 0x7efc1406db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:34.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.315+0000 7efc32d9d640 1 --2- 172.21.15.98:0/2894854845 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7efc1406b6b0 0x7efc1406db70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efc1c032000 tx=0x7efc1c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:34.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.319+0000 7efc2b7fe640 1 -- 172.21.15.98:0/2894854845 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efc24019030 con 0x7efc341484b0 2024-05-09T10:51:34.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.490+0000 7efc390bd640 1 -- 172.21.15.98:0/2894854845 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc340af8c0 con 0x7efc341484b0 2024-05-09T10:51:34.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.491+0000 7efc2b7fe640 1 -- 172.21.15.98:0/2894854845 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7efc24059920 con 0x7efc341484b0 2024-05-09T10:51:34.484 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-05-09T10:51:34.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.492+0000 7efc390bd640 1 -- 172.21.15.98:0/2894854845 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7efc1406b6b0 msgr2=0x7efc1406db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:34.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.492+0000 7efc390bd640 1 --2- 172.21.15.98:0/2894854845 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7efc1406b6b0 0x7efc1406db70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efc1c032000 tx=0x7efc1c02e000 comp rx=0 tx=0).stop 2024-05-09T10:51:34.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.492+0000 7efc390bd640 1 -- 172.21.15.98:0/2894854845 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc341484b0 msgr2=0x7efc3414b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:34.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.492+0000 7efc390bd640 1 --2- 172.21.15.98:0/2894854845 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc341484b0 0x7efc3414b170 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7efc240114e0 tx=0x7efc240119a0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.492+0000 7efc390bd640 1 -- 172.21.15.98:0/2894854845 shutdown_connections 2024-05-09T10:51:34.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.492+0000 7efc390bd640 1 --2- 172.21.15.98:0/2894854845 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7efc1406b6b0 0x7efc1406db70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.492+0000 7efc390bd640 1 --2- 172.21.15.98:0/2894854845 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc34148910 0x7efc3414b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.492+0000 7efc390bd640 1 --2- 172.21.15.98:0/2894854845 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc341484b0 0x7efc3414b170 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.492+0000 7efc390bd640 1 --2- 172.21.15.98:0/2894854845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc200039b0 0x7efc3415f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.492+0000 7efc390bd640 1 -- 172.21.15.98:0/2894854845 >> 172.21.15.98:0/2894854845 conn(0x7efc34066fd0 msgr2=0x7efc34161570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:34.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.492+0000 7efc390bd640 1 -- 172.21.15.98:0/2894854845 shutdown_connections 2024-05-09T10:51:34.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.493+0000 7efc390bd640 1 -- 172.21.15.98:0/2894854845 wait complete. 2024-05-09T10:51:34.497 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-09T10:51:34.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.604+0000 7f0916eee640 1 Processor -- start 2024-05-09T10:51:34.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.604+0000 7f0916eee640 1 -- start start 2024-05-09T10:51:34.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.605+0000 7f0916eee640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f091006bbf0 0x7f091006bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:34.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.605+0000 7f0916eee640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0910072a20 0x7f09100ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:34.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.605+0000 7f0916eee640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09100cbf50 con 0x7f09100ce2c0 2024-05-09T10:51:34.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.605+0000 7f0916eee640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09100cc1c0 con 0x7f0910072a20 2024-05-09T10:51:34.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.605+0000 7f0916eee640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091006f2c0 con 0x7f091006bbf0 2024-05-09T10:51:34.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.605+0000 7f091548b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0910072a20 0x7f09100ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:34.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.605+0000 7f091548b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0910072a20 0x7f09100ce720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54414/0 (socket says 172.21.15.98:54414) 2024-05-09T10:51:34.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.605+0000 7f091548b640 1 -- 172.21.15.98:0/710610443 learned_addr learned my addr 172.21.15.98:0/710610443 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:34.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.605+0000 7f0914c8a640 1 --2- 172.21.15.98:0/710610443 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f091006bbf0 0x7f091006bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:34.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.605+0000 7f091548b640 1 -- 172.21.15.98:0/710610443 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f091006bbf0 msgr2=0x7f091006bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:34.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.605+0000 7f091548b640 1 --2- 172.21.15.98:0/710610443 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f091006bbf0 0x7f091006bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.605+0000 7f091548b640 1 -- 172.21.15.98:0/710610443 >> v1:172.21.15.105:6789/0 conn(0x7f09100ce2c0 legacy=0x7f0910072310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:34.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.605+0000 7f091548b640 1 -- 172.21.15.98:0/710610443 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091006c510 con 0x7f0910072a20 2024-05-09T10:51:34.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.606+0000 7f091548b640 1 --2- 172.21.15.98:0/710610443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0910072a20 0x7f09100ce720 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0908009870 tx=0x7f090802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c556d6747afb043 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:34.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.606+0000 7f09077fe640 1 -- 172.21.15.98:0/710610443 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908037070 con 0x7f0910072a20 2024-05-09T10:51:34.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.606+0000 7f09077fe640 1 -- 172.21.15.98:0/710610443 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090802ae10 con 0x7f0910072a20 2024-05-09T10:51:34.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.606+0000 7f09077fe640 1 -- 172.21.15.98:0/710610443 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908032660 con 0x7f0910072a20 2024-05-09T10:51:34.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.607+0000 7f0916eee640 1 -- 172.21.15.98:0/710610443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0910072a20 msgr2=0x7f09100ce720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:34.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.607+0000 7f0916eee640 1 --2- 172.21.15.98:0/710610443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0910072a20 0x7f09100ce720 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0908009870 tx=0x7f090802a100 comp rx=0 tx=0).stop 2024-05-09T10:51:34.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.607+0000 7f0916eee640 1 -- 172.21.15.98:0/710610443 shutdown_connections 2024-05-09T10:51:34.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.607+0000 7f0916eee640 1 --2- 172.21.15.98:0/710610443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0910072a20 0x7f09100ce720 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.607+0000 7f0916eee640 1 --2- 172.21.15.98:0/710610443 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f091006bbf0 0x7f091006bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.607+0000 7f0916eee640 1 -- 172.21.15.98:0/710610443 >> 172.21.15.98:0/710610443 conn(0x7f09100ccaa0 msgr2=0x7f09100cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:34.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.607+0000 7f0916eee640 1 -- 172.21.15.98:0/710610443 shutdown_connections 2024-05-09T10:51:34.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.607+0000 7f0916eee640 1 -- 172.21.15.98:0/710610443 wait complete. 2024-05-09T10:51:34.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.608+0000 7f0916eee640 1 Processor -- start 2024-05-09T10:51:34.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.608+0000 7f0916eee640 1 -- start start 2024-05-09T10:51:34.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.609+0000 7f0916eee640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f091006bbf0 0x7f09100c3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:34.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.609+0000 7f0916eee640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0910072a20 0x7f09101e4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:34.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.609+0000 7f0916eee640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09100ce2c0 0x7f09100b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:34.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.609+0000 7f0916eee640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09100c37d0 con 0x7f09100ce2c0 2024-05-09T10:51:34.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.609+0000 7f0916eee640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09100cb820 con 0x7f0910072a20 2024-05-09T10:51:34.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.609+0000 7f0916eee640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09100c3950 con 0x7f091006bbf0 2024-05-09T10:51:34.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.609+0000 7f091548b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09100ce2c0 0x7f09100b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:34.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.609+0000 7f091548b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09100ce2c0 0x7f09100b0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54426/0 (socket says 172.21.15.98:54426) 2024-05-09T10:51:34.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.609+0000 7f091548b640 1 -- 172.21.15.98:0/2634875794 learned_addr learned my addr 172.21.15.98:0/2634875794 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:34.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.609+0000 7f0914c8a640 1 --2- 172.21.15.98:0/2634875794 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f091006bbf0 0x7f09100c3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:34.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.609+0000 7f091548b640 1 -- 172.21.15.98:0/2634875794 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f091006bbf0 msgr2=0x7f09100c3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:34.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.609+0000 7f091548b640 1 --2- 172.21.15.98:0/2634875794 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f091006bbf0 0x7f09100c3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.609+0000 7f091548b640 1 -- 172.21.15.98:0/2634875794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0910072a20 msgr2=0x7f09101e4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:34.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.609+0000 7f0907fff640 1 --2- 172.21.15.98:0/2634875794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0910072a20 0x7f09101e4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:34.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.609+0000 7f091548b640 1 --2- 172.21.15.98:0/2634875794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0910072a20 0x7f09101e4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:34.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.609+0000 7f091548b640 1 -- 172.21.15.98:0/2634875794 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09100cb2b0 con 0x7f09100ce2c0 2024-05-09T10:51:34.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.609+0000 7f0914c8a640 1 --2- 172.21.15.98:0/2634875794 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f091006bbf0 0x7f09100c3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:34.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.609+0000 7f091548b640 1 --2- 172.21.15.98:0/2634875794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09100ce2c0 0x7f09100b0260 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f090802a140 tx=0x7f09080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:34.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.609+0000 7f09057fa640 1 -- 172.21.15.98:0/2634875794 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908041070 con 0x7f09100ce2c0 2024-05-09T10:51:34.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.610+0000 7f09057fa640 1 -- 172.21.15.98:0/2634875794 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090803c670 con 0x7f09100ce2c0 2024-05-09T10:51:34.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.610+0000 7f0916eee640 1 -- 172.21.15.98:0/2634875794 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09100c47d0 con 0x7f09100ce2c0 2024-05-09T10:51:34.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.610+0000 7f09057fa640 1 -- 172.21.15.98:0/2634875794 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908037040 con 0x7f09100ce2c0 2024-05-09T10:51:34.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.610+0000 7f0916eee640 1 -- 172.21.15.98:0/2634875794 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09100b40c0 con 0x7f09100ce2c0 2024-05-09T10:51:34.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.611+0000 7f0916eee640 1 -- 172.21.15.98:0/2634875794 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0910069cb0 con 0x7f09100ce2c0 2024-05-09T10:51:34.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.611+0000 7f09057fa640 1 -- 172.21.15.98:0/2634875794 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f0908044070 con 0x7f09100ce2c0 2024-05-09T10:51:34.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.611+0000 7f09057fa640 1 --2- 172.21.15.98:0/2634875794 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f08fc06b680 0x7f08fc06db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:34.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.612+0000 7f09057fa640 1 -- 172.21.15.98:0/2634875794 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f09080aaa10 con 0x7f09100ce2c0 2024-05-09T10:51:34.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.612+0000 7f0914c8a640 1 --2- 172.21.15.98:0/2634875794 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f08fc06b680 0x7f08fc06db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:34.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.612+0000 7f0914c8a640 1 --2- 172.21.15.98:0/2634875794 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f08fc06b680 0x7f08fc06db40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f090c007f50 tx=0x7f090c0074a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:34.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.616+0000 7f09057fa640 1 -- 172.21.15.98:0/2634875794 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0908079910 con 0x7f09100ce2c0 2024-05-09T10:51:34.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.833+0000 7f0916eee640 1 -- 172.21.15.98:0/2634875794 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f09100c4040 con 0x7f09100ce2c0 2024-05-09T10:51:34.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.837+0000 7f09057fa640 1 -- 172.21.15.98:0/2634875794 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v7) v1 ==== 81+0+468618 (secure 0 0 0) 0x7f0908079210 con 0x7f09100ce2c0 2024-05-09T10:51:34.836 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:34.836 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-05-09T10:51:34.836 INFO:teuthology.orchestra.run.smithi098.stdout: "epoch": 7, 2024-05-09T10:51:34.836 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.836 INFO:teuthology.orchestra.run.smithi098.stdout: "active_gid": 4110, 2024-05-09T10:51:34.836 INFO:teuthology.orchestra.run.smithi098.stdout: "active_name": "y", 2024-05-09T10:51:34.836 INFO:teuthology.orchestra.run.smithi098.stdout: "active_addrs": { 2024-05-09T10:51:34.837 INFO:teuthology.orchestra.run.smithi098.stdout: "addrvec": [ 2024-05-09T10:51:34.837 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:34.837 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v2", 2024-05-09T10:51:34.837 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "172.21.15.98:6832", 2024-05-09T10:51:34.837 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 800971426 2024-05-09T10:51:34.837 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.837 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:34.837 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-05-09T10:51:34.837 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "172.21.15.98:6833", 2024-05-09T10:51:34.837 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 800971426 2024-05-09T10:51:34.837 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:34.837 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-05-09T10:51:34.837 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.837 INFO:teuthology.orchestra.run.smithi098.stdout: "active_addr": "172.21.15.98:6833/800971426", 2024-05-09T10:51:34.837 INFO:teuthology.orchestra.run.smithi098.stdout: "active_change": "2024-05-09T10:49:59.355011+0000", 2024-05-09T10:51:34.837 INFO:teuthology.orchestra.run.smithi098.stdout: "active_mgr_features": 4540701547738038271, 2024-05-09T10:51:34.838 INFO:teuthology.orchestra.run.smithi098.stdout: "available": true, 2024-05-09T10:51:34.838 INFO:teuthology.orchestra.run.smithi098.stdout: "standbys": [ 2024-05-09T10:51:34.838 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:34.838 INFO:teuthology.orchestra.run.smithi098.stdout: "gid": 4105, 2024-05-09T10:51:34.838 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "x", 2024-05-09T10:51:34.838 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr_features": 4540701547738038271, 2024-05-09T10:51:34.838 INFO:teuthology.orchestra.run.smithi098.stdout: "available_modules": [ 2024-05-09T10:51:34.838 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:34.838 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "alerts", 2024-05-09T10:51:34.838 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:34.838 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:34.838 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:34.838 INFO:teuthology.orchestra.run.smithi098.stdout: "interval": { 2024-05-09T10:51:34.838 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "interval", 2024-05-09T10:51:34.839 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:34.839 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.839 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.839 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "60", 2024-05-09T10:51:34.839 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.839 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.839 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.839 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T10:51:34.839 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.839 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.839 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.839 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.839 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:34.839 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:34.839 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.839 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.839 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.840 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.840 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.840 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.840 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:34.840 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:34.840 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:34.840 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:34.840 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:34.840 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:34.840 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:34.840 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:34.840 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.840 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.840 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.840 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.840 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.841 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:34.841 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:34.841 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.841 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.841 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.841 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.841 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.841 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.841 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.841 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.841 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.841 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.841 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.841 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.841 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:34.841 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:34.841 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.842 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.842 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.842 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:34.842 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.842 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.842 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:34.842 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:34.842 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:34.842 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:34.842 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:34.842 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:34.842 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:34.842 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:34.842 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.842 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.843 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.843 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.843 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.843 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:34.843 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:34.843 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.843 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.843 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.843 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.843 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.843 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.843 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.843 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.843 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.843 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.843 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.844 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.844 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_destination": { 2024-05-09T10:51:34.844 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_destination", 2024-05-09T10:51:34.844 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.844 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.844 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.844 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.844 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.844 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.844 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.844 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Email address to send alerts to", 2024-05-09T10:51:34.844 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.844 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.844 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.844 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.845 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_from_name": { 2024-05-09T10:51:34.845 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_from_name", 2024-05-09T10:51:34.845 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.845 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.845 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.845 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "Ceph", 2024-05-09T10:51:34.845 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.845 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.845 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.845 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Email From: name", 2024-05-09T10:51:34.845 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.845 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.845 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.845 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.845 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_host": { 2024-05-09T10:51:34.845 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_host", 2024-05-09T10:51:34.846 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.846 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.846 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.846 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.846 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.846 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.846 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.846 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "SMTP server", 2024-05-09T10:51:34.846 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.846 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.846 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.846 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.846 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_password": { 2024-05-09T10:51:34.846 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_password", 2024-05-09T10:51:34.846 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.846 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.847 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.847 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.847 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.847 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.847 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.847 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Password to authenticate with", 2024-05-09T10:51:34.847 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.847 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.847 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.847 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.847 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_port": { 2024-05-09T10:51:34.847 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_port", 2024-05-09T10:51:34.847 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:34.847 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.847 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.847 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "465", 2024-05-09T10:51:34.847 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.847 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.848 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.848 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "SMTP port", 2024-05-09T10:51:34.848 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.848 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.848 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.848 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.848 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_sender": { 2024-05-09T10:51:34.848 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_sender", 2024-05-09T10:51:34.848 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.848 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.848 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.848 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.848 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.848 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.848 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.848 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "SMTP envelope sender", 2024-05-09T10:51:34.848 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.848 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.849 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.849 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.849 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_ssl": { 2024-05-09T10:51:34.849 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_ssl", 2024-05-09T10:51:34.849 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.849 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.849 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.849 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.849 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.849 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.849 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.849 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T10:51:34.850 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.850 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.850 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.850 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.850 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_user": { 2024-05-09T10:51:34.850 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_user", 2024-05-09T10:51:34.850 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.850 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.850 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.850 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.850 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.850 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.850 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.850 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "User to authenticate as", 2024-05-09T10:51:34.850 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.851 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.851 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.851 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:34.851 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:34.851 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.851 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:34.851 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "balancer", 2024-05-09T10:51:34.851 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:34.851 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:34.851 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:34.851 INFO:teuthology.orchestra.run.smithi098.stdout: "active": { 2024-05-09T10:51:34.851 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "active", 2024-05-09T10:51:34.851 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.851 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.851 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.851 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.852 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.852 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.852 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.852 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T10:51:34.852 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.852 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.852 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.852 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.852 INFO:teuthology.orchestra.run.smithi098.stdout: "begin_time": { 2024-05-09T10:51:34.852 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "begin_time", 2024-05-09T10:51:34.852 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.852 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.852 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.852 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0000", 2024-05-09T10:51:34.852 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.852 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.853 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.853 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T10:51:34.853 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T10:51:34.853 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.853 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.853 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.853 INFO:teuthology.orchestra.run.smithi098.stdout: "begin_weekday": { 2024-05-09T10:51:34.853 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "begin_weekday", 2024-05-09T10:51:34.853 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "uint", 2024-05-09T10:51:34.853 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.853 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.853 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0", 2024-05-09T10:51:34.853 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "0", 2024-05-09T10:51:34.853 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "6", 2024-05-09T10:51:34.853 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.853 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T10:51:34.854 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T10:51:34.854 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.854 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.854 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.854 INFO:teuthology.orchestra.run.smithi098.stdout: "crush_compat_max_iterations": { 2024-05-09T10:51:34.854 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "crush_compat_max_iterations", 2024-05-09T10:51:34.854 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "uint", 2024-05-09T10:51:34.854 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.854 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.854 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "25", 2024-05-09T10:51:34.854 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "1", 2024-05-09T10:51:34.854 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "250", 2024-05-09T10:51:34.854 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.854 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T10:51:34.854 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.854 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.855 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.855 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.855 INFO:teuthology.orchestra.run.smithi098.stdout: "crush_compat_metrics": { 2024-05-09T10:51:34.855 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "crush_compat_metrics", 2024-05-09T10:51:34.855 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.855 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.855 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.855 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T10:51:34.855 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.855 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.855 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.855 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T10:51:34.855 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T10:51:34.856 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.856 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.856 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.856 INFO:teuthology.orchestra.run.smithi098.stdout: "crush_compat_step": { 2024-05-09T10:51:34.856 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "crush_compat_step", 2024-05-09T10:51:34.856 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-05-09T10:51:34.856 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.856 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.856 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0.5", 2024-05-09T10:51:34.856 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "0.001", 2024-05-09T10:51:34.856 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "0.999", 2024-05-09T10:51:34.856 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.856 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "aggressiveness of optimization", 2024-05-09T10:51:34.856 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T10:51:34.856 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.857 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.857 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.857 INFO:teuthology.orchestra.run.smithi098.stdout: "end_time": { 2024-05-09T10:51:34.857 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "end_time", 2024-05-09T10:51:34.857 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.857 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.857 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.857 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "2359", 2024-05-09T10:51:34.857 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.857 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.857 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.857 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T10:51:34.857 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T10:51:34.857 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.857 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.857 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.858 INFO:teuthology.orchestra.run.smithi098.stdout: "end_weekday": { 2024-05-09T10:51:34.858 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "end_weekday", 2024-05-09T10:51:34.858 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "uint", 2024-05-09T10:51:34.858 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.858 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.858 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0", 2024-05-09T10:51:34.858 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "0", 2024-05-09T10:51:34.858 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "6", 2024-05-09T10:51:34.858 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.858 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T10:51:34.858 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T10:51:34.858 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.858 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.858 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.859 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:34.859 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:34.859 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.859 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.859 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.859 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.859 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.859 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.859 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:34.859 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:34.859 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:34.859 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:34.859 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:34.859 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:34.859 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:34.859 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:34.860 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.860 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.860 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.860 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.860 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.860 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:34.860 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:34.860 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.860 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.860 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.860 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.860 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.860 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.860 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.860 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.861 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.861 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.861 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.861 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.861 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:34.861 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:34.861 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.861 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.861 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.861 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:34.861 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.861 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.861 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:34.861 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:34.861 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:34.862 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:34.862 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:34.862 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:34.862 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:34.862 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:34.862 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.862 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.862 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.862 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.862 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.862 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:34.862 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:34.862 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.862 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.862 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.863 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.863 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.863 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.863 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.863 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.863 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.863 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.863 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.863 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.863 INFO:teuthology.orchestra.run.smithi098.stdout: "min_score": { 2024-05-09T10:51:34.863 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "min_score", 2024-05-09T10:51:34.863 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-05-09T10:51:34.863 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.863 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.863 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0", 2024-05-09T10:51:34.863 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.864 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.864 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.864 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T10:51:34.864 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.864 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.864 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.864 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.864 INFO:teuthology.orchestra.run.smithi098.stdout: "mode": { 2024-05-09T10:51:34.864 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "mode", 2024-05-09T10:51:34.864 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.864 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.864 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.864 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "upmap", 2024-05-09T10:51:34.864 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.864 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.864 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:34.864 INFO:teuthology.orchestra.run.smithi098.stdout: "crush-compat", 2024-05-09T10:51:34.865 INFO:teuthology.orchestra.run.smithi098.stdout: "none", 2024-05-09T10:51:34.865 INFO:teuthology.orchestra.run.smithi098.stdout: "read", 2024-05-09T10:51:34.865 INFO:teuthology.orchestra.run.smithi098.stdout: "upmap", 2024-05-09T10:51:34.865 INFO:teuthology.orchestra.run.smithi098.stdout: "upmap-read" 2024-05-09T10:51:34.865 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:34.865 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Balancer mode", 2024-05-09T10:51:34.865 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.865 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.865 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.865 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.865 INFO:teuthology.orchestra.run.smithi098.stdout: "pool_ids": { 2024-05-09T10:51:34.865 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "pool_ids", 2024-05-09T10:51:34.865 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.865 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.865 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.865 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.866 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.866 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.866 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.866 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T10:51:34.866 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.866 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.866 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.866 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.866 INFO:teuthology.orchestra.run.smithi098.stdout: "sleep_interval": { 2024-05-09T10:51:34.866 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "sleep_interval", 2024-05-09T10:51:34.866 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:34.866 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.866 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.866 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "60", 2024-05-09T10:51:34.866 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.867 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.867 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.867 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T10:51:34.867 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.867 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.867 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.867 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.867 INFO:teuthology.orchestra.run.smithi098.stdout: "upmap_max_deviation": { 2024-05-09T10:51:34.867 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "upmap_max_deviation", 2024-05-09T10:51:34.867 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:34.867 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.867 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.867 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "5", 2024-05-09T10:51:34.867 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "1", 2024-05-09T10:51:34.867 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.867 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.868 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T10:51:34.868 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T10:51:34.868 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.868 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.868 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.868 INFO:teuthology.orchestra.run.smithi098.stdout: "upmap_max_optimizations": { 2024-05-09T10:51:34.868 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "upmap_max_optimizations", 2024-05-09T10:51:34.868 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "uint", 2024-05-09T10:51:34.868 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.868 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.868 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-05-09T10:51:34.868 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.868 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.868 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.869 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T10:51:34.869 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.869 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.869 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.869 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:34.869 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:34.869 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.869 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:34.869 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cephadm", 2024-05-09T10:51:34.869 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:34.869 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:34.869 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:34.870 INFO:teuthology.orchestra.run.smithi098.stdout: "agent_down_multiplier": { 2024-05-09T10:51:34.870 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "agent_down_multiplier", 2024-05-09T10:51:34.870 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-05-09T10:51:34.870 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.870 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.870 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "3.0", 2024-05-09T10:51:34.870 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.870 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.870 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.870 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T10:51:34.870 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.870 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.870 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.871 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.871 INFO:teuthology.orchestra.run.smithi098.stdout: "agent_refresh_rate": { 2024-05-09T10:51:34.871 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "agent_refresh_rate", 2024-05-09T10:51:34.871 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:34.871 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.871 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.871 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "20", 2024-05-09T10:51:34.871 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.871 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.871 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.871 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T10:51:34.871 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.871 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.871 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.872 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.872 INFO:teuthology.orchestra.run.smithi098.stdout: "agent_starting_port": { 2024-05-09T10:51:34.872 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "agent_starting_port", 2024-05-09T10:51:34.872 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:34.872 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.872 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.872 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "4721", 2024-05-09T10:51:34.872 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.872 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.872 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.872 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T10:51:34.872 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.872 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.872 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.873 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.873 INFO:teuthology.orchestra.run.smithi098.stdout: "allow_ptrace": { 2024-05-09T10:51:34.873 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "allow_ptrace", 2024-05-09T10:51:34.873 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.873 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.873 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.873 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.873 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.873 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.873 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.873 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T10:51:34.873 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T10:51:34.874 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.874 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.874 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.874 INFO:teuthology.orchestra.run.smithi098.stdout: "autotune_interval": { 2024-05-09T10:51:34.874 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "autotune_interval", 2024-05-09T10:51:34.874 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:34.874 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.874 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.874 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "600", 2024-05-09T10:51:34.874 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.874 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.874 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.874 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T10:51:34.875 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.875 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.875 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.875 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.875 INFO:teuthology.orchestra.run.smithi098.stdout: "autotune_memory_target_ratio": { 2024-05-09T10:51:34.875 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T10:51:34.875 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-05-09T10:51:34.875 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.875 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.875 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0.7", 2024-05-09T10:51:34.875 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.875 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.875 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.875 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T10:51:34.875 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.876 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.876 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.876 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.876 INFO:teuthology.orchestra.run.smithi098.stdout: "cephadm_log_destination": { 2024-05-09T10:51:34.876 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cephadm_log_destination", 2024-05-09T10:51:34.876 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.876 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.876 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.876 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.876 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.876 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.876 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:34.876 INFO:teuthology.orchestra.run.smithi098.stdout: "file", 2024-05-09T10:51:34.876 INFO:teuthology.orchestra.run.smithi098.stdout: "file,syslog", 2024-05-09T10:51:34.877 INFO:teuthology.orchestra.run.smithi098.stdout: "syslog" 2024-05-09T10:51:34.877 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:34.877 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T10:51:34.877 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.877 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.877 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.877 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.877 INFO:teuthology.orchestra.run.smithi098.stdout: "cgroups_split": { 2024-05-09T10:51:34.877 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cgroups_split", 2024-05-09T10:51:34.877 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.877 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.877 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.877 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.877 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.877 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.878 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.878 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T10:51:34.878 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.878 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.878 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.878 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.878 INFO:teuthology.orchestra.run.smithi098.stdout: "config_checks_enabled": { 2024-05-09T10:51:34.878 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "config_checks_enabled", 2024-05-09T10:51:34.878 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.878 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.878 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.878 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.878 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.878 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.879 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.879 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T10:51:34.879 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.879 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.879 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.879 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.879 INFO:teuthology.orchestra.run.smithi098.stdout: "config_dashboard": { 2024-05-09T10:51:34.879 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "config_dashboard", 2024-05-09T10:51:34.879 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.879 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.879 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.879 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.879 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.879 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.880 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.880 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T10:51:34.880 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.880 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.880 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.880 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.880 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_alertmanager": { 2024-05-09T10:51:34.880 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_alertmanager", 2024-05-09T10:51:34.880 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.880 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.880 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.880 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T10:51:34.880 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.880 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.880 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.881 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Prometheus container image", 2024-05-09T10:51:34.881 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.881 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.881 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.881 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.881 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_base": { 2024-05-09T10:51:34.881 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_base", 2024-05-09T10:51:34.881 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.881 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.881 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.881 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T10:51:34.881 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.881 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.881 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.881 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Container image name, without the tag", 2024-05-09T10:51:34.881 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.882 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.882 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.882 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.882 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_elasticsearch": { 2024-05-09T10:51:34.882 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_elasticsearch", 2024-05-09T10:51:34.882 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.882 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.882 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.882 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T10:51:34.882 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.882 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.882 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.882 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "elasticsearch container image", 2024-05-09T10:51:34.882 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.882 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.882 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.883 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.883 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_grafana": { 2024-05-09T10:51:34.883 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_grafana", 2024-05-09T10:51:34.883 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.883 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.883 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.883 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-05-09T10:51:34.883 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.883 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.883 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.883 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Prometheus container image", 2024-05-09T10:51:34.883 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.883 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.883 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.883 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.883 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_haproxy": { 2024-05-09T10:51:34.883 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_haproxy", 2024-05-09T10:51:34.883 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.884 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.884 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.884 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T10:51:34.884 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.884 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.884 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.884 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "HAproxy container image", 2024-05-09T10:51:34.884 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.884 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.884 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.884 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.884 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_jaeger_agent": { 2024-05-09T10:51:34.884 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_jaeger_agent", 2024-05-09T10:51:34.884 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.884 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.884 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.884 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T10:51:34.885 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.885 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.885 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.885 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Jaeger agent container image", 2024-05-09T10:51:34.885 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.885 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.885 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.885 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.885 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_jaeger_collector": { 2024-05-09T10:51:34.885 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_jaeger_collector", 2024-05-09T10:51:34.885 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.885 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.885 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.885 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T10:51:34.885 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.885 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.885 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.886 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Jaeger collector container image", 2024-05-09T10:51:34.886 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.886 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.886 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.886 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.886 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_jaeger_query": { 2024-05-09T10:51:34.886 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_jaeger_query", 2024-05-09T10:51:34.886 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.886 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.886 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.886 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T10:51:34.886 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.886 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.886 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.887 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Jaeger query container image", 2024-05-09T10:51:34.887 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.887 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.887 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.887 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.887 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_keepalived": { 2024-05-09T10:51:34.887 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_keepalived", 2024-05-09T10:51:34.887 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.887 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.887 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.887 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T10:51:34.887 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.887 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.887 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.887 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Keepalived container image", 2024-05-09T10:51:34.888 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.888 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.888 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.888 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.888 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_loki": { 2024-05-09T10:51:34.888 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_loki", 2024-05-09T10:51:34.888 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.888 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.888 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.888 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T10:51:34.888 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.888 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.888 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.888 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Loki container image", 2024-05-09T10:51:34.888 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.888 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.888 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.888 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.889 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_node_exporter": { 2024-05-09T10:51:34.889 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_node_exporter", 2024-05-09T10:51:34.889 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.889 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.889 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.889 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T10:51:34.889 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.889 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.889 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.889 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Prometheus container image", 2024-05-09T10:51:34.889 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.889 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.889 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.889 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.889 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_nvmeof": { 2024-05-09T10:51:34.889 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_nvmeof", 2024-05-09T10:51:34.889 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.890 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.890 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.890 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-05-09T10:51:34.890 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.890 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.890 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.890 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Nvme-of container image", 2024-05-09T10:51:34.890 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.890 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.890 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.890 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.890 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_prometheus": { 2024-05-09T10:51:34.890 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_prometheus", 2024-05-09T10:51:34.891 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.891 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.891 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.891 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T10:51:34.891 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.891 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.891 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.891 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Prometheus container image", 2024-05-09T10:51:34.891 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.891 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.891 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.891 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.891 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_promtail": { 2024-05-09T10:51:34.891 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_promtail", 2024-05-09T10:51:34.891 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.891 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.892 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.892 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T10:51:34.892 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.892 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.892 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.892 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Promtail container image", 2024-05-09T10:51:34.892 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.892 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.892 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.892 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.892 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_snmp_gateway": { 2024-05-09T10:51:34.892 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_snmp_gateway", 2024-05-09T10:51:34.892 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.892 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.892 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.892 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T10:51:34.893 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.893 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.893 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.893 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "SNMP Gateway container image", 2024-05-09T10:51:34.893 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.893 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.893 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.893 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.893 INFO:teuthology.orchestra.run.smithi098.stdout: "container_init": { 2024-05-09T10:51:34.893 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_init", 2024-05-09T10:51:34.893 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.893 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.893 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.893 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.893 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.893 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.893 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.894 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T10:51:34.894 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.894 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.894 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.894 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.894 INFO:teuthology.orchestra.run.smithi098.stdout: "daemon_cache_timeout": { 2024-05-09T10:51:34.894 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "daemon_cache_timeout", 2024-05-09T10:51:34.894 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:34.894 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.894 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.894 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "600", 2024-05-09T10:51:34.894 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.894 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.894 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.894 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T10:51:34.895 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.895 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.895 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.895 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.895 INFO:teuthology.orchestra.run.smithi098.stdout: "default_cephadm_command_timeout": { 2024-05-09T10:51:34.895 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T10:51:34.895 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:34.895 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.895 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.895 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "900", 2024-05-09T10:51:34.895 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.895 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.895 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.895 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T10:51:34.895 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.895 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.895 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.895 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.896 INFO:teuthology.orchestra.run.smithi098.stdout: "default_registry": { 2024-05-09T10:51:34.896 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "default_registry", 2024-05-09T10:51:34.896 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.896 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.896 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.896 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "docker.io", 2024-05-09T10:51:34.896 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.896 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.896 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.896 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T10:51:34.896 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.896 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.896 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.896 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.896 INFO:teuthology.orchestra.run.smithi098.stdout: "device_cache_timeout": { 2024-05-09T10:51:34.896 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "device_cache_timeout", 2024-05-09T10:51:34.896 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:34.897 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.897 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.897 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "1800", 2024-05-09T10:51:34.897 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.897 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.897 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.897 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "seconds to cache device inventory", 2024-05-09T10:51:34.897 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.897 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.897 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.897 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.897 INFO:teuthology.orchestra.run.smithi098.stdout: "device_enhanced_scan": { 2024-05-09T10:51:34.897 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "device_enhanced_scan", 2024-05-09T10:51:34.897 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.897 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.897 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.898 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.898 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.898 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.898 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.898 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T10:51:34.898 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.898 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.898 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.898 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.898 INFO:teuthology.orchestra.run.smithi098.stdout: "facts_cache_timeout": { 2024-05-09T10:51:34.898 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "facts_cache_timeout", 2024-05-09T10:51:34.898 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:34.898 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.898 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.898 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "60", 2024-05-09T10:51:34.899 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.899 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.899 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.899 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "seconds to cache host facts data", 2024-05-09T10:51:34.899 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.899 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.899 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.899 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.899 INFO:teuthology.orchestra.run.smithi098.stdout: "host_check_interval": { 2024-05-09T10:51:34.899 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "host_check_interval", 2024-05-09T10:51:34.899 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:34.899 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.899 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.899 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "600", 2024-05-09T10:51:34.900 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.900 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.900 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.900 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how frequently to perform a host check", 2024-05-09T10:51:34.900 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.900 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.900 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.900 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.900 INFO:teuthology.orchestra.run.smithi098.stdout: "hw_monitoring": { 2024-05-09T10:51:34.900 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "hw_monitoring", 2024-05-09T10:51:34.900 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.900 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.900 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.900 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.900 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.901 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.901 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.901 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T10:51:34.901 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.901 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.901 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.901 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.901 INFO:teuthology.orchestra.run.smithi098.stdout: "inventory_list_all": { 2024-05-09T10:51:34.901 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "inventory_list_all", 2024-05-09T10:51:34.901 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.901 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.901 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.901 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.901 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.901 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.901 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.902 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T10:51:34.902 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.902 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.902 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.902 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.902 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:34.902 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:34.902 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.902 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.902 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.902 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.902 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.902 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.902 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:34.902 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:34.903 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:34.903 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:34.903 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:34.903 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:34.903 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:34.903 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:34.903 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.903 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.903 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.903 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.903 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.903 INFO:teuthology.orchestra.run.smithi098.stdout: "log_refresh_metadata": { 2024-05-09T10:51:34.903 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_refresh_metadata", 2024-05-09T10:51:34.903 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.903 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.904 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.904 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.904 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.904 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.904 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.904 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T10:51:34.904 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.904 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.904 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.904 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.904 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:34.904 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:34.904 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.904 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.904 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.905 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.905 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.905 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.905 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.905 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T10:51:34.905 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.905 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.905 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.905 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.905 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:34.905 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:34.905 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.905 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.906 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.906 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:34.906 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.906 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.906 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:34.906 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:34.906 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:34.906 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:34.906 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:34.906 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:34.906 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:34.906 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:34.906 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.907 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.907 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.907 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.907 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.907 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:34.907 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:34.907 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.907 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.907 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.907 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.907 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.907 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.907 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.908 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.908 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.908 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.908 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.908 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.908 INFO:teuthology.orchestra.run.smithi098.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T10:51:34.908 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T10:51:34.908 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.908 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.908 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.908 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.908 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.909 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.909 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.909 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T10:51:34.909 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.909 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.909 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.909 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.909 INFO:teuthology.orchestra.run.smithi098.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T10:51:34.909 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T10:51:34.909 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.909 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.909 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.909 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "*", 2024-05-09T10:51:34.909 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.910 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.910 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.910 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T10:51:34.910 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.910 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.910 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.910 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.910 INFO:teuthology.orchestra.run.smithi098.stdout: "max_count_per_host": { 2024-05-09T10:51:34.910 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "max_count_per_host", 2024-05-09T10:51:34.910 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:34.910 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.910 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.910 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-05-09T10:51:34.911 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.911 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.911 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.911 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "max number of daemons per service per host", 2024-05-09T10:51:34.911 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.911 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.911 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.911 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.911 INFO:teuthology.orchestra.run.smithi098.stdout: "max_osd_draining_count": { 2024-05-09T10:51:34.911 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "max_osd_draining_count", 2024-05-09T10:51:34.911 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:34.911 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.911 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.911 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-05-09T10:51:34.911 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.911 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.912 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.912 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T10:51:34.912 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.912 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.912 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.912 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.912 INFO:teuthology.orchestra.run.smithi098.stdout: "migration_current": { 2024-05-09T10:51:34.912 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "migration_current", 2024-05-09T10:51:34.912 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:34.912 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.912 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.912 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.912 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.912 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.912 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.913 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "internal - do not modify", 2024-05-09T10:51:34.913 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.913 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.913 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.913 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.913 INFO:teuthology.orchestra.run.smithi098.stdout: "mode": { 2024-05-09T10:51:34.913 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "mode", 2024-05-09T10:51:34.913 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.913 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.913 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.913 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "root", 2024-05-09T10:51:34.913 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.913 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.913 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:34.913 INFO:teuthology.orchestra.run.smithi098.stdout: "cephadm-package", 2024-05-09T10:51:34.914 INFO:teuthology.orchestra.run.smithi098.stdout: "root" 2024-05-09T10:51:34.914 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:34.914 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T10:51:34.914 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.914 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.914 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.914 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.914 INFO:teuthology.orchestra.run.smithi098.stdout: "oob_default_addr": { 2024-05-09T10:51:34.914 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "oob_default_addr", 2024-05-09T10:51:34.914 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.914 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.914 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.914 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "169.254.1.1", 2024-05-09T10:51:34.914 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.914 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.915 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.915 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T10:51:34.915 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.915 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.915 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.915 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.915 INFO:teuthology.orchestra.run.smithi098.stdout: "prometheus_alerts_path": { 2024-05-09T10:51:34.915 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "prometheus_alerts_path", 2024-05-09T10:51:34.915 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.915 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.915 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.915 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T10:51:34.915 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.915 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.915 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.915 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T10:51:34.916 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.916 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.916 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.916 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.916 INFO:teuthology.orchestra.run.smithi098.stdout: "registry_insecure": { 2024-05-09T10:51:34.916 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "registry_insecure", 2024-05-09T10:51:34.916 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.916 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.916 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.916 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.916 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.916 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.916 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.916 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T10:51:34.916 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.916 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.917 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.917 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.917 INFO:teuthology.orchestra.run.smithi098.stdout: "registry_password": { 2024-05-09T10:51:34.917 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "registry_password", 2024-05-09T10:51:34.917 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.917 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.917 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.917 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.917 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.917 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.917 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.917 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T10:51:34.917 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.917 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.917 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.917 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.918 INFO:teuthology.orchestra.run.smithi098.stdout: "registry_url": { 2024-05-09T10:51:34.918 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "registry_url", 2024-05-09T10:51:34.918 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.918 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.918 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.918 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.918 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.918 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.918 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.918 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T10:51:34.918 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.918 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.918 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.918 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.919 INFO:teuthology.orchestra.run.smithi098.stdout: "registry_username": { 2024-05-09T10:51:34.919 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "registry_username", 2024-05-09T10:51:34.919 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.919 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.919 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.919 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.919 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.919 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.919 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.919 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T10:51:34.919 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.919 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.919 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.919 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.919 INFO:teuthology.orchestra.run.smithi098.stdout: "secure_monitoring_stack": { 2024-05-09T10:51:34.919 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "secure_monitoring_stack", 2024-05-09T10:51:34.920 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.920 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.920 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.920 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.920 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.920 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.920 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.920 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T10:51:34.920 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.920 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.920 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.920 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.920 INFO:teuthology.orchestra.run.smithi098.stdout: "service_discovery_port": { 2024-05-09T10:51:34.920 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "service_discovery_port", 2024-05-09T10:51:34.920 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:34.921 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.921 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.921 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "8765", 2024-05-09T10:51:34.921 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.921 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.921 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.921 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "cephadm service discovery port", 2024-05-09T10:51:34.921 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.921 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.921 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.921 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.921 INFO:teuthology.orchestra.run.smithi098.stdout: "ssh_config_file": { 2024-05-09T10:51:34.921 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ssh_config_file", 2024-05-09T10:51:34.921 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.922 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.922 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.922 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.922 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.922 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.922 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.922 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T10:51:34.922 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.922 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.922 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.922 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.922 INFO:teuthology.orchestra.run.smithi098.stdout: "use_agent": { 2024-05-09T10:51:34.922 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "use_agent", 2024-05-09T10:51:34.922 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.922 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.922 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.923 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.923 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.923 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.923 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.923 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T10:51:34.923 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.923 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.923 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.923 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.923 INFO:teuthology.orchestra.run.smithi098.stdout: "use_repo_digest": { 2024-05-09T10:51:34.923 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "use_repo_digest", 2024-05-09T10:51:34.923 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.923 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.923 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.923 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.924 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.924 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.924 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.924 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T10:51:34.924 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.924 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.924 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.924 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.924 INFO:teuthology.orchestra.run.smithi098.stdout: "warn_on_failed_host_check": { 2024-05-09T10:51:34.924 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "warn_on_failed_host_check", 2024-05-09T10:51:34.924 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.924 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.924 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.924 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.925 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.925 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.925 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.925 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T10:51:34.925 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.925 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.925 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.925 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.925 INFO:teuthology.orchestra.run.smithi098.stdout: "warn_on_stray_daemons": { 2024-05-09T10:51:34.925 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "warn_on_stray_daemons", 2024-05-09T10:51:34.925 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.925 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.925 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.925 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.925 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.925 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.926 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.926 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T10:51:34.926 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.926 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.926 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.926 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.926 INFO:teuthology.orchestra.run.smithi098.stdout: "warn_on_stray_hosts": { 2024-05-09T10:51:34.926 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "warn_on_stray_hosts", 2024-05-09T10:51:34.926 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.926 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.926 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.926 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.926 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.926 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.926 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.927 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T10:51:34.927 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.927 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.927 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.927 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:34.927 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:34.927 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.927 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:34.927 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "crash", 2024-05-09T10:51:34.927 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:34.927 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:34.927 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:34.927 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:34.927 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:34.927 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.928 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.928 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.928 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.928 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.928 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.928 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:34.928 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:34.928 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:34.928 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:34.928 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:34.928 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:34.928 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:34.928 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:34.928 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.929 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.929 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.929 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.929 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.929 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:34.929 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:34.929 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.929 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.929 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.929 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.929 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.929 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.929 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.929 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.930 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.930 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.930 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.930 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.930 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:34.930 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:34.930 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.930 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.930 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.930 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:34.930 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.930 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.930 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:34.930 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:34.930 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:34.931 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:34.931 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:34.931 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:34.931 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:34.931 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:34.931 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.931 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.931 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.931 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.931 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.931 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:34.931 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:34.931 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.931 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.931 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.931 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.932 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.932 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.932 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.932 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.932 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.932 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.932 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.932 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.932 INFO:teuthology.orchestra.run.smithi098.stdout: "retain_interval": { 2024-05-09T10:51:34.932 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "retain_interval", 2024-05-09T10:51:34.932 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:34.932 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.932 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.932 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "31536000", 2024-05-09T10:51:34.932 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.933 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.933 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.933 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T10:51:34.933 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.933 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.933 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.933 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.933 INFO:teuthology.orchestra.run.smithi098.stdout: "warn_recent_interval": { 2024-05-09T10:51:34.933 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "warn_recent_interval", 2024-05-09T10:51:34.933 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:34.933 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.933 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.933 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "1209600", 2024-05-09T10:51:34.933 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.934 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.934 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.934 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T10:51:34.934 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.934 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.934 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.934 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:34.934 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:34.934 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.934 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:34.934 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "dashboard", 2024-05-09T10:51:34.934 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:34.934 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:34.934 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:34.934 INFO:teuthology.orchestra.run.smithi098.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T10:51:34.935 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T10:51:34.935 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:34.935 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.935 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.935 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-05-09T10:51:34.935 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.935 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.935 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.935 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.935 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.935 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.935 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.935 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.935 INFO:teuthology.orchestra.run.smithi098.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T10:51:34.935 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T10:51:34.935 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.935 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.936 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.936 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.936 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.936 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.936 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.936 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.936 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.936 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.936 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.936 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.936 INFO:teuthology.orchestra.run.smithi098.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T10:51:34.936 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T10:51:34.936 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.936 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.936 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.936 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.937 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.937 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.937 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.937 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.937 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.937 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.937 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.937 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.937 INFO:teuthology.orchestra.run.smithi098.stdout: "AUDIT_API_ENABLED": { 2024-05-09T10:51:34.937 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T10:51:34.937 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.937 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.937 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.937 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.937 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.937 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.938 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.938 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.938 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.938 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.938 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.938 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.938 INFO:teuthology.orchestra.run.smithi098.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T10:51:34.938 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T10:51:34.938 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.938 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.938 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.938 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.938 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.938 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.939 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.939 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.939 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.939 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.939 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.939 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.939 INFO:teuthology.orchestra.run.smithi098.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T10:51:34.939 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T10:51:34.939 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.939 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.939 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.939 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.939 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.939 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.940 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.940 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.940 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.940 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.940 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.940 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.940 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T10:51:34.940 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T10:51:34.940 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.940 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.940 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.940 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.940 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.940 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.940 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.941 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.941 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.941 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.941 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.941 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.941 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T10:51:34.941 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T10:51:34.941 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.941 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.941 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.941 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.941 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.941 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.941 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.941 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.941 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.942 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.942 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.942 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.942 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T10:51:34.942 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T10:51:34.942 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.942 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.942 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.942 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.942 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.942 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.942 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.942 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.943 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.943 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.943 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.943 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.943 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T10:51:34.943 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T10:51:34.943 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.943 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.943 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.943 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.943 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.943 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.943 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.943 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.943 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.944 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.944 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.944 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.944 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T10:51:34.944 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T10:51:34.944 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.944 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.944 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.944 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.944 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.944 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.944 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.944 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.944 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.944 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.945 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.945 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.945 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T10:51:34.945 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T10:51:34.945 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.945 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.945 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.945 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.945 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.945 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.945 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.945 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.945 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.945 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.945 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.945 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.946 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T10:51:34.946 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T10:51:34.946 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.946 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.946 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.946 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.946 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.946 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.946 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.946 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.946 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.946 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.946 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.947 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.947 INFO:teuthology.orchestra.run.smithi098.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T10:51:34.947 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T10:51:34.947 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.947 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.947 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.947 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.947 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.947 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.947 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.947 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.947 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.947 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.947 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.947 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.947 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T10:51:34.947 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T10:51:34.948 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.948 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.948 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.948 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "admin", 2024-05-09T10:51:34.948 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.948 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.948 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.948 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.948 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.948 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.948 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.948 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.948 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T10:51:34.948 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T10:51:34.948 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.948 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.948 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.949 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.949 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.949 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.949 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.949 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.949 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.949 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.949 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.949 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.949 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_API_URL": { 2024-05-09T10:51:34.949 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_API_URL", 2024-05-09T10:51:34.949 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.949 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.949 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.949 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.949 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.950 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.950 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.950 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.950 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.950 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.950 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.950 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.950 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T10:51:34.950 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T10:51:34.950 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.950 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.950 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.950 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "admin", 2024-05-09T10:51:34.950 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.950 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.950 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.950 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.951 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.951 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.951 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.951 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.951 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T10:51:34.951 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T10:51:34.951 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.951 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.951 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.951 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.951 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.951 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.951 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.951 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.951 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.951 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.952 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.952 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.952 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T10:51:34.952 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T10:51:34.952 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.952 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.952 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.952 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.952 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.952 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.952 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.952 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.952 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.952 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.952 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.952 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.952 INFO:teuthology.orchestra.run.smithi098.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T10:51:34.953 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T10:51:34.953 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.953 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.953 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.953 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.953 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.953 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.953 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.953 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.953 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.953 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.953 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.953 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.954 INFO:teuthology.orchestra.run.smithi098.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T10:51:34.954 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T10:51:34.954 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.954 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.954 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.954 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.954 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.954 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.954 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.954 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.954 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.954 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.954 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.955 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.955 INFO:teuthology.orchestra.run.smithi098.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T10:51:34.955 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T10:51:34.955 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.955 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.955 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.955 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.955 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.955 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.955 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.955 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.955 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.955 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.955 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.955 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.955 INFO:teuthology.orchestra.run.smithi098.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T10:51:34.956 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T10:51:34.956 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.956 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.956 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.956 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.956 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.956 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.956 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.956 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.956 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.956 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.956 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.956 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.956 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T10:51:34.956 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T10:51:34.956 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.956 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.956 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.957 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.957 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.957 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.957 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.957 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.957 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.957 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.957 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.957 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.957 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T10:51:34.957 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T10:51:34.957 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.957 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.957 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.957 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.957 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.957 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.958 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.958 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.958 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.958 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.958 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.958 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.958 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T10:51:34.958 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T10:51:34.958 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.958 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.958 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.958 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.958 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.958 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.958 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.958 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.959 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.959 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.959 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.959 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.959 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T10:51:34.959 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T10:51:34.959 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.959 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.959 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.959 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.959 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.959 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.959 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.959 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.959 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.959 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.959 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.960 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.960 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T10:51:34.960 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T10:51:34.960 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.960 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.960 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.960 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.960 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.960 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.960 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.960 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.960 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.960 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.960 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.960 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.960 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T10:51:34.960 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T10:51:34.960 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.961 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.961 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.961 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.961 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.961 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.961 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.961 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.961 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.961 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.961 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.961 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.961 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T10:51:34.961 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T10:51:34.961 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.961 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.961 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.961 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.962 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.962 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.962 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.962 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.962 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.962 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.962 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.962 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.962 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T10:51:34.962 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T10:51:34.962 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.962 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.962 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.962 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.962 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.962 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.963 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.963 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.963 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.963 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.963 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.963 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.963 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T10:51:34.963 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T10:51:34.963 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.963 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.963 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.963 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T10:51:34.963 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.963 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.963 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.963 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.963 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.964 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.964 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.964 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.964 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T10:51:34.964 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T10:51:34.964 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:34.964 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.964 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.964 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-05-09T10:51:34.964 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.964 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.964 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.964 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.964 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.964 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.964 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.964 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.965 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T10:51:34.965 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T10:51:34.965 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:34.965 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.965 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.965 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "8", 2024-05-09T10:51:34.965 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.965 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.965 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.965 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.965 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.965 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.965 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.965 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.965 INFO:teuthology.orchestra.run.smithi098.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T10:51:34.965 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T10:51:34.965 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:34.966 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.966 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.966 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "45", 2024-05-09T10:51:34.966 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.966 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.966 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.966 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.966 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.966 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.966 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.966 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.966 INFO:teuthology.orchestra.run.smithi098.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T10:51:34.966 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T10:51:34.966 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.966 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.966 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.967 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.967 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.967 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.967 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.967 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.967 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.967 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.967 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.967 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.967 INFO:teuthology.orchestra.run.smithi098.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T10:51:34.967 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T10:51:34.967 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.967 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.967 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.967 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "admin", 2024-05-09T10:51:34.967 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.967 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.968 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.968 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.968 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.968 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.968 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.968 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.968 INFO:teuthology.orchestra.run.smithi098.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T10:51:34.968 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T10:51:34.968 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.968 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.968 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.968 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.968 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.968 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.968 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.968 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.969 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.969 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.969 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.969 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.969 INFO:teuthology.orchestra.run.smithi098.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T10:51:34.969 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T10:51:34.969 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.969 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.969 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.969 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.969 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.969 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.969 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.969 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.969 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.970 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.970 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.970 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.970 INFO:teuthology.orchestra.run.smithi098.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T10:51:34.970 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T10:51:34.970 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:34.970 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.970 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.970 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0", 2024-05-09T10:51:34.970 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.970 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.970 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.970 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.971 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.971 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.971 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.971 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.971 INFO:teuthology.orchestra.run.smithi098.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T10:51:34.971 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T10:51:34.971 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:34.971 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.971 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.971 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-05-09T10:51:34.971 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.971 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.971 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.971 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.971 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.972 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.972 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.972 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.972 INFO:teuthology.orchestra.run.smithi098.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T10:51:34.972 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T10:51:34.972 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:34.972 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.972 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.972 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "5", 2024-05-09T10:51:34.972 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.972 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.972 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.972 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.972 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.973 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.973 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.973 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.973 INFO:teuthology.orchestra.run.smithi098.stdout: "cross_origin_url": { 2024-05-09T10:51:34.973 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cross_origin_url", 2024-05-09T10:51:34.973 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.973 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.973 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.973 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.973 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.973 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.973 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.973 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.973 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.973 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.974 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.974 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.974 INFO:teuthology.orchestra.run.smithi098.stdout: "crt_file": { 2024-05-09T10:51:34.974 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "crt_file", 2024-05-09T10:51:34.974 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.974 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.974 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.974 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.974 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.974 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.974 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.974 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.974 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.974 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.974 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.975 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.975 INFO:teuthology.orchestra.run.smithi098.stdout: "debug": { 2024-05-09T10:51:34.975 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "debug", 2024-05-09T10:51:34.975 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.975 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.975 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.975 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.975 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.975 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.975 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.975 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Enable/disable debug options", 2024-05-09T10:51:34.975 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.975 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.975 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.975 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.976 INFO:teuthology.orchestra.run.smithi098.stdout: "jwt_token_ttl": { 2024-05-09T10:51:34.976 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "jwt_token_ttl", 2024-05-09T10:51:34.976 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:34.976 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.976 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.976 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "28800", 2024-05-09T10:51:34.976 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.976 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.976 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.976 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.976 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.976 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.976 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.976 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.977 INFO:teuthology.orchestra.run.smithi098.stdout: "key_file": { 2024-05-09T10:51:34.977 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "key_file", 2024-05-09T10:51:34.977 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.977 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.977 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.977 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.977 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.977 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.977 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.977 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.977 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.977 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.977 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.977 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.978 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:34.978 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:34.978 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.978 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.978 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.978 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.978 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.978 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.978 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:34.978 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:34.978 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:34.978 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:34.978 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:34.978 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:34.978 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:34.979 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:34.979 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.979 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.979 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.979 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.979 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.979 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:34.979 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:34.979 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.979 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.979 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.979 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.979 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.980 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.980 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.980 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.980 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.980 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.980 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.980 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:34.980 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:34.980 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.980 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.980 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.980 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:34.980 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.981 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.981 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:34.981 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:34.981 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:34.981 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:34.981 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:34.981 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:34.981 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:34.981 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:34.981 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.981 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.981 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.981 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.981 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.981 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:34.982 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:34.982 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.982 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.982 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.982 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.982 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.982 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.982 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.982 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.982 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.982 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.982 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.982 INFO:teuthology.orchestra.run.smithi098.stdout: "motd": { 2024-05-09T10:51:34.983 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "motd", 2024-05-09T10:51:34.983 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.983 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.983 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.983 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.983 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.983 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.983 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.983 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "The message of the day", 2024-05-09T10:51:34.983 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.983 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.983 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.983 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.983 INFO:teuthology.orchestra.run.smithi098.stdout: "redirect_resolve_ip_addr": { 2024-05-09T10:51:34.983 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T10:51:34.983 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.984 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.984 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.984 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.984 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.984 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.984 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.984 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.984 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.984 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.984 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.984 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.984 INFO:teuthology.orchestra.run.smithi098.stdout: "server_addr": { 2024-05-09T10:51:34.984 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_addr", 2024-05-09T10:51:34.984 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.984 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.984 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.984 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "::", 2024-05-09T10:51:34.984 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.985 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.985 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.985 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.985 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.985 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.985 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.985 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.985 INFO:teuthology.orchestra.run.smithi098.stdout: "server_port": { 2024-05-09T10:51:34.985 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_port", 2024-05-09T10:51:34.985 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:34.985 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.985 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.985 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "8080", 2024-05-09T10:51:34.985 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.985 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.985 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.985 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.986 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.986 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.986 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.986 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.986 INFO:teuthology.orchestra.run.smithi098.stdout: "ssl": { 2024-05-09T10:51:34.986 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ssl", 2024-05-09T10:51:34.986 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.986 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.986 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.986 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.986 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.986 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.986 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.986 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.986 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.987 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.987 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.987 INFO:teuthology.orchestra.run.smithi098.stdout: "ssl_server_port": { 2024-05-09T10:51:34.987 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ssl_server_port", 2024-05-09T10:51:34.987 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:34.987 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.987 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.987 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "8443", 2024-05-09T10:51:34.987 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.987 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.987 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.987 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.987 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.987 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.988 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.988 INFO:teuthology.orchestra.run.smithi098.stdout: "standby_behaviour": { 2024-05-09T10:51:34.988 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "standby_behaviour", 2024-05-09T10:51:34.988 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.988 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.988 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.988 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "redirect", 2024-05-09T10:51:34.988 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.988 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.988 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:34.988 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:34.988 INFO:teuthology.orchestra.run.smithi098.stdout: "redirect" 2024-05-09T10:51:34.988 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:34.988 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.989 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.989 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.989 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.989 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.989 INFO:teuthology.orchestra.run.smithi098.stdout: "standby_error_status_code": { 2024-05-09T10:51:34.989 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "standby_error_status_code", 2024-05-09T10:51:34.989 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:34.989 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.989 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.989 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "500", 2024-05-09T10:51:34.989 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "400", 2024-05-09T10:51:34.989 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "599", 2024-05-09T10:51:34.989 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.989 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.989 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.990 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.990 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.990 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.990 INFO:teuthology.orchestra.run.smithi098.stdout: "url_prefix": { 2024-05-09T10:51:34.990 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "url_prefix", 2024-05-09T10:51:34.990 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.990 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.990 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:34.990 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.990 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.990 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.990 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.990 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.991 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.991 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.991 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.991 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:34.991 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:34.991 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.991 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:34.991 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "devicehealth", 2024-05-09T10:51:34.991 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:34.991 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:34.991 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:34.991 INFO:teuthology.orchestra.run.smithi098.stdout: "enable_monitoring": { 2024-05-09T10:51:34.991 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "enable_monitoring", 2024-05-09T10:51:34.991 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.991 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.991 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.991 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:34.992 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.992 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.992 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.992 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "monitor device health metrics", 2024-05-09T10:51:34.992 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.992 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.992 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.992 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.992 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:34.992 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:34.992 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.992 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.992 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.992 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:34.992 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.993 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.993 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:34.993 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:34.993 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:34.993 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:34.993 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:34.993 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:34.993 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:34.993 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:34.993 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.993 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.993 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.993 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.993 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.993 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:34.993 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:34.993 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.994 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.994 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.994 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.994 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.994 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.994 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.994 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.994 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.994 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.994 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.994 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.994 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:34.994 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:34.994 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.994 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.994 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.995 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:34.995 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.995 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.995 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:34.995 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:34.995 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:34.995 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:34.995 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:34.995 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:34.995 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:34.995 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:34.995 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.995 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.995 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.995 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.996 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:34.996 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:34.996 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:34.996 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.996 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.996 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:34.996 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.996 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.996 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.996 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:34.996 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.996 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.996 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.996 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.996 INFO:teuthology.orchestra.run.smithi098.stdout: "mark_out_threshold": { 2024-05-09T10:51:34.996 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "mark_out_threshold", 2024-05-09T10:51:34.997 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:34.997 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.997 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.997 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "2419200", 2024-05-09T10:51:34.997 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.997 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.997 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.997 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T10:51:34.997 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.997 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.997 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.997 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.997 INFO:teuthology.orchestra.run.smithi098.stdout: "pool_name": { 2024-05-09T10:51:34.997 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "pool_name", 2024-05-09T10:51:34.997 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:34.997 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.998 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.998 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "device_health_metrics", 2024-05-09T10:51:34.998 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.998 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.998 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.998 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T10:51:34.998 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.998 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.998 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.998 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.998 INFO:teuthology.orchestra.run.smithi098.stdout: "retention_period": { 2024-05-09T10:51:34.998 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "retention_period", 2024-05-09T10:51:34.998 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:34.998 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.998 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.999 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "15552000", 2024-05-09T10:51:34.999 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:34.999 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:34.999 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:34.999 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how long to retain device health metrics", 2024-05-09T10:51:34.999 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:34.999 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:34.999 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:34.999 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:34.999 INFO:teuthology.orchestra.run.smithi098.stdout: "scrape_frequency": { 2024-05-09T10:51:34.999 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "scrape_frequency", 2024-05-09T10:51:34.999 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:34.999 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:34.999 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:34.999 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "86400", 2024-05-09T10:51:35.000 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.000 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.000 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.000 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T10:51:35.000 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.000 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.000 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.000 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.000 INFO:teuthology.orchestra.run.smithi098.stdout: "self_heal": { 2024-05-09T10:51:35.000 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "self_heal", 2024-05-09T10:51:35.000 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.000 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.000 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.000 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.000 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.000 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.000 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.001 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T10:51:35.001 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.001 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.001 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.001 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.001 INFO:teuthology.orchestra.run.smithi098.stdout: "sleep_interval": { 2024-05-09T10:51:35.001 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "sleep_interval", 2024-05-09T10:51:35.001 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.001 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.001 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.001 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "600", 2024-05-09T10:51:35.001 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.001 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.001 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.001 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T10:51:35.001 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.001 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.002 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.002 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.002 INFO:teuthology.orchestra.run.smithi098.stdout: "warn_threshold": { 2024-05-09T10:51:35.002 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "warn_threshold", 2024-05-09T10:51:35.002 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.002 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.002 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.002 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "7257600", 2024-05-09T10:51:35.002 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.002 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.002 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.002 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T10:51:35.002 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.002 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.003 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.003 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.003 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.003 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.003 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.003 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "diskprediction_local", 2024-05-09T10:51:35.003 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.003 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.003 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.003 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.003 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.003 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.003 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.003 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.003 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.004 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.004 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.004 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.004 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.004 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.004 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.004 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.004 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.004 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.004 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.004 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.004 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.004 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.004 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.004 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.004 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.005 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.005 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.005 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.005 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.005 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.005 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.005 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.005 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.005 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.005 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.005 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.005 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.005 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.005 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.005 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.005 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.006 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.006 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.006 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.006 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.006 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.006 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.006 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.006 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.006 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.006 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.006 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.006 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.006 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.006 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.006 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.007 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.007 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.007 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.007 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.007 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.007 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.007 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.007 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.007 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.007 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.007 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.007 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.007 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.007 INFO:teuthology.orchestra.run.smithi098.stdout: "predict_interval": { 2024-05-09T10:51:35.007 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "predict_interval", 2024-05-09T10:51:35.008 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.008 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.008 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.008 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "86400", 2024-05-09T10:51:35.008 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.008 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.008 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.008 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.008 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.008 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.008 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.008 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.008 INFO:teuthology.orchestra.run.smithi098.stdout: "predictor_model": { 2024-05-09T10:51:35.008 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "predictor_model", 2024-05-09T10:51:35.008 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.008 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.008 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.009 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "prophetstor", 2024-05-09T10:51:35.009 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.009 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.009 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.009 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.009 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.009 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.009 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.009 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.009 INFO:teuthology.orchestra.run.smithi098.stdout: "sleep_interval": { 2024-05-09T10:51:35.009 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "sleep_interval", 2024-05-09T10:51:35.009 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.009 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.009 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.009 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "600", 2024-05-09T10:51:35.010 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.010 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.010 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.010 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.010 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.010 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.010 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.010 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.010 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.010 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.010 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "influx", 2024-05-09T10:51:35.010 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": false, 2024-05-09T10:51:35.011 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "influxdb python module not found", 2024-05-09T10:51:35.011 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.011 INFO:teuthology.orchestra.run.smithi098.stdout: "batch_size": { 2024-05-09T10:51:35.011 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "batch_size", 2024-05-09T10:51:35.011 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.011 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.011 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.011 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "5000", 2024-05-09T10:51:35.011 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.011 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.011 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.011 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T10:51:35.011 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.011 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.011 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.012 INFO:teuthology.orchestra.run.smithi098.stdout: "database": { 2024-05-09T10:51:35.012 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "database", 2024-05-09T10:51:35.012 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.012 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.012 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.012 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "ceph", 2024-05-09T10:51:35.012 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.012 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.012 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.012 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T10:51:35.012 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.012 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.012 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.012 INFO:teuthology.orchestra.run.smithi098.stdout: "hostname": { 2024-05-09T10:51:35.012 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "hostname", 2024-05-09T10:51:35.012 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.013 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.013 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.013 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.013 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.013 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.013 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.013 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "InfluxDB server hostname", 2024-05-09T10:51:35.013 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.013 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.013 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.013 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.013 INFO:teuthology.orchestra.run.smithi098.stdout: "interval": { 2024-05-09T10:51:35.013 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "interval", 2024-05-09T10:51:35.013 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.013 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.013 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.013 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "30", 2024-05-09T10:51:35.014 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "5", 2024-05-09T10:51:35.014 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.014 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.014 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T10:51:35.014 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.014 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.014 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.014 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.014 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.014 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.014 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.014 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.014 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.014 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.014 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.014 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.014 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.015 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.015 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.015 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.015 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.015 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.015 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.015 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.015 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.015 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.015 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.015 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.015 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.015 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.015 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.016 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.016 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.016 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.016 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.016 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.016 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.016 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.016 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.016 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.016 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.016 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.016 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.016 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.016 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.016 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.016 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.016 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.017 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.017 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.017 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.017 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.017 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.017 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.017 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.017 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.017 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.017 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.017 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.017 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.017 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.017 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.017 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.018 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.018 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.018 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.018 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.018 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.018 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.018 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.018 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.018 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.018 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.018 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.018 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.018 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.018 INFO:teuthology.orchestra.run.smithi098.stdout: "password": { 2024-05-09T10:51:35.019 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "password", 2024-05-09T10:51:35.019 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.019 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.019 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.019 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.019 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.019 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.019 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.019 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "password of InfluxDB server user", 2024-05-09T10:51:35.019 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.019 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.019 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.019 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.019 INFO:teuthology.orchestra.run.smithi098.stdout: "port": { 2024-05-09T10:51:35.019 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "port", 2024-05-09T10:51:35.019 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.020 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.020 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.020 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "8086", 2024-05-09T10:51:35.020 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.020 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.020 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.020 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "InfluxDB server port", 2024-05-09T10:51:35.020 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.020 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.020 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.020 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.020 INFO:teuthology.orchestra.run.smithi098.stdout: "ssl": { 2024-05-09T10:51:35.020 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ssl", 2024-05-09T10:51:35.020 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.020 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.020 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.020 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "false", 2024-05-09T10:51:35.020 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.021 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.021 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.021 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T10:51:35.021 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.021 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.021 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.021 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.021 INFO:teuthology.orchestra.run.smithi098.stdout: "threads": { 2024-05-09T10:51:35.021 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "threads", 2024-05-09T10:51:35.021 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.021 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.021 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.021 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "5", 2024-05-09T10:51:35.021 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "1", 2024-05-09T10:51:35.021 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "32", 2024-05-09T10:51:35.021 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.022 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T10:51:35.022 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.022 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.022 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.022 INFO:teuthology.orchestra.run.smithi098.stdout: "username": { 2024-05-09T10:51:35.022 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "username", 2024-05-09T10:51:35.022 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.022 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.022 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.022 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.022 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.022 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.022 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.022 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "username of InfluxDB server user", 2024-05-09T10:51:35.022 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.023 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.023 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.023 INFO:teuthology.orchestra.run.smithi098.stdout: "verify_ssl": { 2024-05-09T10:51:35.023 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "verify_ssl", 2024-05-09T10:51:35.023 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.023 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.023 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.023 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "true", 2024-05-09T10:51:35.023 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.023 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.023 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.023 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T10:51:35.023 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.023 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.023 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.024 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.024 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.024 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.024 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.024 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "insights", 2024-05-09T10:51:35.024 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.024 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.024 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.024 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.024 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.024 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.024 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.024 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.024 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.024 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.024 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.025 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.025 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.025 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.025 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.025 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.025 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.025 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.025 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.025 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.025 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.025 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.025 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.025 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.025 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.025 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.025 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.025 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.026 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.026 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.026 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.026 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.026 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.026 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.026 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.026 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.026 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.026 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.026 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.026 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.026 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.026 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.027 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.027 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.027 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.027 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.027 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.027 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.027 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.027 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.027 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.027 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.027 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.027 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.027 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.027 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.027 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.027 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.028 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.028 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.028 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.028 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.028 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.028 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.028 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.028 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.028 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.028 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.028 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.028 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.028 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.028 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.028 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.028 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.029 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "iostat", 2024-05-09T10:51:35.029 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.029 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.029 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.029 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.029 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.029 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.029 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.029 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.029 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.029 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.029 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.029 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.029 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.029 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.029 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.029 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.030 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.030 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.030 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.030 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.030 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.030 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.030 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.030 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.030 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.030 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.030 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.030 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.030 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.030 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.030 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.030 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.031 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.031 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.031 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.031 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.031 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.031 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.031 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.031 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.031 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.031 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.031 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.031 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.031 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.031 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.031 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.031 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.032 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.032 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.032 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.032 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.032 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.032 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.032 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.032 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.032 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.032 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.032 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.032 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.032 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.032 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.032 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.032 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.032 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.033 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.033 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.033 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.033 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.033 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.033 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.033 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.033 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.033 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.033 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.033 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "localpool", 2024-05-09T10:51:35.033 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.033 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.033 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.033 INFO:teuthology.orchestra.run.smithi098.stdout: "failure_domain": { 2024-05-09T10:51:35.033 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "failure_domain", 2024-05-09T10:51:35.033 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.033 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.034 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.034 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "host", 2024-05-09T10:51:35.034 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.034 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.034 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.034 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "failure domain for any created local pool", 2024-05-09T10:51:35.034 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T10:51:35.034 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.034 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.034 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.034 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.034 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.034 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.034 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.034 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.035 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.035 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.035 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.035 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.035 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.035 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.035 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.035 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.035 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.035 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.035 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.035 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.035 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.035 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.035 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.035 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.035 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.036 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.036 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.036 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.036 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.036 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.036 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.036 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.036 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.036 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.036 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.036 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.036 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.036 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.036 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.036 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.037 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.037 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.037 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.037 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.037 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.037 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.037 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.037 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.037 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.037 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.037 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.037 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.037 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.037 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.038 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.038 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.038 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.038 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.038 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.038 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.038 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.038 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.038 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.038 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.038 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.038 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.038 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.038 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.038 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.039 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.039 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.039 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.039 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.039 INFO:teuthology.orchestra.run.smithi098.stdout: "min_size": { 2024-05-09T10:51:35.039 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "min_size", 2024-05-09T10:51:35.039 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.039 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.039 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.039 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.039 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.039 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.039 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.039 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "default min_size for any created local pool", 2024-05-09T10:51:35.040 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T10:51:35.040 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.040 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.040 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.040 INFO:teuthology.orchestra.run.smithi098.stdout: "num_rep": { 2024-05-09T10:51:35.040 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "num_rep", 2024-05-09T10:51:35.040 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.040 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.040 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.040 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "3", 2024-05-09T10:51:35.040 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.040 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.040 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.040 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "default replica count for any created local pool", 2024-05-09T10:51:35.040 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.040 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.041 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.041 INFO:teuthology.orchestra.run.smithi098.stdout: "pg_num": { 2024-05-09T10:51:35.041 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "pg_num", 2024-05-09T10:51:35.041 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.041 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.041 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.041 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "128", 2024-05-09T10:51:35.041 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.041 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.041 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.041 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T10:51:35.041 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.041 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.041 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.042 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.042 INFO:teuthology.orchestra.run.smithi098.stdout: "prefix": { 2024-05-09T10:51:35.042 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "prefix", 2024-05-09T10:51:35.042 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.042 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.042 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.042 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.042 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.042 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.042 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.042 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "name prefix for any created local pool", 2024-05-09T10:51:35.042 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.042 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.042 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.042 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.043 INFO:teuthology.orchestra.run.smithi098.stdout: "subtree": { 2024-05-09T10:51:35.043 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "subtree", 2024-05-09T10:51:35.043 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.043 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.043 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.043 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "rack", 2024-05-09T10:51:35.043 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.043 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.043 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.043 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T10:51:35.043 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T10:51:35.043 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.043 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.043 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.043 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.044 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.044 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.044 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "mds_autoscaler", 2024-05-09T10:51:35.044 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.044 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.044 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.044 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.044 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.044 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.044 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.044 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.044 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.044 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.044 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.044 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.045 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.045 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.045 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.045 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.045 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.045 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.045 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.045 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.045 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.045 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.045 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.045 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.045 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.045 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.045 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.046 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.046 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.046 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.046 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.046 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.046 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.046 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.046 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.046 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.046 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.046 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.046 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.046 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.046 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.046 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.047 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.047 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.047 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.047 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.047 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.047 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.047 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.047 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.047 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.047 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.047 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.047 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.047 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.047 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.047 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.048 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.048 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.048 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.048 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.048 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.048 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.048 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.048 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.048 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.048 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.048 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.048 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.048 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.048 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.048 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.048 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.048 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "mirroring", 2024-05-09T10:51:35.049 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.049 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.049 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.049 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.049 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.049 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.049 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.049 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.049 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.049 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.049 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.049 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.049 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.049 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.049 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.050 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.050 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.050 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.050 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.050 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.050 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.050 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.050 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.050 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.050 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.050 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.050 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.050 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.050 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.050 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.051 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.051 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.051 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.051 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.051 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.051 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.051 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.051 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.051 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.051 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.051 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.051 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.051 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.051 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.052 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.052 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.052 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.052 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.052 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.052 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.052 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.052 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.052 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.052 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.052 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.052 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.052 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.052 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.053 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.053 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.053 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.053 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.053 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.053 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.053 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.053 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.053 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.053 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.053 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.053 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.053 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.053 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.053 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.054 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.054 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.054 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "nfs", 2024-05-09T10:51:35.054 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.054 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.054 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.054 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.054 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.054 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.054 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.054 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.054 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.054 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.054 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.055 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.055 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.055 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.055 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.055 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.055 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.055 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.055 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.055 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.055 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.055 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.055 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.055 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.055 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.055 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.056 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.056 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.056 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.056 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.056 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.056 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.056 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.056 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.056 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.056 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.056 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.056 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.056 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.056 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.056 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.056 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.056 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.057 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.057 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.057 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.057 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.057 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.057 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.057 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.057 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.057 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.057 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.057 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.057 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.057 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.057 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.057 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.058 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.058 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.058 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.058 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.058 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.058 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.058 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.058 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.058 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.058 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.058 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.058 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.058 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.058 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.058 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.059 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.059 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.059 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "orchestrator", 2024-05-09T10:51:35.059 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.059 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.059 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.059 INFO:teuthology.orchestra.run.smithi098.stdout: "fail_fs": { 2024-05-09T10:51:35.059 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "fail_fs", 2024-05-09T10:51:35.059 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.059 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.059 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.059 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.059 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.059 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.059 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.060 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T10:51:35.060 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.060 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.060 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.060 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.060 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.060 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.060 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.060 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.060 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.060 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.060 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.060 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.060 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.060 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.061 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.061 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.061 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.061 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.061 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.061 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.061 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.061 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.061 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.061 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.061 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.061 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.061 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.061 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.061 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.062 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.062 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.062 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.062 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.062 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.062 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.062 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.062 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.062 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.062 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.062 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.062 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.062 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.062 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.063 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.063 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.063 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.063 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.063 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.063 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.063 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.063 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.063 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.063 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.063 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.063 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.063 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.063 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.063 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.064 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.064 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.064 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.064 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.064 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.064 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.064 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.064 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.064 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.064 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.064 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.064 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.064 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.064 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.064 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.065 INFO:teuthology.orchestra.run.smithi098.stdout: "orchestrator": { 2024-05-09T10:51:35.065 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "orchestrator", 2024-05-09T10:51:35.065 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.065 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.065 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.065 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.065 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.065 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.065 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.065 INFO:teuthology.orchestra.run.smithi098.stdout: "cephadm", 2024-05-09T10:51:35.066 INFO:teuthology.orchestra.run.smithi098.stdout: "rook", 2024-05-09T10:51:35.066 INFO:teuthology.orchestra.run.smithi098.stdout: "test_orchestrator" 2024-05-09T10:51:35.066 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.066 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Orchestrator backend", 2024-05-09T10:51:35.066 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.066 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.066 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.066 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.066 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.066 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.066 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "osd_perf_query", 2024-05-09T10:51:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.067 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.068 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.068 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.068 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.068 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.068 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.068 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.068 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.068 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.068 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.068 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.068 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.068 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.068 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.068 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.069 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.069 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.069 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.069 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.069 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.069 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.069 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.069 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.069 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.069 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.069 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.069 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.069 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.070 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.070 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.070 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.070 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.070 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.070 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.070 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.070 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.070 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.070 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.070 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.070 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.070 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.070 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.070 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.071 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.071 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.071 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.071 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.071 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.071 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.071 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.071 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.071 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.071 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.071 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.071 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.071 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.071 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.071 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.071 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.071 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.071 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.072 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.072 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.072 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "osd_support", 2024-05-09T10:51:35.072 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.072 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.072 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.072 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.072 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.072 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.072 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.072 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.072 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.072 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.072 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.072 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.072 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.072 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.073 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.073 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.073 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.073 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.073 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.073 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.073 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.073 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.073 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.073 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.073 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.073 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.073 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.073 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.073 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.073 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.073 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.073 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.074 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.074 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.074 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.074 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.074 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.074 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.074 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.074 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.074 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.074 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.074 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.074 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.074 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.074 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.074 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.074 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.075 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.075 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.075 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.075 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.075 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.075 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.075 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.075 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.075 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.075 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.075 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.075 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.075 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.075 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.075 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.075 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.076 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.076 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.076 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.076 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.076 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.076 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.076 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.076 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.076 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.076 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.076 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.076 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.076 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "pg_autoscaler", 2024-05-09T10:51:35.076 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.076 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.077 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.077 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.077 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.077 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.077 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.077 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.077 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.077 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.077 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.077 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.077 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.077 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.077 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.077 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.077 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.078 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.078 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.078 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.078 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.078 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.078 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.078 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.078 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.078 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.078 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.078 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.078 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.078 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.078 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.078 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.078 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.079 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.079 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.079 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.079 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.079 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.079 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.079 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.079 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.079 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.079 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.079 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.079 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.079 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.079 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.080 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.080 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.080 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.080 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.080 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.080 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.080 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.080 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.080 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.080 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.080 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.080 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.080 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.080 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.080 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.080 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.080 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.081 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.081 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.081 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.081 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.081 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.081 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.081 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.081 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.081 INFO:teuthology.orchestra.run.smithi098.stdout: "sleep_interval": { 2024-05-09T10:51:35.081 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "sleep_interval", 2024-05-09T10:51:35.081 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.081 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.081 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.081 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "60", 2024-05-09T10:51:35.082 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.082 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.082 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.082 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.082 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.082 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.082 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.082 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.082 INFO:teuthology.orchestra.run.smithi098.stdout: "threshold": { 2024-05-09T10:51:35.082 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "threshold", 2024-05-09T10:51:35.082 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-05-09T10:51:35.082 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.082 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.082 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "3.0", 2024-05-09T10:51:35.082 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "1.0", 2024-05-09T10:51:35.082 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.083 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.083 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "scaling threshold", 2024-05-09T10:51:35.083 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T10:51:35.083 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.083 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.083 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.083 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.083 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.083 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.083 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "progress", 2024-05-09T10:51:35.083 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.083 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.083 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.083 INFO:teuthology.orchestra.run.smithi098.stdout: "allow_pg_recovery_event": { 2024-05-09T10:51:35.083 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "allow_pg_recovery_event", 2024-05-09T10:51:35.083 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.083 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.083 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.084 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.084 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.084 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.084 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.084 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T10:51:35.084 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.084 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.084 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.084 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.084 INFO:teuthology.orchestra.run.smithi098.stdout: "enabled": { 2024-05-09T10:51:35.084 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "enabled", 2024-05-09T10:51:35.084 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.084 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.084 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.084 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.084 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.084 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.085 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.085 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.085 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.085 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.085 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.085 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.085 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.085 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.085 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.085 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.085 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.085 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.085 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.085 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.085 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.085 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.085 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.085 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.086 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.086 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.086 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.086 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.086 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.086 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.086 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.086 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.086 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.086 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.086 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.086 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.086 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.086 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.086 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.086 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.087 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.087 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.087 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.087 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.087 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.087 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.087 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.087 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.087 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.087 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.087 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.087 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.087 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.087 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.087 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.087 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.088 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.088 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.088 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.088 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.088 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.088 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.088 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.088 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.088 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.088 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.088 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.088 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.088 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.088 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.088 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.088 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.088 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.089 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.089 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.089 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.089 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.089 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.089 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.089 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.089 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.089 INFO:teuthology.orchestra.run.smithi098.stdout: "max_completed_events": { 2024-05-09T10:51:35.089 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "max_completed_events", 2024-05-09T10:51:35.089 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.089 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.089 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.089 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "50", 2024-05-09T10:51:35.089 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.089 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.090 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.090 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "number of past completed events to remember", 2024-05-09T10:51:35.090 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.090 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.090 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.090 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.090 INFO:teuthology.orchestra.run.smithi098.stdout: "sleep_interval": { 2024-05-09T10:51:35.090 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "sleep_interval", 2024-05-09T10:51:35.090 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.090 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.090 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.090 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "5", 2024-05-09T10:51:35.090 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.090 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.090 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.091 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how long the module is going to sleep", 2024-05-09T10:51:35.091 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.091 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.091 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.091 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.091 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.091 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.091 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.091 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "prometheus", 2024-05-09T10:51:35.091 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.091 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.091 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.091 INFO:teuthology.orchestra.run.smithi098.stdout: "cache": { 2024-05-09T10:51:35.091 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cache", 2024-05-09T10:51:35.091 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.091 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.091 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.092 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.092 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.092 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.092 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.092 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.092 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.092 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.092 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.092 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.092 INFO:teuthology.orchestra.run.smithi098.stdout: "exclude_perf_counters": { 2024-05-09T10:51:35.092 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "exclude_perf_counters", 2024-05-09T10:51:35.092 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.092 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.092 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.092 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.092 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.092 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.093 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.093 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T10:51:35.093 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T10:51:35.093 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.093 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.093 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.093 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.093 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.093 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.093 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.093 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.093 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.093 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.093 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.093 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.093 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.094 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.094 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.094 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.094 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.094 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.094 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.094 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.094 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.094 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.094 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.094 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.094 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.094 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.094 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.094 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.095 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.095 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.095 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.095 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.095 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.095 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.095 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.095 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.095 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.095 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.095 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.095 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.095 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.095 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.095 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.095 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.095 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.096 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.096 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.096 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.096 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.096 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.096 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.096 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.096 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.096 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.096 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.096 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.096 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.096 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.096 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.096 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.096 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.096 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.096 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.097 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.097 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.097 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.097 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.097 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.097 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.097 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.097 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.097 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.097 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.097 INFO:teuthology.orchestra.run.smithi098.stdout: "rbd_stats_pools": { 2024-05-09T10:51:35.097 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rbd_stats_pools", 2024-05-09T10:51:35.097 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.097 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.097 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.097 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.097 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.097 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.098 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.098 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.098 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.098 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.098 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.098 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.098 INFO:teuthology.orchestra.run.smithi098.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T10:51:35.098 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T10:51:35.098 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.098 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.098 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.098 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "300", 2024-05-09T10:51:35.098 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.098 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.099 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.099 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.099 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.099 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.099 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.099 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.099 INFO:teuthology.orchestra.run.smithi098.stdout: "scrape_interval": { 2024-05-09T10:51:35.099 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "scrape_interval", 2024-05-09T10:51:35.099 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-05-09T10:51:35.099 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.099 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.099 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "15.0", 2024-05-09T10:51:35.099 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.099 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.100 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.100 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.100 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.100 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.100 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.100 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.100 INFO:teuthology.orchestra.run.smithi098.stdout: "server_addr": { 2024-05-09T10:51:35.100 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_addr", 2024-05-09T10:51:35.100 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.100 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.100 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.100 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "::", 2024-05-09T10:51:35.100 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.100 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.101 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.101 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T10:51:35.101 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.101 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.101 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.101 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.101 INFO:teuthology.orchestra.run.smithi098.stdout: "server_port": { 2024-05-09T10:51:35.101 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_port", 2024-05-09T10:51:35.101 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.101 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.101 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.101 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "9283", 2024-05-09T10:51:35.101 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.101 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.101 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.102 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T10:51:35.102 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.102 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.102 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.102 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.102 INFO:teuthology.orchestra.run.smithi098.stdout: "stale_cache_strategy": { 2024-05-09T10:51:35.102 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "stale_cache_strategy", 2024-05-09T10:51:35.102 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.102 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.102 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.102 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "log", 2024-05-09T10:51:35.102 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.102 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.102 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.103 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.103 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.103 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.103 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.103 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.103 INFO:teuthology.orchestra.run.smithi098.stdout: "standby_behaviour": { 2024-05-09T10:51:35.103 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "standby_behaviour", 2024-05-09T10:51:35.103 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.103 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.103 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.103 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "default", 2024-05-09T10:51:35.103 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.103 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.103 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.104 INFO:teuthology.orchestra.run.smithi098.stdout: "default", 2024-05-09T10:51:35.104 INFO:teuthology.orchestra.run.smithi098.stdout: "error" 2024-05-09T10:51:35.104 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.104 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.104 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.104 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.104 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.104 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.104 INFO:teuthology.orchestra.run.smithi098.stdout: "standby_error_status_code": { 2024-05-09T10:51:35.104 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "standby_error_status_code", 2024-05-09T10:51:35.104 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.104 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.104 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.104 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "500", 2024-05-09T10:51:35.105 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "400", 2024-05-09T10:51:35.105 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "599", 2024-05-09T10:51:35.105 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.105 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.105 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.105 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.105 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.105 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.105 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.105 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.105 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.105 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rbd_support", 2024-05-09T10:51:35.105 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.105 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.105 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.106 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.106 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.106 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.106 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.106 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.106 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.106 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.106 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.106 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.106 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.106 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.106 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.106 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.106 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.107 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.107 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.107 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.107 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.107 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.107 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.107 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.107 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.107 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.107 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.107 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.107 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.107 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.107 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.107 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.108 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.108 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.108 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.108 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.108 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.108 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.108 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.108 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.108 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.108 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.108 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.108 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.108 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.108 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.108 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.109 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.109 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.109 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.109 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.109 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.109 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.109 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.109 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.109 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.109 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.109 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.109 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.109 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.109 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.110 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.110 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.110 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.110 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.110 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.110 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.110 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.110 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.110 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.110 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.110 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.110 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.110 INFO:teuthology.orchestra.run.smithi098.stdout: "max_concurrent_snap_create": { 2024-05-09T10:51:35.111 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "max_concurrent_snap_create", 2024-05-09T10:51:35.111 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.111 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.111 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.111 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-05-09T10:51:35.111 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.111 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.111 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.111 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.111 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.111 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.111 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.111 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.111 INFO:teuthology.orchestra.run.smithi098.stdout: "mirror_snapshot_schedule": { 2024-05-09T10:51:35.111 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T10:51:35.111 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.111 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.112 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.112 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.112 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.112 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.112 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.112 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.112 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.112 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.112 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.112 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.112 INFO:teuthology.orchestra.run.smithi098.stdout: "trash_purge_schedule": { 2024-05-09T10:51:35.112 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "trash_purge_schedule", 2024-05-09T10:51:35.112 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.112 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.112 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.112 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.112 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.113 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.113 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.113 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.113 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.113 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.113 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.113 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.113 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.113 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.113 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.113 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "restful", 2024-05-09T10:51:35.113 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.113 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.113 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.113 INFO:teuthology.orchestra.run.smithi098.stdout: "enable_auth": { 2024-05-09T10:51:35.113 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "enable_auth", 2024-05-09T10:51:35.113 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.113 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.114 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.114 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.114 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.114 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.114 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.114 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.114 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.114 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.114 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.114 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.114 INFO:teuthology.orchestra.run.smithi098.stdout: "key_file": { 2024-05-09T10:51:35.114 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "key_file", 2024-05-09T10:51:35.114 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.114 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.114 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.114 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.115 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.115 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.115 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.115 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.115 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.115 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.115 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.115 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.115 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.115 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.115 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.115 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.115 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.115 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.115 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.115 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.115 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.115 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.116 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.116 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.116 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.116 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.116 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.116 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.116 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.116 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.116 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.116 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.116 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.116 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.116 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.116 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.117 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.117 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.117 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.117 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.117 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.117 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.117 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.117 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.117 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.117 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.117 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.117 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.117 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.117 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.117 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.117 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.118 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.118 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.118 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.118 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.118 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.118 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.118 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.118 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.118 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.118 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.118 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.118 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.118 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.118 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.118 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.119 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.119 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.119 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.119 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.119 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.119 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.119 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.119 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.119 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.119 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.119 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.119 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.119 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.119 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.120 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.120 INFO:teuthology.orchestra.run.smithi098.stdout: "server_addr": { 2024-05-09T10:51:35.120 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_addr", 2024-05-09T10:51:35.120 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.120 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.120 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.120 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.120 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.120 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.120 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.120 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.120 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.120 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.120 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.121 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.121 INFO:teuthology.orchestra.run.smithi098.stdout: "server_port": { 2024-05-09T10:51:35.121 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_port", 2024-05-09T10:51:35.121 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.121 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.121 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.121 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.121 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.121 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.121 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.121 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.121 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.121 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.121 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.121 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.121 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.122 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.122 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.122 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rgw", 2024-05-09T10:51:35.122 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.122 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.122 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.122 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.122 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.122 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.122 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.122 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.122 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.122 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.122 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.122 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.123 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.123 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.123 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.123 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.123 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.123 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.123 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.123 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.123 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.123 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.123 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.123 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.123 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.123 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.123 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.123 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.123 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.123 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.124 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.124 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.124 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.124 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.124 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.124 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.124 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.124 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.124 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.124 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.124 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.124 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.124 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.125 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.125 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.125 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.125 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.125 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.125 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.125 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.125 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.125 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.125 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.125 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.125 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.125 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.125 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.125 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.125 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.126 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.126 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.126 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.126 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.126 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.126 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.126 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.126 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.126 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.126 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.126 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.126 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.126 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.126 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.127 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.127 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.127 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.127 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rook", 2024-05-09T10:51:35.127 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.127 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.127 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.127 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.127 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.127 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.127 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.127 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.127 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.127 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.127 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.128 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.128 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.128 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.128 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.128 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.128 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.128 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.128 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.128 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.128 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.128 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.128 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.128 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.128 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.128 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.129 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.129 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.129 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.129 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.129 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.129 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.129 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.129 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.129 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.129 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.129 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.129 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.129 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.129 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.129 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.129 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.130 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.130 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.130 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.130 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.130 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.130 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.130 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.130 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.130 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.130 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.130 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.130 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.130 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.130 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.130 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.131 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.131 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.131 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.131 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.131 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.131 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.131 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.131 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.131 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.131 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.131 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.131 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.131 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.132 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.132 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.132 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.132 INFO:teuthology.orchestra.run.smithi098.stdout: "storage_class": { 2024-05-09T10:51:35.132 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "storage_class", 2024-05-09T10:51:35.132 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.132 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.132 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.132 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "local", 2024-05-09T10:51:35.132 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.132 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.132 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.132 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T10:51:35.132 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.132 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.133 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.133 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.133 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.133 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.133 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.133 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "selftest", 2024-05-09T10:51:35.133 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.133 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.133 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.133 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.133 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.133 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.133 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.133 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.133 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.133 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.134 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.134 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.134 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.134 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.134 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.134 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.134 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.134 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.134 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.134 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.134 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.134 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.134 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.134 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.134 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.135 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.135 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.135 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.135 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.135 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.135 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.135 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.135 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.135 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.135 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.135 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.135 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.135 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.135 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.135 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.136 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.136 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.136 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.136 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.136 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.136 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.136 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.136 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.136 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.136 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.136 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.136 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.136 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.136 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.136 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.136 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.137 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.137 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.137 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.137 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.137 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.137 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.137 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.137 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.137 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.137 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.137 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.137 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.137 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.137 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.137 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.137 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.138 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.138 INFO:teuthology.orchestra.run.smithi098.stdout: "roption1": { 2024-05-09T10:51:35.138 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "roption1", 2024-05-09T10:51:35.138 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.138 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.138 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.138 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.138 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.138 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.138 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.138 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.138 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.138 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.138 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.139 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.139 INFO:teuthology.orchestra.run.smithi098.stdout: "roption2": { 2024-05-09T10:51:35.139 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "roption2", 2024-05-09T10:51:35.139 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.139 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.139 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.139 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "xyz", 2024-05-09T10:51:35.139 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.139 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.139 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.139 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.139 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.139 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.139 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.139 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.140 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption1": { 2024-05-09T10:51:35.140 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption1", 2024-05-09T10:51:35.140 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.140 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.140 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.140 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.140 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.140 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.140 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.140 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.140 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.140 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.140 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.140 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.140 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption2": { 2024-05-09T10:51:35.141 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption2", 2024-05-09T10:51:35.141 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.141 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.141 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.141 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.141 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.141 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.141 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.141 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.141 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.141 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.141 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.141 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.141 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption3": { 2024-05-09T10:51:35.141 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption3", 2024-05-09T10:51:35.141 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-05-09T10:51:35.142 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.142 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.142 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.142 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.142 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.142 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.142 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.142 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.142 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.142 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.142 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.142 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption4": { 2024-05-09T10:51:35.142 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption4", 2024-05-09T10:51:35.142 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.142 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.142 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.143 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.143 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.143 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.143 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.143 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.143 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.143 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.143 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.143 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.143 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption5": { 2024-05-09T10:51:35.143 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption5", 2024-05-09T10:51:35.143 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.143 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.143 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.143 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.144 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.144 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.144 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.144 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.144 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.144 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.144 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.144 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.144 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption6": { 2024-05-09T10:51:35.144 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption6", 2024-05-09T10:51:35.144 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.144 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.144 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.144 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.144 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.144 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.145 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.145 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.145 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.145 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.145 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.145 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.145 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption7": { 2024-05-09T10:51:35.145 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption7", 2024-05-09T10:51:35.145 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.145 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.145 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.145 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.145 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "1", 2024-05-09T10:51:35.145 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "42", 2024-05-09T10:51:35.145 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.146 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.146 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.146 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.146 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.146 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.146 INFO:teuthology.orchestra.run.smithi098.stdout: "testkey": { 2024-05-09T10:51:35.146 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "testkey", 2024-05-09T10:51:35.146 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.146 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.146 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.146 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.146 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.146 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.146 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.146 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.147 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.147 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.147 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.147 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.147 INFO:teuthology.orchestra.run.smithi098.stdout: "testlkey": { 2024-05-09T10:51:35.147 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "testlkey", 2024-05-09T10:51:35.147 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.147 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.147 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.147 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.147 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.147 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.147 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.147 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.147 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.147 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.148 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.148 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.148 INFO:teuthology.orchestra.run.smithi098.stdout: "testnewline": { 2024-05-09T10:51:35.148 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "testnewline", 2024-05-09T10:51:35.148 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.148 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.148 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.148 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.148 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.148 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.148 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.148 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.148 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.148 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.148 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.148 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.148 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.149 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.149 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.149 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "snap_schedule", 2024-05-09T10:51:35.149 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.149 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.149 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.149 INFO:teuthology.orchestra.run.smithi098.stdout: "allow_m_granularity": { 2024-05-09T10:51:35.149 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "allow_m_granularity", 2024-05-09T10:51:35.149 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.149 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.149 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.149 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.149 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.149 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.149 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.149 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T10:51:35.150 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.150 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.150 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.150 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.150 INFO:teuthology.orchestra.run.smithi098.stdout: "dump_on_update": { 2024-05-09T10:51:35.150 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "dump_on_update", 2024-05-09T10:51:35.150 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.150 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.150 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.150 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.150 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.150 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.150 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.150 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "dump database to debug log on update", 2024-05-09T10:51:35.150 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.151 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.151 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.151 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.151 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.151 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.151 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.151 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.151 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.151 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.151 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.151 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.151 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.151 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.151 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.151 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.152 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.152 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.152 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.152 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.152 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.152 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.152 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.152 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.152 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.152 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.152 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.152 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.152 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.152 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.152 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.153 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.153 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.153 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.153 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.153 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.153 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.153 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.153 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.153 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.153 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.153 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.153 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.153 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.153 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.153 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.153 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.154 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.154 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.154 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.154 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.154 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.154 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.154 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.154 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.154 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.154 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.154 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.154 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.154 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.154 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.154 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.155 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.155 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.155 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.155 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.155 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.155 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.155 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.155 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.155 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.155 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.155 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.155 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.155 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.155 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.155 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.155 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "stats", 2024-05-09T10:51:35.156 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.156 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.156 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.156 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.156 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.156 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.156 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.156 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.156 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.156 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.156 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.156 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.156 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.156 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.156 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.156 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.157 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.157 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.157 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.157 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.157 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.157 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.157 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.157 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.157 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.157 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.157 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.157 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.157 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.157 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.157 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.157 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.158 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.158 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.158 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.158 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.158 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.158 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.158 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.158 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.158 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.158 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.158 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.158 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.158 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.158 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.158 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.159 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.159 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.159 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.159 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.159 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.159 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.159 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.159 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.159 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.159 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.159 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.159 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.159 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.159 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.159 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.159 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.160 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.160 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.160 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.160 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.160 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.160 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.160 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.160 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.160 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.160 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.160 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.160 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.160 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.160 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "status", 2024-05-09T10:51:35.160 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.160 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.161 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.161 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.161 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.161 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.161 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.161 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.161 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.161 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.161 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.161 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.161 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.161 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.161 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.161 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.161 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.161 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.161 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.162 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.162 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.162 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.162 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.162 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.162 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.162 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.162 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.162 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.162 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.162 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.162 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.162 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.162 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.162 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.163 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.163 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.163 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.163 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.163 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.163 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.163 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.163 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.163 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.163 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.163 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.163 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.163 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.163 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.163 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.164 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.164 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.164 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.164 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.164 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.164 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.164 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.164 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.164 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.164 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.164 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.164 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.164 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.164 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.164 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.164 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.165 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.165 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.165 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.165 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.165 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.165 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.165 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.165 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.165 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.165 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.165 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.165 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "telegraf", 2024-05-09T10:51:35.165 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.165 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.165 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.165 INFO:teuthology.orchestra.run.smithi098.stdout: "address": { 2024-05-09T10:51:35.165 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "address", 2024-05-09T10:51:35.165 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.166 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.166 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.166 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T10:51:35.166 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.166 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.166 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.166 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.166 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.166 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.166 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.166 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.166 INFO:teuthology.orchestra.run.smithi098.stdout: "interval": { 2024-05-09T10:51:35.166 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "interval", 2024-05-09T10:51:35.166 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.166 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.167 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.167 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "15", 2024-05-09T10:51:35.167 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.167 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.167 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.167 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.167 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.167 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.167 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.167 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.167 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.167 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.167 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.167 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.167 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.167 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.167 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.168 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.168 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.168 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.168 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.168 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.168 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.168 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.168 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.168 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.168 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.168 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.168 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.168 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.168 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.168 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.168 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.168 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.169 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.169 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.169 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.169 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.169 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.169 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.169 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.169 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.169 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.169 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.169 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.169 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.169 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.169 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.169 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.169 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.170 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.170 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.170 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.170 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.170 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.170 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.170 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.170 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.170 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.170 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.170 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.170 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.170 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.170 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.170 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.171 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.171 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.171 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.171 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.171 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.171 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.171 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.171 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.171 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.171 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.171 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.171 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.171 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.171 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.171 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.171 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.172 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.172 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.172 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "telemetry", 2024-05-09T10:51:35.172 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.172 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.172 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.172 INFO:teuthology.orchestra.run.smithi098.stdout: "channel_basic": { 2024-05-09T10:51:35.172 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "channel_basic", 2024-05-09T10:51:35.172 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.172 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.172 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.172 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.172 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.172 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.172 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.173 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T10:51:35.173 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.173 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.173 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.173 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.173 INFO:teuthology.orchestra.run.smithi098.stdout: "channel_crash": { 2024-05-09T10:51:35.173 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "channel_crash", 2024-05-09T10:51:35.173 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.173 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.173 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.173 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.173 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.173 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.173 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.173 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T10:51:35.173 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.174 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.174 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.174 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.174 INFO:teuthology.orchestra.run.smithi098.stdout: "channel_device": { 2024-05-09T10:51:35.174 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "channel_device", 2024-05-09T10:51:35.174 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.174 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.174 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.174 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.174 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.174 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.174 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.174 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T10:51:35.174 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.175 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.175 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.175 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.175 INFO:teuthology.orchestra.run.smithi098.stdout: "channel_ident": { 2024-05-09T10:51:35.175 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "channel_ident", 2024-05-09T10:51:35.175 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.175 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.175 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.175 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.175 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.175 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.175 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.175 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T10:51:35.176 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.176 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.176 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.176 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.176 INFO:teuthology.orchestra.run.smithi098.stdout: "channel_perf": { 2024-05-09T10:51:35.176 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "channel_perf", 2024-05-09T10:51:35.176 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.176 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.176 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.176 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.176 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.176 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.176 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.176 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T10:51:35.176 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.177 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.177 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.177 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.177 INFO:teuthology.orchestra.run.smithi098.stdout: "contact": { 2024-05-09T10:51:35.177 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "contact", 2024-05-09T10:51:35.177 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.177 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.177 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.177 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.177 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.177 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.177 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.177 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.177 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.177 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.177 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.178 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.178 INFO:teuthology.orchestra.run.smithi098.stdout: "description": { 2024-05-09T10:51:35.178 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "description", 2024-05-09T10:51:35.178 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.178 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.178 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.178 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.178 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.178 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.178 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.178 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.178 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.178 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.178 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.178 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.179 INFO:teuthology.orchestra.run.smithi098.stdout: "device_url": { 2024-05-09T10:51:35.179 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "device_url", 2024-05-09T10:51:35.179 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.179 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.179 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.179 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T10:51:35.179 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.179 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.179 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.179 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.179 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.179 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.179 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.179 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.179 INFO:teuthology.orchestra.run.smithi098.stdout: "enabled": { 2024-05-09T10:51:35.180 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "enabled", 2024-05-09T10:51:35.180 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.180 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.180 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.180 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.180 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.180 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.180 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.180 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.180 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.180 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.180 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.180 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.180 INFO:teuthology.orchestra.run.smithi098.stdout: "interval": { 2024-05-09T10:51:35.180 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "interval", 2024-05-09T10:51:35.181 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.181 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.181 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.181 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "24", 2024-05-09T10:51:35.181 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "8", 2024-05-09T10:51:35.181 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.181 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.181 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.181 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.181 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.181 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.181 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.181 INFO:teuthology.orchestra.run.smithi098.stdout: "last_opt_revision": { 2024-05-09T10:51:35.181 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "last_opt_revision", 2024-05-09T10:51:35.181 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.181 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.182 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.182 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "1", 2024-05-09T10:51:35.182 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.182 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.182 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.182 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.182 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.182 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.182 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.182 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.182 INFO:teuthology.orchestra.run.smithi098.stdout: "leaderboard": { 2024-05-09T10:51:35.182 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "leaderboard", 2024-05-09T10:51:35.182 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.182 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.182 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.182 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.183 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.183 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.183 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.183 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.183 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.183 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.183 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.183 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.183 INFO:teuthology.orchestra.run.smithi098.stdout: "leaderboard_description": { 2024-05-09T10:51:35.183 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "leaderboard_description", 2024-05-09T10:51:35.183 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.183 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.183 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.183 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.183 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.184 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.184 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.184 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.184 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.184 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.184 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.184 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.184 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.184 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.184 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.184 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.184 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.184 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.184 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.184 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.185 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.185 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.185 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.185 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.185 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.185 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.185 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.185 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.185 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.185 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.185 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.185 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.185 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.185 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.185 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.186 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.186 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.186 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.186 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.186 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.186 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.186 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.186 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.186 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.186 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.186 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.186 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.186 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.186 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.186 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.186 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.187 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.187 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.187 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.187 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.187 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.187 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.187 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.187 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.187 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.187 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.187 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.187 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.187 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.187 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.188 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.188 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.188 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.188 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.188 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.188 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.188 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.188 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.188 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.188 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.188 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.188 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.188 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.188 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.189 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.189 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.189 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.189 INFO:teuthology.orchestra.run.smithi098.stdout: "organization": { 2024-05-09T10:51:35.189 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "organization", 2024-05-09T10:51:35.189 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.189 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.189 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.189 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.189 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.189 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.189 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.189 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.189 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.189 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.189 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.190 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.190 INFO:teuthology.orchestra.run.smithi098.stdout: "proxy": { 2024-05-09T10:51:35.190 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "proxy", 2024-05-09T10:51:35.190 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.190 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.190 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.190 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.190 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.190 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.190 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.190 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.190 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.190 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.190 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.190 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.191 INFO:teuthology.orchestra.run.smithi098.stdout: "url": { 2024-05-09T10:51:35.191 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "url", 2024-05-09T10:51:35.191 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.191 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.191 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.191 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T10:51:35.191 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.191 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.191 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.191 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.191 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.191 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.191 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.191 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.191 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.192 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.192 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.192 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "test_orchestrator", 2024-05-09T10:51:35.192 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.192 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.192 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.192 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.192 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.192 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.192 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.192 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.192 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.192 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.192 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.193 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.193 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.193 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.193 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.193 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.193 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.193 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.193 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.193 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.193 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.193 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.193 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.193 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.193 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.193 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.194 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.194 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.194 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.194 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.194 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.194 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.194 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.194 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.194 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.194 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.194 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.194 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.194 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.194 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.194 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.194 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.195 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.195 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.195 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.195 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.195 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.195 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.195 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.195 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.195 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.195 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.195 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.195 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.195 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.195 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.195 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.196 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.196 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.196 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.196 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.196 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.196 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.196 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.196 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.196 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.196 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.196 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.196 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.196 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.196 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.196 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.196 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.196 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.197 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.197 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.197 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "volumes", 2024-05-09T10:51:35.197 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.197 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.197 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.197 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.197 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.197 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.197 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.197 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.197 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.197 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.197 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.197 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.197 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.197 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.198 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.198 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.198 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.198 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.198 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.198 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.198 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.198 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.198 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.198 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.198 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.198 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.198 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.198 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.198 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.199 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.199 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.199 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.199 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.199 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.199 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.199 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.199 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.199 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.199 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.199 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.199 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.199 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.199 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.199 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.199 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.200 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.200 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.200 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.200 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.200 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.200 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.200 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.200 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.200 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.200 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.200 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.200 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.200 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.200 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.200 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.201 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.201 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.201 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.201 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.201 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.201 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.201 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.201 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.201 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.201 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.201 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.201 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.201 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.201 INFO:teuthology.orchestra.run.smithi098.stdout: "max_concurrent_clones": { 2024-05-09T10:51:35.202 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "max_concurrent_clones", 2024-05-09T10:51:35.202 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.202 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.202 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.202 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "4", 2024-05-09T10:51:35.202 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.202 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.202 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.202 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T10:51:35.202 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.202 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.202 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.202 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.202 INFO:teuthology.orchestra.run.smithi098.stdout: "periodic_async_work": { 2024-05-09T10:51:35.202 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "periodic_async_work", 2024-05-09T10:51:35.202 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.203 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.203 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.203 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.203 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.203 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.203 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.203 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Periodically check for async work", 2024-05-09T10:51:35.203 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.203 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.203 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.203 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.203 INFO:teuthology.orchestra.run.smithi098.stdout: "snapshot_clone_delay": { 2024-05-09T10:51:35.203 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "snapshot_clone_delay", 2024-05-09T10:51:35.203 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.203 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.203 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.203 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0", 2024-05-09T10:51:35.203 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.204 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.204 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.204 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T10:51:35.204 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.204 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.204 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.204 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.204 INFO:teuthology.orchestra.run.smithi098.stdout: "snapshot_clone_no_wait": { 2024-05-09T10:51:35.204 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T10:51:35.204 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.204 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.204 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.204 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.204 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.204 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.204 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.205 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T10:51:35.205 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.205 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.205 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.205 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.205 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.205 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.205 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.205 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "zabbix", 2024-05-09T10:51:35.205 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.205 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.205 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.205 INFO:teuthology.orchestra.run.smithi098.stdout: "discovery_interval": { 2024-05-09T10:51:35.205 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "discovery_interval", 2024-05-09T10:51:35.205 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "uint", 2024-05-09T10:51:35.205 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.205 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.206 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "100", 2024-05-09T10:51:35.206 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.206 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.206 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.206 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.206 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.206 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.206 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.206 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.206 INFO:teuthology.orchestra.run.smithi098.stdout: "identifier": { 2024-05-09T10:51:35.206 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "identifier", 2024-05-09T10:51:35.206 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.206 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.206 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.207 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.207 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.207 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.207 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.207 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.207 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.207 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.207 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.207 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.207 INFO:teuthology.orchestra.run.smithi098.stdout: "interval": { 2024-05-09T10:51:35.207 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "interval", 2024-05-09T10:51:35.207 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.207 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.207 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.207 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "60", 2024-05-09T10:51:35.207 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.208 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.208 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.208 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.208 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.208 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.208 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.208 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.208 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.208 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.208 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.208 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.208 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.208 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.208 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.208 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.208 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.209 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.209 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.209 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.209 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.209 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.209 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.209 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.209 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.209 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.209 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.209 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.209 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.209 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.209 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.209 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.209 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.209 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.211 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.211 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.211 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.211 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.211 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.211 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.211 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.211 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.211 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.211 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.211 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.211 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.211 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.211 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.211 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.211 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.212 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.212 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.212 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.212 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.212 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.212 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.212 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.212 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.212 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.212 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.212 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.212 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.212 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.212 INFO:teuthology.orchestra.run.smithi098.stdout: "zabbix_host": { 2024-05-09T10:51:35.212 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "zabbix_host", 2024-05-09T10:51:35.212 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.213 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.213 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.213 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.213 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.213 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.213 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.213 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.213 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.213 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.213 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.213 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.213 INFO:teuthology.orchestra.run.smithi098.stdout: "zabbix_port": { 2024-05-09T10:51:35.213 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "zabbix_port", 2024-05-09T10:51:35.213 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.213 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.213 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.213 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10051", 2024-05-09T10:51:35.214 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.214 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.214 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.214 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.214 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.214 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.214 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.214 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.214 INFO:teuthology.orchestra.run.smithi098.stdout: "zabbix_sender": { 2024-05-09T10:51:35.214 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "zabbix_sender", 2024-05-09T10:51:35.214 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.214 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.214 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.214 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T10:51:35.214 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.214 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.215 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.215 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.215 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.215 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.215 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.215 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.215 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.215 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.215 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-05-09T10:51:35.215 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.215 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.215 INFO:teuthology.orchestra.run.smithi098.stdout: "modules": [ 2024-05-09T10:51:35.215 INFO:teuthology.orchestra.run.smithi098.stdout: "iostat", 2024-05-09T10:51:35.215 INFO:teuthology.orchestra.run.smithi098.stdout: "nfs", 2024-05-09T10:51:35.215 INFO:teuthology.orchestra.run.smithi098.stdout: "restful" 2024-05-09T10:51:35.215 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.216 INFO:teuthology.orchestra.run.smithi098.stdout: "available_modules": [ 2024-05-09T10:51:35.216 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.216 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "alerts", 2024-05-09T10:51:35.216 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.216 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.216 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.216 INFO:teuthology.orchestra.run.smithi098.stdout: "interval": { 2024-05-09T10:51:35.216 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "interval", 2024-05-09T10:51:35.216 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.216 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.216 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.216 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "60", 2024-05-09T10:51:35.216 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.216 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.216 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.216 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T10:51:35.217 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.217 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.217 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.217 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.217 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.217 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.217 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.217 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.217 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.217 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.217 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.217 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.217 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.217 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.217 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.217 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.217 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.218 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.218 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.218 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.218 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.218 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.218 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.218 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.218 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.218 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.218 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.218 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.218 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.218 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.218 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.218 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.219 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.219 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.219 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.219 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.219 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.219 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.219 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.219 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.219 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.219 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.219 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.219 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.219 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.219 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.220 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.220 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.220 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.220 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.220 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.220 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.220 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.220 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.220 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.220 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.220 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.220 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.220 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.220 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.220 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.220 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.220 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.221 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.221 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.221 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.221 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.221 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.221 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.221 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.221 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.221 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.221 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.221 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.221 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_destination": { 2024-05-09T10:51:35.221 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_destination", 2024-05-09T10:51:35.221 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.221 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.221 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.221 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.222 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.222 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.222 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.222 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Email address to send alerts to", 2024-05-09T10:51:35.222 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.222 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.222 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.222 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.222 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_from_name": { 2024-05-09T10:51:35.222 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_from_name", 2024-05-09T10:51:35.222 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.222 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.222 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.222 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "Ceph", 2024-05-09T10:51:35.222 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.223 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.223 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.223 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Email From: name", 2024-05-09T10:51:35.223 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.223 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.223 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.223 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.223 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_host": { 2024-05-09T10:51:35.223 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_host", 2024-05-09T10:51:35.223 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.223 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.223 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.223 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.223 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.223 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.223 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.223 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "SMTP server", 2024-05-09T10:51:35.224 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.224 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.224 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.224 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.224 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_password": { 2024-05-09T10:51:35.224 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_password", 2024-05-09T10:51:35.224 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.224 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.224 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.224 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.224 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.224 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.224 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.224 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Password to authenticate with", 2024-05-09T10:51:35.224 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.224 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.224 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.225 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.225 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_port": { 2024-05-09T10:51:35.225 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_port", 2024-05-09T10:51:35.225 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.225 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.225 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.225 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "465", 2024-05-09T10:51:35.225 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.225 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.225 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.225 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "SMTP port", 2024-05-09T10:51:35.225 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.225 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.225 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.225 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.225 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_sender": { 2024-05-09T10:51:35.225 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_sender", 2024-05-09T10:51:35.226 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.226 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.226 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.226 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.226 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.226 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.226 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.226 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "SMTP envelope sender", 2024-05-09T10:51:35.226 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.226 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.226 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.226 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.226 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_ssl": { 2024-05-09T10:51:35.226 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_ssl", 2024-05-09T10:51:35.226 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.227 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.227 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.227 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.227 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.227 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.227 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.227 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T10:51:35.227 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.227 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.227 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.227 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.227 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_user": { 2024-05-09T10:51:35.227 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_user", 2024-05-09T10:51:35.227 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.227 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.228 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.228 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.228 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.228 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.228 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.228 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "User to authenticate as", 2024-05-09T10:51:35.228 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.228 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.228 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.228 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.228 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.228 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.228 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.228 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "balancer", 2024-05-09T10:51:35.228 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.228 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.229 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.229 INFO:teuthology.orchestra.run.smithi098.stdout: "active": { 2024-05-09T10:51:35.229 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "active", 2024-05-09T10:51:35.229 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.229 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.229 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.229 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.229 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.229 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.229 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.229 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T10:51:35.229 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.229 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.229 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.229 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.229 INFO:teuthology.orchestra.run.smithi098.stdout: "begin_time": { 2024-05-09T10:51:35.229 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "begin_time", 2024-05-09T10:51:35.230 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.230 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.230 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.230 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0000", 2024-05-09T10:51:35.230 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.230 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.230 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.230 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T10:51:35.230 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T10:51:35.230 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.230 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.230 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.230 INFO:teuthology.orchestra.run.smithi098.stdout: "begin_weekday": { 2024-05-09T10:51:35.230 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "begin_weekday", 2024-05-09T10:51:35.231 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "uint", 2024-05-09T10:51:35.231 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.231 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.231 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0", 2024-05-09T10:51:35.231 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "0", 2024-05-09T10:51:35.231 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "6", 2024-05-09T10:51:35.231 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.231 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T10:51:35.231 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T10:51:35.231 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.231 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.231 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.231 INFO:teuthology.orchestra.run.smithi098.stdout: "crush_compat_max_iterations": { 2024-05-09T10:51:35.231 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "crush_compat_max_iterations", 2024-05-09T10:51:35.231 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "uint", 2024-05-09T10:51:35.232 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.232 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.232 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "25", 2024-05-09T10:51:35.232 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "1", 2024-05-09T10:51:35.232 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "250", 2024-05-09T10:51:35.232 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.232 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T10:51:35.232 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.232 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.232 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.232 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.232 INFO:teuthology.orchestra.run.smithi098.stdout: "crush_compat_metrics": { 2024-05-09T10:51:35.232 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "crush_compat_metrics", 2024-05-09T10:51:35.232 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.232 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.233 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.233 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T10:51:35.233 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.233 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.233 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.233 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T10:51:35.233 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T10:51:35.233 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.233 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.233 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.233 INFO:teuthology.orchestra.run.smithi098.stdout: "crush_compat_step": { 2024-05-09T10:51:35.233 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "crush_compat_step", 2024-05-09T10:51:35.233 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-05-09T10:51:35.234 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.234 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.234 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0.5", 2024-05-09T10:51:35.234 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "0.001", 2024-05-09T10:51:35.234 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "0.999", 2024-05-09T10:51:35.234 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.234 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "aggressiveness of optimization", 2024-05-09T10:51:35.234 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T10:51:35.234 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.234 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.234 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.234 INFO:teuthology.orchestra.run.smithi098.stdout: "end_time": { 2024-05-09T10:51:35.234 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "end_time", 2024-05-09T10:51:35.234 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.234 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.234 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.235 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "2359", 2024-05-09T10:51:35.235 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.235 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.235 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.235 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T10:51:35.235 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T10:51:35.235 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.235 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.235 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.235 INFO:teuthology.orchestra.run.smithi098.stdout: "end_weekday": { 2024-05-09T10:51:35.235 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "end_weekday", 2024-05-09T10:51:35.235 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "uint", 2024-05-09T10:51:35.235 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.235 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.235 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0", 2024-05-09T10:51:35.236 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "0", 2024-05-09T10:51:35.236 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "6", 2024-05-09T10:51:35.236 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.236 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T10:51:35.236 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T10:51:35.236 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.236 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.236 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.236 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.236 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.236 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.236 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.236 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.236 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.236 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.236 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.237 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.237 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.237 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.237 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.237 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.237 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.237 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.237 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.237 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.237 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.237 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.237 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.237 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.237 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.237 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.238 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.238 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.238 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.238 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.238 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.238 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.238 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.238 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.238 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.238 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.238 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.238 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.238 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.238 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.239 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.239 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.239 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.239 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.239 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.239 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.239 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.239 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.239 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.239 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.239 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.239 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.239 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.239 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.239 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.240 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.240 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.240 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.240 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.240 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.240 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.240 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.240 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.240 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.240 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.240 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.240 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.240 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.240 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.240 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.240 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.241 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.241 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.241 INFO:teuthology.orchestra.run.smithi098.stdout: "min_score": { 2024-05-09T10:51:35.241 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "min_score", 2024-05-09T10:51:35.241 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-05-09T10:51:35.241 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.241 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.241 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0", 2024-05-09T10:51:35.241 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.241 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.241 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.241 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T10:51:35.241 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.241 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.241 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.241 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.241 INFO:teuthology.orchestra.run.smithi098.stdout: "mode": { 2024-05-09T10:51:35.242 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "mode", 2024-05-09T10:51:35.242 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.242 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.242 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.242 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "upmap", 2024-05-09T10:51:35.242 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.242 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.242 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.242 INFO:teuthology.orchestra.run.smithi098.stdout: "crush-compat", 2024-05-09T10:51:35.242 INFO:teuthology.orchestra.run.smithi098.stdout: "none", 2024-05-09T10:51:35.242 INFO:teuthology.orchestra.run.smithi098.stdout: "read", 2024-05-09T10:51:35.242 INFO:teuthology.orchestra.run.smithi098.stdout: "upmap", 2024-05-09T10:51:35.242 INFO:teuthology.orchestra.run.smithi098.stdout: "upmap-read" 2024-05-09T10:51:35.242 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.242 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Balancer mode", 2024-05-09T10:51:35.242 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.243 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.243 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.243 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.243 INFO:teuthology.orchestra.run.smithi098.stdout: "pool_ids": { 2024-05-09T10:51:35.243 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "pool_ids", 2024-05-09T10:51:35.243 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.243 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.243 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.243 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.243 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.243 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.243 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.243 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T10:51:35.243 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.243 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.244 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.244 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.244 INFO:teuthology.orchestra.run.smithi098.stdout: "sleep_interval": { 2024-05-09T10:51:35.244 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "sleep_interval", 2024-05-09T10:51:35.244 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.244 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.244 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.244 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "60", 2024-05-09T10:51:35.244 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.244 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.244 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.244 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T10:51:35.244 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.244 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.244 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.244 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.244 INFO:teuthology.orchestra.run.smithi098.stdout: "upmap_max_deviation": { 2024-05-09T10:51:35.245 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "upmap_max_deviation", 2024-05-09T10:51:35.245 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.245 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.245 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.245 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "5", 2024-05-09T10:51:35.245 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "1", 2024-05-09T10:51:35.245 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.245 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.245 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T10:51:35.245 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T10:51:35.245 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.245 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.245 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.245 INFO:teuthology.orchestra.run.smithi098.stdout: "upmap_max_optimizations": { 2024-05-09T10:51:35.245 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "upmap_max_optimizations", 2024-05-09T10:51:35.245 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "uint", 2024-05-09T10:51:35.245 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.246 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.246 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-05-09T10:51:35.246 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.246 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.246 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.246 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T10:51:35.246 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.246 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.246 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.246 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.246 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.246 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.246 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.246 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cephadm", 2024-05-09T10:51:35.246 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.247 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.247 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.247 INFO:teuthology.orchestra.run.smithi098.stdout: "agent_down_multiplier": { 2024-05-09T10:51:35.247 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "agent_down_multiplier", 2024-05-09T10:51:35.247 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-05-09T10:51:35.247 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.247 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.247 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "3.0", 2024-05-09T10:51:35.247 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.247 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.247 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.247 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T10:51:35.247 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.247 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.247 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.248 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.248 INFO:teuthology.orchestra.run.smithi098.stdout: "agent_refresh_rate": { 2024-05-09T10:51:35.248 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "agent_refresh_rate", 2024-05-09T10:51:35.248 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.248 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.248 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.248 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "20", 2024-05-09T10:51:35.248 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.248 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.248 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.248 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T10:51:35.248 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.248 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.248 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.248 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.248 INFO:teuthology.orchestra.run.smithi098.stdout: "agent_starting_port": { 2024-05-09T10:51:35.249 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "agent_starting_port", 2024-05-09T10:51:35.249 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.249 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.249 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.249 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "4721", 2024-05-09T10:51:35.249 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.249 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.249 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.249 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T10:51:35.249 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.249 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.249 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.249 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.249 INFO:teuthology.orchestra.run.smithi098.stdout: "allow_ptrace": { 2024-05-09T10:51:35.249 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "allow_ptrace", 2024-05-09T10:51:35.250 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.250 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.250 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.250 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.250 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.250 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.250 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.250 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T10:51:35.250 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T10:51:35.250 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.250 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.250 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.250 INFO:teuthology.orchestra.run.smithi098.stdout: "autotune_interval": { 2024-05-09T10:51:35.250 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "autotune_interval", 2024-05-09T10:51:35.250 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.250 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.251 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.251 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "600", 2024-05-09T10:51:35.251 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.251 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.251 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.251 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T10:51:35.251 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.251 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.251 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.251 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.251 INFO:teuthology.orchestra.run.smithi098.stdout: "autotune_memory_target_ratio": { 2024-05-09T10:51:35.251 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T10:51:35.251 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-05-09T10:51:35.251 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.251 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.251 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0.7", 2024-05-09T10:51:35.251 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.251 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.252 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.252 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T10:51:35.252 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.252 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.252 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.252 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.252 INFO:teuthology.orchestra.run.smithi098.stdout: "cephadm_log_destination": { 2024-05-09T10:51:35.252 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cephadm_log_destination", 2024-05-09T10:51:35.252 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.252 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.252 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.252 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.252 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.252 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.253 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.253 INFO:teuthology.orchestra.run.smithi098.stdout: "file", 2024-05-09T10:51:35.253 INFO:teuthology.orchestra.run.smithi098.stdout: "file,syslog", 2024-05-09T10:51:35.253 INFO:teuthology.orchestra.run.smithi098.stdout: "syslog" 2024-05-09T10:51:35.253 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.253 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T10:51:35.253 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.253 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.253 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.253 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.253 INFO:teuthology.orchestra.run.smithi098.stdout: "cgroups_split": { 2024-05-09T10:51:35.253 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cgroups_split", 2024-05-09T10:51:35.253 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.253 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.253 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.253 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.253 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.254 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.254 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.254 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T10:51:35.254 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.254 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.254 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.254 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.254 INFO:teuthology.orchestra.run.smithi098.stdout: "config_checks_enabled": { 2024-05-09T10:51:35.254 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "config_checks_enabled", 2024-05-09T10:51:35.254 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.254 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.254 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.254 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.254 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.254 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.254 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.255 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T10:51:35.255 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.255 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.255 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.255 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.255 INFO:teuthology.orchestra.run.smithi098.stdout: "config_dashboard": { 2024-05-09T10:51:35.255 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "config_dashboard", 2024-05-09T10:51:35.255 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.255 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.255 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.255 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.255 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.255 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.255 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.255 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T10:51:35.255 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.255 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.256 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.256 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.256 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_alertmanager": { 2024-05-09T10:51:35.256 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_alertmanager", 2024-05-09T10:51:35.256 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.256 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.256 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.256 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T10:51:35.256 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.256 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.256 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.256 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Prometheus container image", 2024-05-09T10:51:35.256 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.256 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.256 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.256 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.256 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_base": { 2024-05-09T10:51:35.257 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_base", 2024-05-09T10:51:35.257 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.257 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.257 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.257 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T10:51:35.257 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.257 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.257 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.257 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Container image name, without the tag", 2024-05-09T10:51:35.257 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.257 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.257 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.257 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.257 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_elasticsearch": { 2024-05-09T10:51:35.257 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_elasticsearch", 2024-05-09T10:51:35.257 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.257 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.258 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.258 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T10:51:35.258 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.258 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.258 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.258 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "elasticsearch container image", 2024-05-09T10:51:35.258 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.258 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.258 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.258 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.258 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_grafana": { 2024-05-09T10:51:35.258 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_grafana", 2024-05-09T10:51:35.258 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.258 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.258 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.258 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-05-09T10:51:35.259 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.259 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.259 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.259 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Prometheus container image", 2024-05-09T10:51:35.259 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.259 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.259 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.259 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.259 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_haproxy": { 2024-05-09T10:51:35.259 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_haproxy", 2024-05-09T10:51:35.259 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.259 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.259 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.259 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T10:51:35.260 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.260 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.260 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.260 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "HAproxy container image", 2024-05-09T10:51:35.260 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.260 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.260 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.260 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.260 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_jaeger_agent": { 2024-05-09T10:51:35.260 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_jaeger_agent", 2024-05-09T10:51:35.260 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.260 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.260 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.260 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T10:51:35.260 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.260 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.260 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.261 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Jaeger agent container image", 2024-05-09T10:51:35.261 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.261 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.261 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.261 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.261 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_jaeger_collector": { 2024-05-09T10:51:35.261 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_jaeger_collector", 2024-05-09T10:51:35.261 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.261 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.261 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.261 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T10:51:35.261 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.261 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.261 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.261 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Jaeger collector container image", 2024-05-09T10:51:35.261 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.261 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.262 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.262 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.262 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_jaeger_query": { 2024-05-09T10:51:35.262 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_jaeger_query", 2024-05-09T10:51:35.262 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.262 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.262 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.262 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T10:51:35.262 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.262 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.262 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.262 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Jaeger query container image", 2024-05-09T10:51:35.262 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.262 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.262 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.262 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.262 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_keepalived": { 2024-05-09T10:51:35.262 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_keepalived", 2024-05-09T10:51:35.263 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.263 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.263 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.263 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T10:51:35.263 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.263 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.263 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.263 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Keepalived container image", 2024-05-09T10:51:35.263 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.263 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.263 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.263 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.263 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_loki": { 2024-05-09T10:51:35.263 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_loki", 2024-05-09T10:51:35.263 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.263 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.263 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.264 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T10:51:35.264 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.264 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.264 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.264 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Loki container image", 2024-05-09T10:51:35.264 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.264 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.264 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.264 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.264 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_node_exporter": { 2024-05-09T10:51:35.264 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_node_exporter", 2024-05-09T10:51:35.264 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.264 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.264 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.264 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T10:51:35.264 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.264 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.264 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.265 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Prometheus container image", 2024-05-09T10:51:35.265 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.265 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.265 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.265 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.265 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_nvmeof": { 2024-05-09T10:51:35.265 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_nvmeof", 2024-05-09T10:51:35.265 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.265 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.265 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.265 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-05-09T10:51:35.265 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.265 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.265 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.265 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Nvme-of container image", 2024-05-09T10:51:35.265 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.265 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.266 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.266 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.266 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_prometheus": { 2024-05-09T10:51:35.266 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_prometheus", 2024-05-09T10:51:35.266 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.266 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.266 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.266 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T10:51:35.266 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.266 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.266 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.266 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Prometheus container image", 2024-05-09T10:51:35.266 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.266 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.266 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.266 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.266 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_promtail": { 2024-05-09T10:51:35.267 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_promtail", 2024-05-09T10:51:35.267 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.267 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.267 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.267 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T10:51:35.267 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.267 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.267 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.267 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Promtail container image", 2024-05-09T10:51:35.267 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.267 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.267 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.267 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.267 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_snmp_gateway": { 2024-05-09T10:51:35.267 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_snmp_gateway", 2024-05-09T10:51:35.268 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.268 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.268 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.268 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T10:51:35.268 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.268 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.268 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.268 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "SNMP Gateway container image", 2024-05-09T10:51:35.268 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.268 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.268 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.268 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.268 INFO:teuthology.orchestra.run.smithi098.stdout: "container_init": { 2024-05-09T10:51:35.268 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_init", 2024-05-09T10:51:35.268 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.268 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.268 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.269 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.269 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.269 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.269 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.269 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T10:51:35.269 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.269 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.269 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.269 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.269 INFO:teuthology.orchestra.run.smithi098.stdout: "daemon_cache_timeout": { 2024-05-09T10:51:35.269 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "daemon_cache_timeout", 2024-05-09T10:51:35.269 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.269 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.269 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.269 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "600", 2024-05-09T10:51:35.270 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.270 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.270 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.270 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T10:51:35.270 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.270 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.270 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.270 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.270 INFO:teuthology.orchestra.run.smithi098.stdout: "default_cephadm_command_timeout": { 2024-05-09T10:51:35.270 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T10:51:35.270 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.270 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.270 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.270 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "900", 2024-05-09T10:51:35.270 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.270 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.271 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.271 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T10:51:35.271 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.271 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.271 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.271 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.271 INFO:teuthology.orchestra.run.smithi098.stdout: "default_registry": { 2024-05-09T10:51:35.271 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "default_registry", 2024-05-09T10:51:35.271 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.271 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.271 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.271 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "docker.io", 2024-05-09T10:51:35.271 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.271 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.271 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.271 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T10:51:35.271 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.272 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.272 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.272 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.272 INFO:teuthology.orchestra.run.smithi098.stdout: "device_cache_timeout": { 2024-05-09T10:51:35.272 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "device_cache_timeout", 2024-05-09T10:51:35.272 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.272 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.272 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.272 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "1800", 2024-05-09T10:51:35.272 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.272 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.272 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.272 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "seconds to cache device inventory", 2024-05-09T10:51:35.272 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.272 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.272 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.273 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.273 INFO:teuthology.orchestra.run.smithi098.stdout: "device_enhanced_scan": { 2024-05-09T10:51:35.273 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "device_enhanced_scan", 2024-05-09T10:51:35.273 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.273 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.273 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.273 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.273 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.273 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.273 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.273 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T10:51:35.273 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.273 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.273 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.273 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.273 INFO:teuthology.orchestra.run.smithi098.stdout: "facts_cache_timeout": { 2024-05-09T10:51:35.274 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "facts_cache_timeout", 2024-05-09T10:51:35.274 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.274 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.274 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.274 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "60", 2024-05-09T10:51:35.274 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.274 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.274 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.274 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "seconds to cache host facts data", 2024-05-09T10:51:35.274 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.274 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.274 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.274 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.274 INFO:teuthology.orchestra.run.smithi098.stdout: "host_check_interval": { 2024-05-09T10:51:35.275 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "host_check_interval", 2024-05-09T10:51:35.275 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.275 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.275 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.275 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "600", 2024-05-09T10:51:35.275 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.275 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.275 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.275 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how frequently to perform a host check", 2024-05-09T10:51:35.275 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.275 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.275 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.275 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.275 INFO:teuthology.orchestra.run.smithi098.stdout: "hw_monitoring": { 2024-05-09T10:51:35.276 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "hw_monitoring", 2024-05-09T10:51:35.276 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.276 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.276 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.276 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.276 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.276 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.276 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.276 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T10:51:35.276 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.276 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.276 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.276 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.276 INFO:teuthology.orchestra.run.smithi098.stdout: "inventory_list_all": { 2024-05-09T10:51:35.276 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "inventory_list_all", 2024-05-09T10:51:35.277 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.277 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.277 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.277 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.277 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.277 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.277 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.277 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T10:51:35.277 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.277 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.277 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.277 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.277 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.277 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.277 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.277 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.278 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.278 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.278 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.278 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.278 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.278 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.278 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.278 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.278 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.278 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.278 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.278 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.278 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.278 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.279 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.279 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.279 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.279 INFO:teuthology.orchestra.run.smithi098.stdout: "log_refresh_metadata": { 2024-05-09T10:51:35.279 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_refresh_metadata", 2024-05-09T10:51:35.279 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.279 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.279 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.279 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.279 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.279 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.279 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.279 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T10:51:35.279 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.279 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.279 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.280 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.280 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.280 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.280 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.280 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.280 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.280 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.280 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.280 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.280 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.280 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T10:51:35.280 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.280 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.280 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.280 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.280 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.280 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.281 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.281 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.281 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.281 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.281 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.281 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.281 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.281 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.281 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.281 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.281 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.281 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.281 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.281 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.281 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.281 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.281 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.282 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.282 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.282 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.282 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.282 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.282 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.282 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.282 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.282 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.282 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.282 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.282 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.282 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.282 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.282 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.282 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.283 INFO:teuthology.orchestra.run.smithi098.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T10:51:35.283 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T10:51:35.283 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.283 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.283 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.283 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.283 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.283 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.283 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.283 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T10:51:35.283 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.283 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.283 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.283 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.283 INFO:teuthology.orchestra.run.smithi098.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T10:51:35.283 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T10:51:35.283 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.283 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.284 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.284 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "*", 2024-05-09T10:51:35.284 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.284 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.284 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.284 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T10:51:35.284 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.284 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.284 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.284 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.284 INFO:teuthology.orchestra.run.smithi098.stdout: "max_count_per_host": { 2024-05-09T10:51:35.284 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "max_count_per_host", 2024-05-09T10:51:35.284 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.284 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.284 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.284 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-05-09T10:51:35.284 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.284 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.285 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.285 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "max number of daemons per service per host", 2024-05-09T10:51:35.285 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.285 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.285 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.285 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.285 INFO:teuthology.orchestra.run.smithi098.stdout: "max_osd_draining_count": { 2024-05-09T10:51:35.285 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "max_osd_draining_count", 2024-05-09T10:51:35.285 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.285 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.285 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.285 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-05-09T10:51:35.285 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.285 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.285 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.285 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T10:51:35.285 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.286 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.286 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.286 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.286 INFO:teuthology.orchestra.run.smithi098.stdout: "migration_current": { 2024-05-09T10:51:35.286 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "migration_current", 2024-05-09T10:51:35.286 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.286 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.286 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.286 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.286 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.286 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.286 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.286 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "internal - do not modify", 2024-05-09T10:51:35.286 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.286 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.287 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.287 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.287 INFO:teuthology.orchestra.run.smithi098.stdout: "mode": { 2024-05-09T10:51:35.287 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "mode", 2024-05-09T10:51:35.287 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.287 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.287 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.287 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "root", 2024-05-09T10:51:35.287 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.287 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.287 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.287 INFO:teuthology.orchestra.run.smithi098.stdout: "cephadm-package", 2024-05-09T10:51:35.287 INFO:teuthology.orchestra.run.smithi098.stdout: "root" 2024-05-09T10:51:35.287 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.287 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T10:51:35.287 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.288 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.288 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.288 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.288 INFO:teuthology.orchestra.run.smithi098.stdout: "oob_default_addr": { 2024-05-09T10:51:35.288 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "oob_default_addr", 2024-05-09T10:51:35.288 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.288 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.288 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.288 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "169.254.1.1", 2024-05-09T10:51:35.288 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.288 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.288 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.288 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T10:51:35.288 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.288 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.288 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.288 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.289 INFO:teuthology.orchestra.run.smithi098.stdout: "prometheus_alerts_path": { 2024-05-09T10:51:35.289 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "prometheus_alerts_path", 2024-05-09T10:51:35.289 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.289 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.289 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.289 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T10:51:35.289 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.289 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.289 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.289 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T10:51:35.289 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.289 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.289 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.289 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.289 INFO:teuthology.orchestra.run.smithi098.stdout: "registry_insecure": { 2024-05-09T10:51:35.290 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "registry_insecure", 2024-05-09T10:51:35.290 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.290 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.290 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.290 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.290 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.290 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.290 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.290 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T10:51:35.290 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.290 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.290 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.290 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.290 INFO:teuthology.orchestra.run.smithi098.stdout: "registry_password": { 2024-05-09T10:51:35.290 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "registry_password", 2024-05-09T10:51:35.291 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.291 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.291 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.291 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.291 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.291 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.291 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.291 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T10:51:35.291 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.291 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.291 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.291 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.291 INFO:teuthology.orchestra.run.smithi098.stdout: "registry_url": { 2024-05-09T10:51:35.291 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "registry_url", 2024-05-09T10:51:35.291 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.291 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.291 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.292 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.292 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.292 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.292 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.292 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T10:51:35.292 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.292 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.292 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.292 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.292 INFO:teuthology.orchestra.run.smithi098.stdout: "registry_username": { 2024-05-09T10:51:35.292 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "registry_username", 2024-05-09T10:51:35.292 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.292 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.292 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.292 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.293 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.293 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.293 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.293 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T10:51:35.293 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.293 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.293 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.293 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.293 INFO:teuthology.orchestra.run.smithi098.stdout: "secure_monitoring_stack": { 2024-05-09T10:51:35.293 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "secure_monitoring_stack", 2024-05-09T10:51:35.293 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.293 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.293 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.293 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.293 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.293 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.293 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.293 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T10:51:35.294 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.294 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.294 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.294 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.294 INFO:teuthology.orchestra.run.smithi098.stdout: "service_discovery_port": { 2024-05-09T10:51:35.294 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "service_discovery_port", 2024-05-09T10:51:35.294 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.294 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.294 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.294 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "8765", 2024-05-09T10:51:35.294 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.294 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.294 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.294 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "cephadm service discovery port", 2024-05-09T10:51:35.294 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.295 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.295 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.295 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.295 INFO:teuthology.orchestra.run.smithi098.stdout: "ssh_config_file": { 2024-05-09T10:51:35.295 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ssh_config_file", 2024-05-09T10:51:35.295 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.295 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.295 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.295 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.295 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.295 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.295 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.295 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T10:51:35.295 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.295 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.295 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.295 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.296 INFO:teuthology.orchestra.run.smithi098.stdout: "use_agent": { 2024-05-09T10:51:35.296 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "use_agent", 2024-05-09T10:51:35.296 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.296 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.296 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.296 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.296 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.296 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.296 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.296 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T10:51:35.296 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.296 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.296 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.296 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.296 INFO:teuthology.orchestra.run.smithi098.stdout: "use_repo_digest": { 2024-05-09T10:51:35.296 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "use_repo_digest", 2024-05-09T10:51:35.296 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.296 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.297 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.297 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.297 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.297 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.297 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.297 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T10:51:35.297 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.297 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.297 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.297 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.297 INFO:teuthology.orchestra.run.smithi098.stdout: "warn_on_failed_host_check": { 2024-05-09T10:51:35.297 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "warn_on_failed_host_check", 2024-05-09T10:51:35.297 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.297 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.297 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.297 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.297 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.297 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.298 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.298 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T10:51:35.298 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.298 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.298 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.298 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.298 INFO:teuthology.orchestra.run.smithi098.stdout: "warn_on_stray_daemons": { 2024-05-09T10:51:35.298 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "warn_on_stray_daemons", 2024-05-09T10:51:35.298 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.298 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.298 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.298 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.298 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.298 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.298 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.298 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T10:51:35.298 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.299 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.299 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.299 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.299 INFO:teuthology.orchestra.run.smithi098.stdout: "warn_on_stray_hosts": { 2024-05-09T10:51:35.299 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "warn_on_stray_hosts", 2024-05-09T10:51:35.299 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.299 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.299 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.299 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.299 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.299 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.299 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.299 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T10:51:35.299 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.300 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.300 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.300 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.300 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.300 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.300 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.300 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "crash", 2024-05-09T10:51:35.300 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.300 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.300 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.300 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.300 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.300 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.300 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.300 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.301 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.301 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.301 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.301 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.301 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.301 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.301 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.301 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.301 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.301 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.301 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.301 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.301 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.302 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.302 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.302 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.302 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.302 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.302 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.302 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.302 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.302 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.302 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.302 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.302 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.302 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.303 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.303 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.303 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.303 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.303 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.303 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.303 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.303 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.303 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.303 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.303 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.303 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.303 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.303 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.303 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.304 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.304 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.304 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.304 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.304 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.304 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.304 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.304 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.304 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.304 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.304 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.304 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.304 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.304 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.304 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.304 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.305 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.305 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.305 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.305 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.305 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.305 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.305 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.305 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.305 INFO:teuthology.orchestra.run.smithi098.stdout: "retain_interval": { 2024-05-09T10:51:35.305 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "retain_interval", 2024-05-09T10:51:35.305 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.305 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.305 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.305 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "31536000", 2024-05-09T10:51:35.305 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.305 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.306 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.306 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T10:51:35.306 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.306 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.306 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.306 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.306 INFO:teuthology.orchestra.run.smithi098.stdout: "warn_recent_interval": { 2024-05-09T10:51:35.306 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "warn_recent_interval", 2024-05-09T10:51:35.306 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.306 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.306 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.306 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "1209600", 2024-05-09T10:51:35.306 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.306 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.306 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.307 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T10:51:35.307 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.307 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.307 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.307 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.307 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.307 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.307 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.307 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "dashboard", 2024-05-09T10:51:35.307 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.307 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.307 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.307 INFO:teuthology.orchestra.run.smithi098.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T10:51:35.307 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T10:51:35.307 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.307 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.308 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.308 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-05-09T10:51:35.308 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.308 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.308 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.308 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.308 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.308 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.308 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.308 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.308 INFO:teuthology.orchestra.run.smithi098.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T10:51:35.308 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T10:51:35.308 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.308 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.309 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.309 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.309 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.309 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.309 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.309 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.309 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.309 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.309 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.309 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.309 INFO:teuthology.orchestra.run.smithi098.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T10:51:35.309 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T10:51:35.309 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.309 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.309 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.309 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.309 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.310 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.310 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.310 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.310 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.310 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.310 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.310 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.310 INFO:teuthology.orchestra.run.smithi098.stdout: "AUDIT_API_ENABLED": { 2024-05-09T10:51:35.310 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T10:51:35.310 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.310 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.310 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.310 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.310 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.310 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.310 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.311 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.311 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.311 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.311 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.311 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.311 INFO:teuthology.orchestra.run.smithi098.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T10:51:35.311 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T10:51:35.311 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.311 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.311 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.311 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.311 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.311 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.311 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.311 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.311 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.311 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.311 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.312 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.312 INFO:teuthology.orchestra.run.smithi098.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T10:51:35.312 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T10:51:35.312 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.312 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.312 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.312 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.312 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.312 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.312 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.312 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.312 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.312 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.312 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.312 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.312 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T10:51:35.312 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T10:51:35.312 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.313 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.313 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.313 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.313 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.313 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.313 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.313 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.313 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.313 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.313 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.313 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.313 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T10:51:35.313 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T10:51:35.313 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.313 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.314 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.314 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.314 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.314 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.314 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.314 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.314 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.314 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.314 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.314 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.314 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T10:51:35.314 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T10:51:35.314 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.314 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.314 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.315 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.315 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.315 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.315 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.315 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.315 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.315 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.315 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.315 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.315 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T10:51:35.315 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T10:51:35.315 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.315 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.315 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.316 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.316 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.316 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.316 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.316 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.316 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.316 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.316 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.316 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.316 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T10:51:35.316 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T10:51:35.316 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.316 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.316 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.316 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.316 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.316 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.317 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.317 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.317 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.317 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.317 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.317 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.317 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T10:51:35.317 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T10:51:35.317 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.317 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.317 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.317 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.317 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.317 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.317 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.317 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.317 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.317 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.318 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.318 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.318 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T10:51:35.318 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T10:51:35.318 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.318 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.318 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.318 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.318 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.318 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.318 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.318 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.318 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.318 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.318 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.318 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.319 INFO:teuthology.orchestra.run.smithi098.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T10:51:35.319 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T10:51:35.319 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.319 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.319 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.319 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.319 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.319 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.319 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.319 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.319 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.319 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.319 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.319 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.319 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T10:51:35.320 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T10:51:35.320 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.320 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.320 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.320 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "admin", 2024-05-09T10:51:35.320 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.320 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.320 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.320 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.320 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.320 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.320 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.320 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.320 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T10:51:35.320 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T10:51:35.320 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.320 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.320 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.321 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.321 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.321 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.321 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.321 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.321 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.321 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.321 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.321 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.321 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_API_URL": { 2024-05-09T10:51:35.321 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_API_URL", 2024-05-09T10:51:35.321 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.321 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.321 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.321 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.321 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.322 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.322 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.322 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.322 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.322 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.322 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.322 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.322 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T10:51:35.322 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T10:51:35.322 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.322 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.322 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.322 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "admin", 2024-05-09T10:51:35.322 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.322 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.322 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.323 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.323 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.323 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.323 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.323 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.323 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T10:51:35.323 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T10:51:35.323 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.323 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.323 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.323 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.323 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.323 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.323 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.323 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.323 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.323 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.323 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.324 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.324 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T10:51:35.324 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T10:51:35.324 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.324 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.324 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.324 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.324 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.324 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.324 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.324 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.324 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.324 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.324 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.324 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.324 INFO:teuthology.orchestra.run.smithi098.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T10:51:35.324 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T10:51:35.324 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.325 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.325 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.325 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.325 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.325 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.325 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.325 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.325 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.325 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.325 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.325 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.325 INFO:teuthology.orchestra.run.smithi098.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T10:51:35.325 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T10:51:35.325 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.325 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.325 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.325 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.325 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.326 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.326 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.326 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.326 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.326 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.326 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.326 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.326 INFO:teuthology.orchestra.run.smithi098.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T10:51:35.326 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T10:51:35.326 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.326 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.326 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.326 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.326 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.326 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.327 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.327 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.327 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.327 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.327 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.327 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.327 INFO:teuthology.orchestra.run.smithi098.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T10:51:35.327 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T10:51:35.327 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.327 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.327 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.327 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.327 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.327 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.328 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.328 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.328 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.328 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.328 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.328 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.328 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T10:51:35.328 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T10:51:35.328 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.328 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.328 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.328 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.328 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.328 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.329 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.329 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.329 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.329 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.329 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.329 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.329 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T10:51:35.329 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T10:51:35.329 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.329 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.329 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.329 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.329 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.329 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.329 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.330 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.330 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.330 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.330 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.330 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.330 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T10:51:35.330 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T10:51:35.330 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.330 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.330 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.330 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.330 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.330 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.330 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.330 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.331 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.331 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.331 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.331 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.331 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T10:51:35.331 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T10:51:35.331 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.331 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.331 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.331 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.331 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.331 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.331 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.331 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.331 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.331 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.332 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.332 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.332 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T10:51:35.332 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T10:51:35.332 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.332 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.332 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.332 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.332 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.332 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.332 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.332 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.332 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.332 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.332 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.332 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.332 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T10:51:35.333 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T10:51:35.333 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.333 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.333 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.333 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.333 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.333 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.333 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.333 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.333 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.333 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.333 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.333 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.333 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T10:51:35.333 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T10:51:35.334 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.334 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.334 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.334 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.334 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.334 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.334 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.334 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.334 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.334 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.334 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.334 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.334 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T10:51:35.334 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T10:51:35.334 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.335 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.335 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.335 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.335 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.335 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.335 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.335 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.335 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.335 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.335 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.335 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.335 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T10:51:35.335 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T10:51:35.335 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.335 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.336 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.336 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T10:51:35.336 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.336 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.336 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.336 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.336 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.336 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.336 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.336 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.336 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T10:51:35.336 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T10:51:35.336 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.336 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.336 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.336 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-05-09T10:51:35.337 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.337 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.337 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.337 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.337 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.337 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.337 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.337 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.337 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T10:51:35.337 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T10:51:35.337 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.337 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.337 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.337 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "8", 2024-05-09T10:51:35.338 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.338 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.338 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.338 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.338 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.338 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.338 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.338 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.338 INFO:teuthology.orchestra.run.smithi098.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T10:51:35.338 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T10:51:35.338 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.338 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.338 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.338 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "45", 2024-05-09T10:51:35.338 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.338 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.339 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.339 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.339 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.339 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.339 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.339 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.339 INFO:teuthology.orchestra.run.smithi098.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T10:51:35.339 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T10:51:35.339 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.339 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.339 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.339 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.339 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.339 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.339 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.340 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.340 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.340 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.340 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.340 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.340 INFO:teuthology.orchestra.run.smithi098.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T10:51:35.340 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T10:51:35.340 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.340 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.340 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.340 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "admin", 2024-05-09T10:51:35.340 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.340 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.340 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.340 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.340 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.341 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.341 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.341 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.341 INFO:teuthology.orchestra.run.smithi098.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T10:51:35.341 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T10:51:35.341 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.341 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.341 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.341 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.341 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.341 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.341 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.341 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.341 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.341 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.341 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.342 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.342 INFO:teuthology.orchestra.run.smithi098.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T10:51:35.342 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T10:51:35.342 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.342 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.342 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.342 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.342 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.342 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.342 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.342 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.342 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.342 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.342 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.342 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.343 INFO:teuthology.orchestra.run.smithi098.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T10:51:35.343 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T10:51:35.343 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.343 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.343 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.343 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0", 2024-05-09T10:51:35.343 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.343 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.343 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.343 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.343 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.343 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.343 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.343 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.343 INFO:teuthology.orchestra.run.smithi098.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T10:51:35.343 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T10:51:35.344 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.344 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.344 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.344 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-05-09T10:51:35.344 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.344 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.344 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.344 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.344 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.344 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.344 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.344 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.344 INFO:teuthology.orchestra.run.smithi098.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T10:51:35.344 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T10:51:35.344 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.345 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.345 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.345 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "5", 2024-05-09T10:51:35.345 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.345 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.345 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.345 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.345 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.345 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.345 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.345 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.345 INFO:teuthology.orchestra.run.smithi098.stdout: "cross_origin_url": { 2024-05-09T10:51:35.345 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cross_origin_url", 2024-05-09T10:51:35.345 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.345 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.345 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.346 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.346 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.346 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.346 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.346 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.346 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.346 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.346 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.346 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.346 INFO:teuthology.orchestra.run.smithi098.stdout: "crt_file": { 2024-05-09T10:51:35.346 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "crt_file", 2024-05-09T10:51:35.346 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.346 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.346 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.347 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.347 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.347 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.347 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.347 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.347 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.347 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.347 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.347 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.347 INFO:teuthology.orchestra.run.smithi098.stdout: "debug": { 2024-05-09T10:51:35.347 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "debug", 2024-05-09T10:51:35.347 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.347 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.347 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.347 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.347 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.347 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.348 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.348 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Enable/disable debug options", 2024-05-09T10:51:35.348 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.348 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.348 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.348 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.348 INFO:teuthology.orchestra.run.smithi098.stdout: "jwt_token_ttl": { 2024-05-09T10:51:35.348 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "jwt_token_ttl", 2024-05-09T10:51:35.348 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.348 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.348 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.348 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "28800", 2024-05-09T10:51:35.348 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.348 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.348 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.348 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.348 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.349 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.349 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.349 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.349 INFO:teuthology.orchestra.run.smithi098.stdout: "key_file": { 2024-05-09T10:51:35.349 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "key_file", 2024-05-09T10:51:35.349 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.349 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.349 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.349 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.349 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.349 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.349 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.349 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.349 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.349 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.349 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.349 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.349 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.350 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.350 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.350 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.350 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.350 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.350 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.350 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.350 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.350 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.350 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.350 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.350 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.350 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.350 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.350 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.350 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.351 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.351 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.351 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.351 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.351 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.351 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.351 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.351 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.351 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.351 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.351 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.351 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.351 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.351 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.351 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.351 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.351 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.351 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.352 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.352 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.352 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.352 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.352 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.352 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.352 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.352 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.352 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.352 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.352 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.352 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.352 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.352 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.352 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.352 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.353 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.353 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.353 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.353 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.353 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.353 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.353 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.353 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.353 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.353 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.353 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.353 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.353 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.353 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.353 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.353 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.353 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.354 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.354 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.354 INFO:teuthology.orchestra.run.smithi098.stdout: "motd": { 2024-05-09T10:51:35.354 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "motd", 2024-05-09T10:51:35.354 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.354 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.354 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.354 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.354 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.354 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.354 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.354 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "The message of the day", 2024-05-09T10:51:35.354 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.354 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.354 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.354 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.355 INFO:teuthology.orchestra.run.smithi098.stdout: "redirect_resolve_ip_addr": { 2024-05-09T10:51:35.355 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T10:51:35.355 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.355 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.355 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.355 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.355 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.355 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.355 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.355 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.355 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.355 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.355 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.355 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.355 INFO:teuthology.orchestra.run.smithi098.stdout: "server_addr": { 2024-05-09T10:51:35.355 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_addr", 2024-05-09T10:51:35.355 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.356 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.356 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.356 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "::", 2024-05-09T10:51:35.356 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.356 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.356 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.356 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.356 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.356 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.356 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.356 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.356 INFO:teuthology.orchestra.run.smithi098.stdout: "server_port": { 2024-05-09T10:51:35.356 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_port", 2024-05-09T10:51:35.356 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.356 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.356 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.356 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "8080", 2024-05-09T10:51:35.357 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.357 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.357 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.357 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.357 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.357 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.357 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.357 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.357 INFO:teuthology.orchestra.run.smithi098.stdout: "ssl": { 2024-05-09T10:51:35.357 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ssl", 2024-05-09T10:51:35.357 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.357 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.357 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.357 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.357 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.357 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.357 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.357 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.358 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.358 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.358 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.358 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.358 INFO:teuthology.orchestra.run.smithi098.stdout: "ssl_server_port": { 2024-05-09T10:51:35.358 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ssl_server_port", 2024-05-09T10:51:35.358 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.358 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.358 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.358 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "8443", 2024-05-09T10:51:35.358 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.358 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.358 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.358 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.359 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.359 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.359 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.359 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.359 INFO:teuthology.orchestra.run.smithi098.stdout: "standby_behaviour": { 2024-05-09T10:51:35.359 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "standby_behaviour", 2024-05-09T10:51:35.359 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.359 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.359 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.359 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "redirect", 2024-05-09T10:51:35.359 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.359 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.359 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.359 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.360 INFO:teuthology.orchestra.run.smithi098.stdout: "redirect" 2024-05-09T10:51:35.360 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.360 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.360 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.360 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.360 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.360 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.360 INFO:teuthology.orchestra.run.smithi098.stdout: "standby_error_status_code": { 2024-05-09T10:51:35.360 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "standby_error_status_code", 2024-05-09T10:51:35.360 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.360 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.360 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.360 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "500", 2024-05-09T10:51:35.360 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "400", 2024-05-09T10:51:35.360 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "599", 2024-05-09T10:51:35.360 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.360 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.361 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.361 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.361 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.361 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.361 INFO:teuthology.orchestra.run.smithi098.stdout: "url_prefix": { 2024-05-09T10:51:35.361 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "url_prefix", 2024-05-09T10:51:35.361 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.361 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.361 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.361 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.361 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.361 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.361 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.361 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.361 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.361 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.362 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.362 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.362 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.362 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.362 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.362 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "devicehealth", 2024-05-09T10:51:35.362 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.362 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.362 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.362 INFO:teuthology.orchestra.run.smithi098.stdout: "enable_monitoring": { 2024-05-09T10:51:35.362 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "enable_monitoring", 2024-05-09T10:51:35.362 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.362 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.362 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.362 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.363 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.363 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.363 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.363 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "monitor device health metrics", 2024-05-09T10:51:35.363 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.363 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.363 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.363 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.363 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.363 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.363 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.363 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.363 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.363 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.363 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.363 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.363 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.363 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.364 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.364 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.364 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.364 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.364 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.364 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.364 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.364 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.364 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.364 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.364 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.364 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.364 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.364 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.364 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.364 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.364 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.365 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.365 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.365 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.365 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.365 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.365 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.365 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.365 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.365 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.365 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.365 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.365 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.365 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.365 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.365 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.365 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.365 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.365 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.366 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.366 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.366 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.366 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.366 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.366 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.366 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.366 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.366 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.366 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.366 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.366 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.366 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.366 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.366 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.366 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.367 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.367 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.367 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.367 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.367 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.367 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.367 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.367 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.367 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.367 INFO:teuthology.orchestra.run.smithi098.stdout: "mark_out_threshold": { 2024-05-09T10:51:35.367 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "mark_out_threshold", 2024-05-09T10:51:35.367 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.367 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.367 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.367 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "2419200", 2024-05-09T10:51:35.367 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.367 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.367 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.368 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T10:51:35.368 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.368 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.368 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.368 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.368 INFO:teuthology.orchestra.run.smithi098.stdout: "pool_name": { 2024-05-09T10:51:35.368 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "pool_name", 2024-05-09T10:51:35.368 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.368 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.368 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.368 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "device_health_metrics", 2024-05-09T10:51:35.368 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.368 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.368 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.368 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T10:51:35.368 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.368 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.368 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.369 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.369 INFO:teuthology.orchestra.run.smithi098.stdout: "retention_period": { 2024-05-09T10:51:35.369 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "retention_period", 2024-05-09T10:51:35.369 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.369 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.369 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.369 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "15552000", 2024-05-09T10:51:35.369 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.369 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.369 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.369 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how long to retain device health metrics", 2024-05-09T10:51:35.369 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.369 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.369 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.369 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.370 INFO:teuthology.orchestra.run.smithi098.stdout: "scrape_frequency": { 2024-05-09T10:51:35.370 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "scrape_frequency", 2024-05-09T10:51:35.370 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.370 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.370 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.370 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "86400", 2024-05-09T10:51:35.370 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.370 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.370 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.370 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T10:51:35.370 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.370 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.370 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.370 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.370 INFO:teuthology.orchestra.run.smithi098.stdout: "self_heal": { 2024-05-09T10:51:35.370 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "self_heal", 2024-05-09T10:51:35.371 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.371 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.371 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.371 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.371 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.371 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.371 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.371 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T10:51:35.371 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.371 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.371 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.371 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.371 INFO:teuthology.orchestra.run.smithi098.stdout: "sleep_interval": { 2024-05-09T10:51:35.371 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "sleep_interval", 2024-05-09T10:51:35.371 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.371 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.372 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.372 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "600", 2024-05-09T10:51:35.372 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.372 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.372 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.372 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T10:51:35.372 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.372 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.372 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.372 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.372 INFO:teuthology.orchestra.run.smithi098.stdout: "warn_threshold": { 2024-05-09T10:51:35.372 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "warn_threshold", 2024-05-09T10:51:35.372 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.372 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.373 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.373 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "7257600", 2024-05-09T10:51:35.373 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.373 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.373 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.373 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T10:51:35.373 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.373 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.373 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.373 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.373 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.373 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.373 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.373 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "diskprediction_local", 2024-05-09T10:51:35.373 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.373 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.373 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.373 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.374 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.374 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.374 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.374 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.374 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.374 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.374 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.374 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.374 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.374 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.374 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.374 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.374 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.374 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.374 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.374 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.375 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.375 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.375 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.375 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.375 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.375 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.375 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.375 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.375 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.375 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.375 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.375 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.375 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.375 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.375 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.375 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.375 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.375 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.376 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.376 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.376 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.376 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.376 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.376 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.376 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.376 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.376 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.376 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.376 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.376 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.376 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.376 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.376 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.377 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.377 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.377 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.377 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.377 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.377 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.377 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.377 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.377 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.377 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.377 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.377 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.377 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.377 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.377 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.377 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.377 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.377 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.378 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.378 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.378 INFO:teuthology.orchestra.run.smithi098.stdout: "predict_interval": { 2024-05-09T10:51:35.378 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "predict_interval", 2024-05-09T10:51:35.378 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.378 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.378 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.378 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "86400", 2024-05-09T10:51:35.378 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.378 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.378 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.378 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.378 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.378 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.378 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.378 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.379 INFO:teuthology.orchestra.run.smithi098.stdout: "predictor_model": { 2024-05-09T10:51:35.379 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "predictor_model", 2024-05-09T10:51:35.379 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.379 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.379 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.379 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "prophetstor", 2024-05-09T10:51:35.379 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.379 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.379 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.379 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.379 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.379 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.379 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.379 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.379 INFO:teuthology.orchestra.run.smithi098.stdout: "sleep_interval": { 2024-05-09T10:51:35.379 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "sleep_interval", 2024-05-09T10:51:35.380 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.380 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.380 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.380 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "600", 2024-05-09T10:51:35.380 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.380 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.380 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.380 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.380 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.380 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.380 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.380 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.380 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.380 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.380 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.380 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "influx", 2024-05-09T10:51:35.380 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": false, 2024-05-09T10:51:35.381 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "influxdb python module not found", 2024-05-09T10:51:35.381 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.381 INFO:teuthology.orchestra.run.smithi098.stdout: "batch_size": { 2024-05-09T10:51:35.381 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "batch_size", 2024-05-09T10:51:35.381 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.381 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.381 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.381 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "5000", 2024-05-09T10:51:35.381 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.381 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.381 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.381 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T10:51:35.381 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.381 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.381 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.381 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.381 INFO:teuthology.orchestra.run.smithi098.stdout: "database": { 2024-05-09T10:51:35.381 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "database", 2024-05-09T10:51:35.382 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.382 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.382 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.382 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "ceph", 2024-05-09T10:51:35.382 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.382 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.382 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.382 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T10:51:35.382 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.382 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.382 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.382 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.382 INFO:teuthology.orchestra.run.smithi098.stdout: "hostname": { 2024-05-09T10:51:35.382 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "hostname", 2024-05-09T10:51:35.382 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.382 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.383 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.383 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.383 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.383 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.383 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.383 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "InfluxDB server hostname", 2024-05-09T10:51:35.383 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.383 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.383 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.383 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.383 INFO:teuthology.orchestra.run.smithi098.stdout: "interval": { 2024-05-09T10:51:35.383 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "interval", 2024-05-09T10:51:35.383 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.383 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.383 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.383 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "30", 2024-05-09T10:51:35.383 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "5", 2024-05-09T10:51:35.384 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.384 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.384 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T10:51:35.384 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.384 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.384 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.384 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.384 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.384 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.384 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.384 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.384 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.384 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.384 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.384 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.384 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.385 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.385 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.385 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.385 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.385 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.385 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.385 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.385 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.385 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.385 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.385 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.385 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.385 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.385 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.385 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.386 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.386 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.386 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.386 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.386 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.386 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.386 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.386 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.386 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.386 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.386 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.386 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.386 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.387 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.387 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.387 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.387 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.387 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.387 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.387 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.387 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.387 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.387 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.387 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.387 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.387 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.387 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.388 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.388 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.388 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.388 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.388 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.388 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.388 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.388 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.388 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.388 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.388 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.388 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.388 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.388 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.389 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.389 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.389 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.389 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.389 INFO:teuthology.orchestra.run.smithi098.stdout: "password": { 2024-05-09T10:51:35.389 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "password", 2024-05-09T10:51:35.389 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.389 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.389 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.389 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.389 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.389 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.389 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.389 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "password of InfluxDB server user", 2024-05-09T10:51:35.390 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.390 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.390 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.390 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.390 INFO:teuthology.orchestra.run.smithi098.stdout: "port": { 2024-05-09T10:51:35.390 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "port", 2024-05-09T10:51:35.390 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.390 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.390 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.390 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "8086", 2024-05-09T10:51:35.390 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.390 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.390 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.391 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "InfluxDB server port", 2024-05-09T10:51:35.391 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.391 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.391 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.391 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.391 INFO:teuthology.orchestra.run.smithi098.stdout: "ssl": { 2024-05-09T10:51:35.391 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ssl", 2024-05-09T10:51:35.391 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.391 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.391 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.391 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "false", 2024-05-09T10:51:35.393 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.393 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.394 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.394 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T10:51:35.394 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.394 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.394 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.394 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.394 INFO:teuthology.orchestra.run.smithi098.stdout: "threads": { 2024-05-09T10:51:35.394 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "threads", 2024-05-09T10:51:35.394 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.394 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.394 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.394 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "5", 2024-05-09T10:51:35.394 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "1", 2024-05-09T10:51:35.395 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "32", 2024-05-09T10:51:35.395 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.395 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T10:51:35.395 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.395 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.395 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.395 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.395 INFO:teuthology.orchestra.run.smithi098.stdout: "username": { 2024-05-09T10:51:35.395 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "username", 2024-05-09T10:51:35.395 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.395 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.395 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.395 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.395 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.396 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.396 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.396 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "username of InfluxDB server user", 2024-05-09T10:51:35.396 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.396 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.396 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.396 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.396 INFO:teuthology.orchestra.run.smithi098.stdout: "verify_ssl": { 2024-05-09T10:51:35.396 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "verify_ssl", 2024-05-09T10:51:35.396 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.396 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.396 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.396 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "true", 2024-05-09T10:51:35.396 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.396 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.397 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.397 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T10:51:35.397 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.397 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.397 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.397 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.397 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.397 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.397 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.397 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "insights", 2024-05-09T10:51:35.397 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.397 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.397 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.397 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.397 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.397 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.397 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.398 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.398 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.398 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.398 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.398 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.398 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.398 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.398 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.398 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.398 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.398 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.398 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.398 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.398 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.398 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.398 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.399 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.399 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.399 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.399 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.399 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.399 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.399 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.399 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.399 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.399 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.399 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.399 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.399 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.400 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.400 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.400 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.400 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.400 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.400 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.400 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.400 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.400 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.400 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.400 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.400 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.400 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.400 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.400 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.401 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.401 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.401 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.401 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.401 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.401 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.401 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.401 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.401 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.401 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.401 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.401 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.401 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.401 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.402 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.402 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.402 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.402 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.402 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.402 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.402 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.402 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.402 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "iostat", 2024-05-09T10:51:35.402 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.402 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.402 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.402 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.402 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.402 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.402 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.403 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.403 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.403 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.403 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.403 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.403 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.403 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.403 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.403 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.403 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.403 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.403 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.403 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.403 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.403 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.403 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.404 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.404 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.404 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.404 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.404 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.404 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.404 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.404 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.404 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.404 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.404 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.404 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.404 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.404 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.404 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.404 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.404 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.405 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.405 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.405 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.405 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.405 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.405 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.405 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.405 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.405 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.405 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.405 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.405 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.405 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.405 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.405 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.405 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.405 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.406 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.406 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.406 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.406 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.406 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.406 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.406 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.406 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.406 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.406 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.406 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.406 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.406 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.406 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.406 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.406 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.407 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.407 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.407 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "localpool", 2024-05-09T10:51:35.407 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.407 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.407 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.407 INFO:teuthology.orchestra.run.smithi098.stdout: "failure_domain": { 2024-05-09T10:51:35.407 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "failure_domain", 2024-05-09T10:51:35.407 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.407 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.407 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.407 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "host", 2024-05-09T10:51:35.407 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.407 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.408 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.408 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "failure domain for any created local pool", 2024-05-09T10:51:35.408 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T10:51:35.408 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.408 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.408 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.408 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.408 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.408 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.408 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.408 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.408 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.408 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.408 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.409 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.409 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.409 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.409 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.409 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.409 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.409 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.409 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.409 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.409 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.409 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.409 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.409 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.409 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.410 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.410 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.410 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.410 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.410 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.410 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.410 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.410 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.410 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.410 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.410 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.410 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.410 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.410 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.410 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.410 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.411 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.411 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.411 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.411 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.411 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.411 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.411 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.411 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.411 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.411 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.411 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.411 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.411 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.411 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.411 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.411 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.411 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.412 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.412 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.412 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.412 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.412 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.412 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.412 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.412 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.412 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.412 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.412 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.412 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.412 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.412 INFO:teuthology.orchestra.run.smithi098.stdout: "min_size": { 2024-05-09T10:51:35.412 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "min_size", 2024-05-09T10:51:35.412 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.413 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.413 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.413 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.413 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.413 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.413 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.413 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "default min_size for any created local pool", 2024-05-09T10:51:35.413 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T10:51:35.413 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.413 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.413 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.413 INFO:teuthology.orchestra.run.smithi098.stdout: "num_rep": { 2024-05-09T10:51:35.413 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "num_rep", 2024-05-09T10:51:35.413 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.413 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.413 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.413 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "3", 2024-05-09T10:51:35.413 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.414 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.414 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.414 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "default replica count for any created local pool", 2024-05-09T10:51:35.414 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.414 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.414 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.414 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.414 INFO:teuthology.orchestra.run.smithi098.stdout: "pg_num": { 2024-05-09T10:51:35.414 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "pg_num", 2024-05-09T10:51:35.414 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.414 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.414 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.414 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "128", 2024-05-09T10:51:35.414 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.415 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.415 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.415 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T10:51:35.415 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.415 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.415 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.415 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.415 INFO:teuthology.orchestra.run.smithi098.stdout: "prefix": { 2024-05-09T10:51:35.415 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "prefix", 2024-05-09T10:51:35.415 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.415 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.415 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.415 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.415 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.415 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.415 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.415 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "name prefix for any created local pool", 2024-05-09T10:51:35.416 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.416 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.416 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.416 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.416 INFO:teuthology.orchestra.run.smithi098.stdout: "subtree": { 2024-05-09T10:51:35.416 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "subtree", 2024-05-09T10:51:35.416 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.416 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.416 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.416 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "rack", 2024-05-09T10:51:35.416 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.416 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.416 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.416 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T10:51:35.416 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T10:51:35.416 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.416 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.417 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.417 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.417 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.417 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.417 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "mds_autoscaler", 2024-05-09T10:51:35.417 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.417 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.417 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.417 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.417 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.417 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.417 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.417 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.417 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.417 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.417 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.417 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.418 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.418 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.418 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.418 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.418 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.418 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.418 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.418 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.418 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.418 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.418 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.418 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.418 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.418 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.418 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.418 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.419 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.419 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.419 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.419 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.419 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.419 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.419 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.419 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.419 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.419 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.419 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.419 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.419 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.419 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.419 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.419 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.420 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.420 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.420 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.420 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.420 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.420 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.420 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.420 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.420 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.420 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.420 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.420 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.420 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.420 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.420 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.420 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.420 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.420 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.421 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.421 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.421 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.421 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.421 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.421 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.421 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.421 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.421 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.421 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.421 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.421 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.421 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.421 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.421 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "mirroring", 2024-05-09T10:51:35.421 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.422 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.422 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.422 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.422 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.422 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.422 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.422 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.422 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.422 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.422 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.422 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.422 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.422 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.422 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.422 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.422 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.423 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.423 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.423 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.423 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.423 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.423 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.423 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.423 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.423 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.423 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.423 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.423 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.423 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.423 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.423 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.423 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.424 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.424 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.424 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.424 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.424 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.424 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.424 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.424 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.424 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.424 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.424 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.424 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.424 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.424 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.424 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.424 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.424 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.425 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.425 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.425 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.425 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.425 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.425 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.425 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.425 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.425 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.425 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.425 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.425 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.425 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.425 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.425 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.426 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.426 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.426 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.426 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.426 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.426 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.426 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.426 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.426 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "nfs", 2024-05-09T10:51:35.426 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.426 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.426 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.426 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.426 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.426 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.427 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.427 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.427 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.427 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.427 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.427 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.427 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.427 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.427 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.427 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.427 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.427 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.427 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.427 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.427 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.428 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.428 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.428 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.428 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.428 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.428 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.428 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.428 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.428 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.428 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.428 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.428 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.428 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.428 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.428 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.428 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.429 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.429 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.429 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.429 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.429 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.429 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.429 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.429 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.429 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.429 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.429 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.429 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.429 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.429 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.429 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.429 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.430 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.430 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.430 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.430 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.430 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.430 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.430 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.430 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.430 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.430 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.430 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.430 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.430 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.430 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.430 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.430 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.430 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.431 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.431 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.431 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.431 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.431 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.431 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "orchestrator", 2024-05-09T10:51:35.431 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.431 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.431 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.431 INFO:teuthology.orchestra.run.smithi098.stdout: "fail_fs": { 2024-05-09T10:51:35.431 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "fail_fs", 2024-05-09T10:51:35.431 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.431 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.431 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.431 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.431 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.431 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.432 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.432 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T10:51:35.432 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.432 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.432 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.432 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.432 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.432 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.432 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.432 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.432 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.432 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.432 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.432 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.432 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.432 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.432 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.432 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.433 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.433 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.433 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.433 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.433 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.433 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.433 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.433 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.433 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.433 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.433 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.433 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.433 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.433 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.433 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.433 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.433 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.434 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.434 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.434 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.434 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.434 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.434 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.434 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.434 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.434 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.434 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.434 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.434 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.434 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.434 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.434 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.434 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.435 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.435 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.435 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.435 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.435 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.435 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.435 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.435 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.435 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.435 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.435 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.435 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.435 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.435 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.435 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.435 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.435 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.435 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.436 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.436 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.436 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.436 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.436 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.436 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.436 INFO:teuthology.orchestra.run.smithi098.stdout: "orchestrator": { 2024-05-09T10:51:35.436 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "orchestrator", 2024-05-09T10:51:35.436 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.436 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.436 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.436 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.436 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.436 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.436 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.436 INFO:teuthology.orchestra.run.smithi098.stdout: "cephadm", 2024-05-09T10:51:35.437 INFO:teuthology.orchestra.run.smithi098.stdout: "rook", 2024-05-09T10:51:35.437 INFO:teuthology.orchestra.run.smithi098.stdout: "test_orchestrator" 2024-05-09T10:51:35.437 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.437 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Orchestrator backend", 2024-05-09T10:51:35.437 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.437 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.437 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.437 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.437 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.437 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.437 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "osd_perf_query", 2024-05-09T10:51:35.437 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.437 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.437 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.437 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.438 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.438 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.438 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.438 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.438 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.438 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.438 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.438 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.438 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.438 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.438 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.438 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.438 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.438 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.438 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.439 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.439 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.439 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.439 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.439 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.439 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.439 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.439 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.439 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.439 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.439 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.439 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.439 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.439 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.439 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.439 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.440 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.440 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.440 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.440 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.440 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.440 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.440 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.440 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.440 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.440 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.440 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.440 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.440 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.440 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.440 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.441 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.441 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.441 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.441 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.441 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.441 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.441 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.441 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.441 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.441 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.441 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.441 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.441 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.441 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.441 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.441 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.441 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.442 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.442 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.442 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.442 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.442 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.442 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.442 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.442 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "osd_support", 2024-05-09T10:51:35.442 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.442 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.442 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.442 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.442 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.442 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.442 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.443 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.443 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.443 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.443 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.443 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.443 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.443 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.443 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.443 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.443 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.443 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.443 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.443 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.443 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.443 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.443 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.443 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.444 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.444 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.444 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.444 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.444 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.444 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.444 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.444 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.444 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.444 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.444 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.444 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.444 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.444 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.444 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.445 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.445 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.445 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.445 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.445 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.445 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.445 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.445 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.445 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.445 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.445 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.445 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.445 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.445 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.445 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.446 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.446 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.446 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.446 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.446 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.446 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.446 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.446 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.446 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.446 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.446 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.446 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.446 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.446 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.446 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.447 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.447 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.447 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.447 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.447 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.447 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.447 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "pg_autoscaler", 2024-05-09T10:51:35.447 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.447 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.447 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.447 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.447 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.447 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.447 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.447 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.447 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.447 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.448 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.448 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.448 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.448 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.448 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.448 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.448 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.448 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.448 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.448 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.448 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.448 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.448 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.448 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.449 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.449 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.449 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.449 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.449 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.449 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.449 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.449 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.449 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.449 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.449 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.449 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.449 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.449 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.449 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.450 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.450 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.450 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.450 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.450 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.450 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.450 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.450 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.450 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.450 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.450 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.450 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.450 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.450 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.451 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.451 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.451 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.451 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.451 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.451 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.451 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.451 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.451 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.451 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.451 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.451 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.451 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.451 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.452 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.452 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.452 INFO:teuthology.orchestra.run.smithi098.stdout: "sleep_interval": { 2024-05-09T10:51:35.452 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "sleep_interval", 2024-05-09T10:51:35.452 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.452 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.452 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.452 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "60", 2024-05-09T10:51:35.452 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.452 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.452 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.452 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.452 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.452 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.453 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.453 INFO:teuthology.orchestra.run.smithi098.stdout: "threshold": { 2024-05-09T10:51:35.453 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "threshold", 2024-05-09T10:51:35.453 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-05-09T10:51:35.453 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.453 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.453 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "3.0", 2024-05-09T10:51:35.453 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "1.0", 2024-05-09T10:51:35.453 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.453 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.453 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "scaling threshold", 2024-05-09T10:51:35.453 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T10:51:35.453 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.453 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.454 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.454 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.454 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.454 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.454 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "progress", 2024-05-09T10:51:35.454 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.454 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.454 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.454 INFO:teuthology.orchestra.run.smithi098.stdout: "allow_pg_recovery_event": { 2024-05-09T10:51:35.454 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "allow_pg_recovery_event", 2024-05-09T10:51:35.454 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.454 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.454 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.454 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.455 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.455 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.455 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.455 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T10:51:35.455 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.455 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.455 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.455 INFO:teuthology.orchestra.run.smithi098.stdout: "enabled": { 2024-05-09T10:51:35.455 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "enabled", 2024-05-09T10:51:35.455 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.455 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.455 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.455 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.455 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.456 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.456 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.456 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.456 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.456 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.456 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.456 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.456 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.456 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.456 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.456 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.456 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.456 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.456 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.457 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.457 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.457 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.457 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.457 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.457 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.457 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.457 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.457 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.457 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.457 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.457 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.457 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.457 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.457 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.458 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.458 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.458 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.458 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.458 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.458 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.458 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.458 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.458 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.458 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.458 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.458 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.458 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.458 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.458 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.459 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.459 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.459 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.459 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.459 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.459 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.459 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.459 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.459 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.459 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.459 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.459 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.459 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.459 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.459 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.459 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.460 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.460 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.460 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.460 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.460 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.460 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.460 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.460 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.460 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.460 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.460 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.460 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.460 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.460 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.460 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.460 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.461 INFO:teuthology.orchestra.run.smithi098.stdout: "max_completed_events": { 2024-05-09T10:51:35.461 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "max_completed_events", 2024-05-09T10:51:35.461 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.461 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.461 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.461 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "50", 2024-05-09T10:51:35.461 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.461 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.461 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.461 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "number of past completed events to remember", 2024-05-09T10:51:35.461 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.461 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.461 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.461 INFO:teuthology.orchestra.run.smithi098.stdout: "sleep_interval": { 2024-05-09T10:51:35.462 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "sleep_interval", 2024-05-09T10:51:35.462 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.462 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.462 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.462 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "5", 2024-05-09T10:51:35.462 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.462 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.462 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.462 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how long the module is going to sleep", 2024-05-09T10:51:35.462 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.462 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.462 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.462 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.462 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.463 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.463 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "prometheus", 2024-05-09T10:51:35.463 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.463 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.463 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.463 INFO:teuthology.orchestra.run.smithi098.stdout: "cache": { 2024-05-09T10:51:35.463 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cache", 2024-05-09T10:51:35.463 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.463 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.463 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.463 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.463 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.463 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.463 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.463 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.463 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.464 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.464 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.464 INFO:teuthology.orchestra.run.smithi098.stdout: "exclude_perf_counters": { 2024-05-09T10:51:35.464 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "exclude_perf_counters", 2024-05-09T10:51:35.464 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.464 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.464 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.464 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.464 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.464 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.464 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.464 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T10:51:35.464 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T10:51:35.464 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.464 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.465 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.465 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.465 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.465 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.465 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.465 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.465 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.465 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.465 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.465 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.465 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.465 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.465 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.465 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.466 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.466 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.466 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.466 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.466 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.466 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.466 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.466 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.466 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.466 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.466 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.466 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.466 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.466 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.467 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.467 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.467 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.467 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.467 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.467 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.467 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.467 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.467 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.467 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.467 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.467 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.467 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.467 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.467 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.467 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.468 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.468 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.468 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.468 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.468 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.468 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.468 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.468 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.468 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.468 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.468 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.468 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.468 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.468 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.469 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.469 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.469 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.469 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.469 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.469 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.469 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.469 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.469 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.469 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.469 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.469 INFO:teuthology.orchestra.run.smithi098.stdout: "rbd_stats_pools": { 2024-05-09T10:51:35.469 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rbd_stats_pools", 2024-05-09T10:51:35.469 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.470 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.470 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.470 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.470 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.470 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.470 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.470 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.470 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.470 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.470 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.470 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.470 INFO:teuthology.orchestra.run.smithi098.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T10:51:35.470 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T10:51:35.470 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.470 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.471 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.471 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "300", 2024-05-09T10:51:35.471 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.471 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.471 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.471 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.471 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.471 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.471 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.471 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.471 INFO:teuthology.orchestra.run.smithi098.stdout: "scrape_interval": { 2024-05-09T10:51:35.471 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "scrape_interval", 2024-05-09T10:51:35.471 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-05-09T10:51:35.471 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.471 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.471 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "15.0", 2024-05-09T10:51:35.471 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.472 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.472 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.472 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.472 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.472 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.472 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.472 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.472 INFO:teuthology.orchestra.run.smithi098.stdout: "server_addr": { 2024-05-09T10:51:35.472 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_addr", 2024-05-09T10:51:35.472 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.472 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.472 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.472 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "::", 2024-05-09T10:51:35.472 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.472 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.472 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.472 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T10:51:35.473 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.473 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.473 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.473 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.473 INFO:teuthology.orchestra.run.smithi098.stdout: "server_port": { 2024-05-09T10:51:35.473 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_port", 2024-05-09T10:51:35.473 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.473 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.473 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.473 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "9283", 2024-05-09T10:51:35.473 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.473 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.473 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.473 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T10:51:35.473 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.474 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.474 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.474 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.474 INFO:teuthology.orchestra.run.smithi098.stdout: "stale_cache_strategy": { 2024-05-09T10:51:35.474 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "stale_cache_strategy", 2024-05-09T10:51:35.474 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.474 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.474 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.474 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "log", 2024-05-09T10:51:35.474 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.474 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.474 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.474 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.474 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.475 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.475 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.475 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.475 INFO:teuthology.orchestra.run.smithi098.stdout: "standby_behaviour": { 2024-05-09T10:51:35.475 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "standby_behaviour", 2024-05-09T10:51:35.475 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.475 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.475 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.475 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "default", 2024-05-09T10:51:35.475 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.475 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.475 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.475 INFO:teuthology.orchestra.run.smithi098.stdout: "default", 2024-05-09T10:51:35.475 INFO:teuthology.orchestra.run.smithi098.stdout: "error" 2024-05-09T10:51:35.475 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.475 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.476 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.476 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.476 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.476 INFO:teuthology.orchestra.run.smithi098.stdout: "standby_error_status_code": { 2024-05-09T10:51:35.476 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "standby_error_status_code", 2024-05-09T10:51:35.476 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.476 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.476 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.476 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "500", 2024-05-09T10:51:35.476 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "400", 2024-05-09T10:51:35.476 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "599", 2024-05-09T10:51:35.476 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.476 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.476 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.476 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.477 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.477 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.477 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.477 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.477 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.477 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rbd_support", 2024-05-09T10:51:35.477 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.477 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.477 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.477 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.477 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.477 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.477 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.477 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.477 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.477 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.478 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.478 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.478 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.478 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.478 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.478 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.478 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.478 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.478 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.478 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.478 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.478 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.478 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.478 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.478 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.479 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.479 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.479 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.479 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.479 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.479 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.479 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.479 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.479 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.479 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.479 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.479 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.479 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.479 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.479 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.480 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.480 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.480 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.480 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.480 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.480 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.480 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.480 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.480 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.480 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.480 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.480 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.480 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.480 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.480 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.480 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.480 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.480 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.481 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.481 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.481 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.481 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.481 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.481 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.481 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.481 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.481 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.481 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.481 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.481 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.481 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.481 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.481 INFO:teuthology.orchestra.run.smithi098.stdout: "max_concurrent_snap_create": { 2024-05-09T10:51:35.482 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "max_concurrent_snap_create", 2024-05-09T10:51:35.482 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.482 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.482 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.482 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-05-09T10:51:35.482 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.482 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.482 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.482 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.482 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.482 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.482 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.482 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.483 INFO:teuthology.orchestra.run.smithi098.stdout: "mirror_snapshot_schedule": { 2024-05-09T10:51:35.483 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T10:51:35.483 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.483 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.483 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.483 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.483 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.483 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.483 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.483 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.483 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.483 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.483 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.483 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.483 INFO:teuthology.orchestra.run.smithi098.stdout: "trash_purge_schedule": { 2024-05-09T10:51:35.483 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "trash_purge_schedule", 2024-05-09T10:51:35.483 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.484 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.484 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.484 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.484 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.484 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.484 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.484 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.484 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.484 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.484 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.484 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.484 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.484 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.484 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.484 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "restful", 2024-05-09T10:51:35.484 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.485 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.485 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.485 INFO:teuthology.orchestra.run.smithi098.stdout: "enable_auth": { 2024-05-09T10:51:35.485 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "enable_auth", 2024-05-09T10:51:35.485 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.485 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.485 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.485 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.485 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.485 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.485 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.485 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.485 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.485 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.485 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.485 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.485 INFO:teuthology.orchestra.run.smithi098.stdout: "key_file": { 2024-05-09T10:51:35.485 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "key_file", 2024-05-09T10:51:35.486 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.486 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.486 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.486 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.486 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.486 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.486 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.486 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.486 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.486 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.486 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.486 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.486 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.486 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.487 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.487 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.487 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.487 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.487 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.487 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.487 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.487 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.487 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.487 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.487 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.487 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.487 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.487 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.487 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.487 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.487 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.488 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.488 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.488 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.488 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.488 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.488 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.488 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.488 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.488 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.488 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.488 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.488 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.488 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.488 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.488 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.488 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.488 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.489 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.489 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.489 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.489 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.489 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.489 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.489 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.489 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.489 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.489 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.489 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.489 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.489 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.489 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.489 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.490 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.490 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.490 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.490 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.490 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.490 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.490 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.490 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.490 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.490 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.490 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.490 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.490 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.490 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.490 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.491 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.491 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.491 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.491 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.491 INFO:teuthology.orchestra.run.smithi098.stdout: "server_addr": { 2024-05-09T10:51:35.491 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_addr", 2024-05-09T10:51:35.491 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.491 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.491 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.491 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.491 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.491 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.491 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.491 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.491 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.492 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.492 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.492 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.492 INFO:teuthology.orchestra.run.smithi098.stdout: "server_port": { 2024-05-09T10:51:35.492 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_port", 2024-05-09T10:51:35.492 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.492 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.492 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.492 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.492 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.492 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.492 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.492 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.492 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.492 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.493 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.493 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.493 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.493 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.493 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.493 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rgw", 2024-05-09T10:51:35.493 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.493 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.493 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.493 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.493 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.493 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.493 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.493 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.494 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.494 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.494 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.494 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.494 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.494 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.494 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.494 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.494 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.494 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.494 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.494 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.494 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.494 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.495 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.496 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.496 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.496 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.496 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.496 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.496 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.496 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.496 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.496 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.496 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.496 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.496 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.496 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.496 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.496 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.497 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.497 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.497 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.497 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.497 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.497 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.497 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.497 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.497 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.497 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.497 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.497 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.497 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.497 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.497 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.498 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.498 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.498 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.498 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.498 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.498 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.498 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.498 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.498 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.498 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rook", 2024-05-09T10:51:35.498 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.498 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.498 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.499 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.499 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.499 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.499 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.499 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.499 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.499 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.499 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.499 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.499 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.499 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.499 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.499 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.499 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.499 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.500 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.500 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.500 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.500 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.500 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.500 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.500 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.500 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.500 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.500 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.500 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.500 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.500 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.501 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.501 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.501 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.501 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.501 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.501 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.501 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.501 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.501 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.501 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.501 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.501 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.501 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.501 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.502 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.502 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.502 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.502 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.502 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.502 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.502 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.502 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.502 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.502 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.502 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.502 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.502 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.502 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.502 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.503 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.503 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.503 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.503 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.503 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.503 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.503 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.503 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.503 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.503 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.503 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.503 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.503 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.504 INFO:teuthology.orchestra.run.smithi098.stdout: "storage_class": { 2024-05-09T10:51:35.504 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "storage_class", 2024-05-09T10:51:35.504 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.504 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.504 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.504 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "local", 2024-05-09T10:51:35.504 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.504 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.504 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.504 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T10:51:35.504 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.504 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.504 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.504 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.504 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.505 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "selftest", 2024-05-09T10:51:35.505 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.505 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.505 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.505 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.505 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.505 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.505 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.505 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.505 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.505 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.505 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.505 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.506 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.506 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.506 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.506 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.506 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.506 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.506 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.506 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.506 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.506 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.506 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.506 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.506 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.507 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.507 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.507 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.507 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.507 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.507 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.507 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.507 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.507 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.507 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.507 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.507 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.507 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.507 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.508 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.508 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.508 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.508 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.508 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.508 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.508 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.508 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.508 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.508 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.508 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.508 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.508 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.508 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.508 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.508 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.509 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.509 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.509 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.509 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.509 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.509 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.509 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.509 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.509 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.509 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.509 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.509 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.509 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.509 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.510 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.510 INFO:teuthology.orchestra.run.smithi098.stdout: "roption1": { 2024-05-09T10:51:35.510 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "roption1", 2024-05-09T10:51:35.510 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.510 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.510 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.510 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.510 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.510 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.510 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.510 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.510 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.510 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.511 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.511 INFO:teuthology.orchestra.run.smithi098.stdout: "roption2": { 2024-05-09T10:51:35.511 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "roption2", 2024-05-09T10:51:35.511 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.511 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.511 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.511 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "xyz", 2024-05-09T10:51:35.511 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.511 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.511 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.511 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.511 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.511 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.511 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.512 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption1": { 2024-05-09T10:51:35.512 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption1", 2024-05-09T10:51:35.512 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.512 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.512 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.512 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.512 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.512 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.512 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.512 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.512 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.512 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.512 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.513 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption2": { 2024-05-09T10:51:35.513 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption2", 2024-05-09T10:51:35.513 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.513 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.513 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.513 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.513 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.513 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.513 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.513 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.513 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.513 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.513 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.513 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.513 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption3": { 2024-05-09T10:51:35.514 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption3", 2024-05-09T10:51:35.514 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-05-09T10:51:35.514 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.514 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.514 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.514 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.514 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.514 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.514 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.514 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.514 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.514 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.514 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption4": { 2024-05-09T10:51:35.515 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption4", 2024-05-09T10:51:35.515 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.515 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.515 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.515 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.515 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.515 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.515 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.515 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.515 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.515 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.515 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.515 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.515 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption5": { 2024-05-09T10:51:35.515 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption5", 2024-05-09T10:51:35.516 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.516 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.516 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.516 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.516 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.516 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.516 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.516 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.516 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.516 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.516 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.516 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.516 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption6": { 2024-05-09T10:51:35.516 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption6", 2024-05-09T10:51:35.516 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.516 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.517 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.517 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.517 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.517 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.517 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.517 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.517 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.517 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.517 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.517 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.517 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption7": { 2024-05-09T10:51:35.517 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption7", 2024-05-09T10:51:35.517 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.517 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.517 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.517 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.517 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "1", 2024-05-09T10:51:35.518 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "42", 2024-05-09T10:51:35.518 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.518 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.518 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.518 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.518 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.518 INFO:teuthology.orchestra.run.smithi098.stdout: "testkey": { 2024-05-09T10:51:35.518 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "testkey", 2024-05-09T10:51:35.518 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.518 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.518 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.518 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.518 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.518 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.519 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.519 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.519 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.519 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.519 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.519 INFO:teuthology.orchestra.run.smithi098.stdout: "testlkey": { 2024-05-09T10:51:35.519 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "testlkey", 2024-05-09T10:51:35.519 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.519 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.519 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.519 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.519 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.519 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.520 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.520 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.520 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.520 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.520 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.520 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.520 INFO:teuthology.orchestra.run.smithi098.stdout: "testnewline": { 2024-05-09T10:51:35.520 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "testnewline", 2024-05-09T10:51:35.520 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.520 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.520 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.520 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.520 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.520 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.520 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.521 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.521 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.521 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.521 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.521 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.521 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.521 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.521 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.521 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "snap_schedule", 2024-05-09T10:51:35.521 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.521 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.521 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.521 INFO:teuthology.orchestra.run.smithi098.stdout: "allow_m_granularity": { 2024-05-09T10:51:35.521 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "allow_m_granularity", 2024-05-09T10:51:35.521 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.521 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.522 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.522 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.522 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.522 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.522 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.522 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T10:51:35.522 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.522 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.522 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.522 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.522 INFO:teuthology.orchestra.run.smithi098.stdout: "dump_on_update": { 2024-05-09T10:51:35.522 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "dump_on_update", 2024-05-09T10:51:35.522 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.522 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.522 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.523 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.523 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.523 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.523 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.523 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "dump database to debug log on update", 2024-05-09T10:51:35.523 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.523 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.523 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.523 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.523 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.523 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.523 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.523 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.523 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.523 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.523 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.524 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.524 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.524 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.524 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.524 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.524 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.524 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.524 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.524 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.524 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.524 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.524 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.524 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.524 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.524 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.524 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.524 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.525 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.525 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.525 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.525 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.525 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.525 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.525 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.525 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.525 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.525 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.525 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.525 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.525 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.525 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.525 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.525 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.526 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.526 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.526 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.526 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.526 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.526 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.526 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.526 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.526 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.526 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.526 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.526 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.526 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.526 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.527 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.527 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.527 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.527 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.527 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.527 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.527 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.527 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.527 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.527 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.527 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.527 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.527 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.527 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.527 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.527 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.528 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.528 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.528 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.528 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "stats", 2024-05-09T10:51:35.528 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.528 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.528 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.528 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.528 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.528 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.528 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.528 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.528 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.528 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.529 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.529 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.529 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.529 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.529 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.529 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.529 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.529 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.529 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.529 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.529 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.529 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.529 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.529 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.530 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.530 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.530 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.530 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.530 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.530 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.530 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.530 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.530 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.530 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.530 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.530 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.530 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.530 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.531 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.531 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.531 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.531 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.531 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.531 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.531 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.531 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.531 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.531 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.531 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.531 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.531 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.531 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.531 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.531 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.532 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.532 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.532 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.532 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.532 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.532 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.532 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.532 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.532 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.532 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.532 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.532 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.532 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.532 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.532 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.532 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.532 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.532 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.533 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.533 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.533 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.533 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.533 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "status", 2024-05-09T10:51:35.533 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.533 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.533 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.533 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.533 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.533 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.533 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.533 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.533 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.533 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.533 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.533 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.534 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.534 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.534 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.534 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.534 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.534 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.534 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.534 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.534 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.534 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.534 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.534 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.534 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.534 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.534 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.535 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.535 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.535 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.535 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.535 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.535 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.535 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.535 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.535 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.535 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.535 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.535 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.535 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.535 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.535 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.535 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.535 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.536 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.536 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.536 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.536 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.536 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.536 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.536 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.536 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.536 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.536 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.536 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.536 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.536 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.536 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.536 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.536 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.536 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.537 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.537 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.537 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.537 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.537 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.537 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.537 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.537 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.537 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.537 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.537 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.537 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.537 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.537 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.537 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.538 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "telegraf", 2024-05-09T10:51:35.538 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.538 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.538 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.538 INFO:teuthology.orchestra.run.smithi098.stdout: "address": { 2024-05-09T10:51:35.538 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "address", 2024-05-09T10:51:35.538 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.538 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.538 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.538 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T10:51:35.538 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.538 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.538 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.538 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.538 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.538 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.539 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.539 INFO:teuthology.orchestra.run.smithi098.stdout: "interval": { 2024-05-09T10:51:35.539 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "interval", 2024-05-09T10:51:35.539 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.539 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.539 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.539 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "15", 2024-05-09T10:51:35.539 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.539 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.539 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.539 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.539 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.539 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.539 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.539 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.540 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.540 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.540 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.540 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.540 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.540 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.540 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.540 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.540 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.540 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.540 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.540 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.540 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.540 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.540 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.540 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.540 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.541 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.541 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.541 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.541 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.541 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.541 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.541 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.541 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.541 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.541 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.541 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.541 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.541 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.541 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.541 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.541 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.541 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.542 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.542 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.542 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.542 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.542 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.542 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.542 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.542 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.542 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.542 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.542 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.542 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.542 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.542 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.543 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.543 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.543 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.543 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.543 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.543 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.543 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.543 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.543 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.543 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.543 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.543 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.543 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.543 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.543 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.543 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.544 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.544 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.544 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.544 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.544 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.544 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.544 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "telemetry", 2024-05-09T10:51:35.544 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.544 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.544 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.544 INFO:teuthology.orchestra.run.smithi098.stdout: "channel_basic": { 2024-05-09T10:51:35.544 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "channel_basic", 2024-05-09T10:51:35.544 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.544 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.545 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.545 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.545 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.545 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.545 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.545 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T10:51:35.545 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.545 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.545 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.545 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.545 INFO:teuthology.orchestra.run.smithi098.stdout: "channel_crash": { 2024-05-09T10:51:35.545 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "channel_crash", 2024-05-09T10:51:35.545 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.545 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.546 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.546 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.546 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.546 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.546 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.546 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T10:51:35.546 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.546 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.546 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.546 INFO:teuthology.orchestra.run.smithi098.stdout: "channel_device": { 2024-05-09T10:51:35.546 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "channel_device", 2024-05-09T10:51:35.546 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.546 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.547 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.547 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.547 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.547 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.547 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.547 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T10:51:35.547 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.547 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.547 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.547 INFO:teuthology.orchestra.run.smithi098.stdout: "channel_ident": { 2024-05-09T10:51:35.547 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "channel_ident", 2024-05-09T10:51:35.547 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.547 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.547 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.547 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.548 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.548 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.548 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.548 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T10:51:35.548 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.548 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.548 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.548 INFO:teuthology.orchestra.run.smithi098.stdout: "channel_perf": { 2024-05-09T10:51:35.548 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "channel_perf", 2024-05-09T10:51:35.548 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.548 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.548 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.548 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.548 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.548 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.549 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.549 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T10:51:35.549 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.549 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.549 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.549 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.549 INFO:teuthology.orchestra.run.smithi098.stdout: "contact": { 2024-05-09T10:51:35.549 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "contact", 2024-05-09T10:51:35.549 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.549 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.549 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.549 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.549 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.549 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.549 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.550 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.550 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.550 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.550 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.550 INFO:teuthology.orchestra.run.smithi098.stdout: "description": { 2024-05-09T10:51:35.550 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "description", 2024-05-09T10:51:35.550 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.550 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.550 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.550 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.550 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.550 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.550 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.550 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.551 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.551 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.551 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.551 INFO:teuthology.orchestra.run.smithi098.stdout: "device_url": { 2024-05-09T10:51:35.551 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "device_url", 2024-05-09T10:51:35.551 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.551 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.551 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.551 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T10:51:35.551 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.551 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.551 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.551 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.551 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.551 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.552 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.552 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.552 INFO:teuthology.orchestra.run.smithi098.stdout: "enabled": { 2024-05-09T10:51:35.552 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "enabled", 2024-05-09T10:51:35.552 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.552 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.552 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.552 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.552 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.552 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.552 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.552 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.552 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.552 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.552 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.552 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.552 INFO:teuthology.orchestra.run.smithi098.stdout: "interval": { 2024-05-09T10:51:35.553 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "interval", 2024-05-09T10:51:35.553 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.553 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.553 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.553 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "24", 2024-05-09T10:51:35.553 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "8", 2024-05-09T10:51:35.553 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.553 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.553 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.553 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.553 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.553 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.553 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.553 INFO:teuthology.orchestra.run.smithi098.stdout: "last_opt_revision": { 2024-05-09T10:51:35.553 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "last_opt_revision", 2024-05-09T10:51:35.554 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.554 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.554 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.554 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "1", 2024-05-09T10:51:35.554 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.554 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.554 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.554 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.554 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.554 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.554 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.554 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.554 INFO:teuthology.orchestra.run.smithi098.stdout: "leaderboard": { 2024-05-09T10:51:35.554 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "leaderboard", 2024-05-09T10:51:35.555 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.555 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.555 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.555 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.555 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.555 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.555 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.555 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.555 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.555 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.555 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.555 INFO:teuthology.orchestra.run.smithi098.stdout: "leaderboard_description": { 2024-05-09T10:51:35.555 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "leaderboard_description", 2024-05-09T10:51:35.555 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.556 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.556 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.556 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.556 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.556 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.556 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.556 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.556 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.556 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.556 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.556 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.556 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.556 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.556 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.557 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.557 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.557 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.557 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.557 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.557 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.557 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.557 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.557 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.557 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.557 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.557 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.557 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.557 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.557 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.557 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.557 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.558 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.558 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.558 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.558 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.558 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.558 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.558 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.558 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.558 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.558 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.558 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.558 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.558 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.558 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.558 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.559 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.559 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.559 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.559 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.559 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.559 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.559 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.559 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.559 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.559 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.559 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.559 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.559 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.559 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.559 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.560 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.560 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.560 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.560 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.560 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.560 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.560 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.560 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.560 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.560 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.560 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.560 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.560 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.561 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.561 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.561 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.561 INFO:teuthology.orchestra.run.smithi098.stdout: "organization": { 2024-05-09T10:51:35.561 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "organization", 2024-05-09T10:51:35.561 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.561 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.561 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.561 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.561 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.561 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.561 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.561 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.561 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.561 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.562 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.562 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.562 INFO:teuthology.orchestra.run.smithi098.stdout: "proxy": { 2024-05-09T10:51:35.562 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "proxy", 2024-05-09T10:51:35.562 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.562 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.562 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.562 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.562 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.562 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.562 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.562 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.562 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.562 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.563 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.563 INFO:teuthology.orchestra.run.smithi098.stdout: "url": { 2024-05-09T10:51:35.563 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "url", 2024-05-09T10:51:35.563 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.563 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.563 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.563 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T10:51:35.563 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.563 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.563 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.563 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.563 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.563 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.563 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.563 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.564 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.564 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.564 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "test_orchestrator", 2024-05-09T10:51:35.564 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.564 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.564 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.564 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.564 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.564 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.564 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.564 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.564 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.564 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.564 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.564 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.565 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.565 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.565 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.565 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.565 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.565 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.565 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.565 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.565 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.565 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.565 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.565 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.565 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.565 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.565 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.565 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.566 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.566 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.566 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.566 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.566 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.566 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.566 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.566 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.566 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.566 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.566 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.566 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.566 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.566 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.567 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.567 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.567 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.567 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.567 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.567 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.567 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.567 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.567 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.567 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.567 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.567 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.567 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.567 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.568 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.568 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.568 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.568 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.568 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.568 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.568 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.568 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.568 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.568 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.568 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.568 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.568 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.568 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.568 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.569 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.569 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "volumes", 2024-05-09T10:51:35.569 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.569 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.569 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.569 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.569 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.569 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.569 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.569 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.569 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.569 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.569 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.570 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.570 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.570 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.570 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.570 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.570 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.570 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.570 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.570 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.570 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.570 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.570 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.570 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.570 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.571 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.571 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.571 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.571 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.571 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.571 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.571 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.571 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.571 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.571 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.571 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.571 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.571 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.571 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.571 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.571 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.571 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.572 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.572 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.572 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.572 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.572 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.572 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.572 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.572 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.572 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.572 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.572 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.572 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.572 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.572 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.572 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.572 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.572 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.573 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.573 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.573 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.573 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.573 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.573 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.573 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.573 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.573 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.573 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.573 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.573 INFO:teuthology.orchestra.run.smithi098.stdout: "max_concurrent_clones": { 2024-05-09T10:51:35.573 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "max_concurrent_clones", 2024-05-09T10:51:35.573 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.573 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.573 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.574 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "4", 2024-05-09T10:51:35.574 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.574 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.574 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.574 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T10:51:35.574 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.574 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.574 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.574 INFO:teuthology.orchestra.run.smithi098.stdout: "periodic_async_work": { 2024-05-09T10:51:35.574 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "periodic_async_work", 2024-05-09T10:51:35.574 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.574 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.574 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.574 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.574 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.575 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.575 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.575 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Periodically check for async work", 2024-05-09T10:51:35.575 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.575 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.575 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.575 INFO:teuthology.orchestra.run.smithi098.stdout: "snapshot_clone_delay": { 2024-05-09T10:51:35.575 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "snapshot_clone_delay", 2024-05-09T10:51:35.575 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.575 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.575 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.575 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0", 2024-05-09T10:51:35.575 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.575 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.575 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.575 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T10:51:35.576 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.576 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.576 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.576 INFO:teuthology.orchestra.run.smithi098.stdout: "snapshot_clone_no_wait": { 2024-05-09T10:51:35.576 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T10:51:35.576 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.576 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.576 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.576 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-05-09T10:51:35.576 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.576 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.576 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.576 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T10:51:35.576 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.576 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.577 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.577 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.577 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.577 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.577 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.577 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "zabbix", 2024-05-09T10:51:35.577 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-05-09T10:51:35.577 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-05-09T10:51:35.577 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-05-09T10:51:35.577 INFO:teuthology.orchestra.run.smithi098.stdout: "discovery_interval": { 2024-05-09T10:51:35.577 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "discovery_interval", 2024-05-09T10:51:35.577 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "uint", 2024-05-09T10:51:35.577 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.577 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.577 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "100", 2024-05-09T10:51:35.577 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.578 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.578 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.578 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.578 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.578 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.578 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.578 INFO:teuthology.orchestra.run.smithi098.stdout: "identifier": { 2024-05-09T10:51:35.578 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "identifier", 2024-05-09T10:51:35.578 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.578 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.578 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.578 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.578 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.578 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.579 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.579 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.579 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.579 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.579 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.579 INFO:teuthology.orchestra.run.smithi098.stdout: "interval": { 2024-05-09T10:51:35.579 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "interval", 2024-05-09T10:51:35.579 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-05-09T10:51:35.579 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.579 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.579 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "60", 2024-05-09T10:51:35.579 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.579 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.579 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.579 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.579 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.580 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.580 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.580 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.580 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-05-09T10:51:35.580 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-05-09T10:51:35.580 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.580 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.580 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.580 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.580 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.580 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.580 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.580 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.580 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.580 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.580 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.580 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.581 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.581 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.581 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.581 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.581 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.581 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.581 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-05-09T10:51:35.581 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-05-09T10:51:35.581 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.581 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.581 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.581 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.581 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.581 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.582 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.582 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.582 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.582 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.582 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.582 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.582 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-05-09T10:51:35.582 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-05-09T10:51:35.582 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.582 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.582 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.582 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-05-09T10:51:35.582 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.582 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.582 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-05-09T10:51:35.583 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-05-09T10:51:35.583 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-05-09T10:51:35.583 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-05-09T10:51:35.583 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-05-09T10:51:35.583 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-05-09T10:51:35.583 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-05-09T10:51:35.583 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.583 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.583 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.583 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.583 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.583 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-05-09T10:51:35.583 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-05-09T10:51:35.583 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-05-09T10:51:35.583 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.583 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-05-09T10:51:35.584 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-05-09T10:51:35.584 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.584 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.584 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.584 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.584 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.584 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.584 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.584 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.584 INFO:teuthology.orchestra.run.smithi098.stdout: "zabbix_host": { 2024-05-09T10:51:35.584 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "zabbix_host", 2024-05-09T10:51:35.584 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.584 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.584 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.584 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-05-09T10:51:35.584 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.585 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.585 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.585 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.585 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.585 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.585 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.585 INFO:teuthology.orchestra.run.smithi098.stdout: "zabbix_port": { 2024-05-09T10:51:35.585 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "zabbix_port", 2024-05-09T10:51:35.585 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-05-09T10:51:35.585 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.585 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.585 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10051", 2024-05-09T10:51:35.585 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.585 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.585 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.586 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.586 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.586 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.586 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.586 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.586 INFO:teuthology.orchestra.run.smithi098.stdout: "zabbix_sender": { 2024-05-09T10:51:35.586 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "zabbix_sender", 2024-05-09T10:51:35.586 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-05-09T10:51:35.586 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-05-09T10:51:35.586 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-05-09T10:51:35.586 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T10:51:35.586 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-05-09T10:51:35.586 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-05-09T10:51:35.586 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-05-09T10:51:35.586 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-05-09T10:51:35.587 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-05-09T10:51:35.587 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-05-09T10:51:35.587 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-05-09T10:51:35.587 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.587 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.587 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.587 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.587 INFO:teuthology.orchestra.run.smithi098.stdout: "services": {}, 2024-05-09T10:51:35.587 INFO:teuthology.orchestra.run.smithi098.stdout: "always_on_modules": { 2024-05-09T10:51:35.587 INFO:teuthology.orchestra.run.smithi098.stdout: "octopus": [ 2024-05-09T10:51:35.587 INFO:teuthology.orchestra.run.smithi098.stdout: "balancer", 2024-05-09T10:51:35.587 INFO:teuthology.orchestra.run.smithi098.stdout: "crash", 2024-05-09T10:51:35.587 INFO:teuthology.orchestra.run.smithi098.stdout: "devicehealth", 2024-05-09T10:51:35.587 INFO:teuthology.orchestra.run.smithi098.stdout: "orchestrator", 2024-05-09T10:51:35.587 INFO:teuthology.orchestra.run.smithi098.stdout: "pg_autoscaler", 2024-05-09T10:51:35.587 INFO:teuthology.orchestra.run.smithi098.stdout: "progress", 2024-05-09T10:51:35.588 INFO:teuthology.orchestra.run.smithi098.stdout: "rbd_support", 2024-05-09T10:51:35.588 INFO:teuthology.orchestra.run.smithi098.stdout: "status", 2024-05-09T10:51:35.588 INFO:teuthology.orchestra.run.smithi098.stdout: "telemetry", 2024-05-09T10:51:35.588 INFO:teuthology.orchestra.run.smithi098.stdout: "volumes" 2024-05-09T10:51:35.588 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.588 INFO:teuthology.orchestra.run.smithi098.stdout: "pacific": [ 2024-05-09T10:51:35.588 INFO:teuthology.orchestra.run.smithi098.stdout: "balancer", 2024-05-09T10:51:35.588 INFO:teuthology.orchestra.run.smithi098.stdout: "crash", 2024-05-09T10:51:35.588 INFO:teuthology.orchestra.run.smithi098.stdout: "devicehealth", 2024-05-09T10:51:35.588 INFO:teuthology.orchestra.run.smithi098.stdout: "orchestrator", 2024-05-09T10:51:35.588 INFO:teuthology.orchestra.run.smithi098.stdout: "pg_autoscaler", 2024-05-09T10:51:35.588 INFO:teuthology.orchestra.run.smithi098.stdout: "progress", 2024-05-09T10:51:35.588 INFO:teuthology.orchestra.run.smithi098.stdout: "rbd_support", 2024-05-09T10:51:35.588 INFO:teuthology.orchestra.run.smithi098.stdout: "status", 2024-05-09T10:51:35.588 INFO:teuthology.orchestra.run.smithi098.stdout: "telemetry", 2024-05-09T10:51:35.589 INFO:teuthology.orchestra.run.smithi098.stdout: "volumes" 2024-05-09T10:51:35.589 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.589 INFO:teuthology.orchestra.run.smithi098.stdout: "quincy": [ 2024-05-09T10:51:35.589 INFO:teuthology.orchestra.run.smithi098.stdout: "balancer", 2024-05-09T10:51:35.589 INFO:teuthology.orchestra.run.smithi098.stdout: "crash", 2024-05-09T10:51:35.589 INFO:teuthology.orchestra.run.smithi098.stdout: "devicehealth", 2024-05-09T10:51:35.589 INFO:teuthology.orchestra.run.smithi098.stdout: "orchestrator", 2024-05-09T10:51:35.589 INFO:teuthology.orchestra.run.smithi098.stdout: "pg_autoscaler", 2024-05-09T10:51:35.589 INFO:teuthology.orchestra.run.smithi098.stdout: "progress", 2024-05-09T10:51:35.589 INFO:teuthology.orchestra.run.smithi098.stdout: "rbd_support", 2024-05-09T10:51:35.589 INFO:teuthology.orchestra.run.smithi098.stdout: "status", 2024-05-09T10:51:35.589 INFO:teuthology.orchestra.run.smithi098.stdout: "telemetry", 2024-05-09T10:51:35.589 INFO:teuthology.orchestra.run.smithi098.stdout: "volumes" 2024-05-09T10:51:35.589 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.589 INFO:teuthology.orchestra.run.smithi098.stdout: "reef": [ 2024-05-09T10:51:35.590 INFO:teuthology.orchestra.run.smithi098.stdout: "balancer", 2024-05-09T10:51:35.590 INFO:teuthology.orchestra.run.smithi098.stdout: "crash", 2024-05-09T10:51:35.590 INFO:teuthology.orchestra.run.smithi098.stdout: "devicehealth", 2024-05-09T10:51:35.590 INFO:teuthology.orchestra.run.smithi098.stdout: "orchestrator", 2024-05-09T10:51:35.590 INFO:teuthology.orchestra.run.smithi098.stdout: "pg_autoscaler", 2024-05-09T10:51:35.590 INFO:teuthology.orchestra.run.smithi098.stdout: "progress", 2024-05-09T10:51:35.590 INFO:teuthology.orchestra.run.smithi098.stdout: "rbd_support", 2024-05-09T10:51:35.590 INFO:teuthology.orchestra.run.smithi098.stdout: "status", 2024-05-09T10:51:35.590 INFO:teuthology.orchestra.run.smithi098.stdout: "telemetry", 2024-05-09T10:51:35.590 INFO:teuthology.orchestra.run.smithi098.stdout: "volumes" 2024-05-09T10:51:35.590 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-05-09T10:51:35.590 INFO:teuthology.orchestra.run.smithi098.stdout: "squid": [ 2024-05-09T10:51:35.590 INFO:teuthology.orchestra.run.smithi098.stdout: "balancer", 2024-05-09T10:51:35.590 INFO:teuthology.orchestra.run.smithi098.stdout: "crash", 2024-05-09T10:51:35.590 INFO:teuthology.orchestra.run.smithi098.stdout: "devicehealth", 2024-05-09T10:51:35.590 INFO:teuthology.orchestra.run.smithi098.stdout: "orchestrator", 2024-05-09T10:51:35.591 INFO:teuthology.orchestra.run.smithi098.stdout: "pg_autoscaler", 2024-05-09T10:51:35.591 INFO:teuthology.orchestra.run.smithi098.stdout: "progress", 2024-05-09T10:51:35.591 INFO:teuthology.orchestra.run.smithi098.stdout: "rbd_support", 2024-05-09T10:51:35.591 INFO:teuthology.orchestra.run.smithi098.stdout: "status", 2024-05-09T10:51:35.591 INFO:teuthology.orchestra.run.smithi098.stdout: "telemetry", 2024-05-09T10:51:35.591 INFO:teuthology.orchestra.run.smithi098.stdout: "volumes" 2024-05-09T10:51:35.591 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-05-09T10:51:35.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.591 INFO:teuthology.orchestra.run.smithi098.stdout: "last_failure_osd_epoch": 0, 2024-05-09T10:51:35.591 INFO:teuthology.orchestra.run.smithi098.stdout: "active_clients": [ 2024-05-09T10:51:35.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.591 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "devicehealth", 2024-05-09T10:51:35.591 INFO:teuthology.orchestra.run.smithi098.stdout: "addrvec": [ 2024-05-09T10:51:35.591 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.591 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v2", 2024-05-09T10:51:35.591 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "172.21.15.98:0", 2024-05-09T10:51:35.591 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 3031147356 2024-05-09T10:51:35.592 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.592 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-05-09T10:51:35.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.592 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "libcephsqlite", 2024-05-09T10:51:35.592 INFO:teuthology.orchestra.run.smithi098.stdout: "addrvec": [ 2024-05-09T10:51:35.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.592 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v2", 2024-05-09T10:51:35.592 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "172.21.15.98:0", 2024-05-09T10:51:35.592 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 3733137378 2024-05-09T10:51:35.592 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.592 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-05-09T10:51:35.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.592 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.592 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rbd_support", 2024-05-09T10:51:35.592 INFO:teuthology.orchestra.run.smithi098.stdout: "addrvec": [ 2024-05-09T10:51:35.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.593 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v2", 2024-05-09T10:51:35.593 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "172.21.15.98:0", 2024-05-09T10:51:35.593 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 1177398271 2024-05-09T10:51:35.593 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.593 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-05-09T10:51:35.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.593 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "volumes", 2024-05-09T10:51:35.593 INFO:teuthology.orchestra.run.smithi098.stdout: "addrvec": [ 2024-05-09T10:51:35.593 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.593 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v2", 2024-05-09T10:51:35.593 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "172.21.15.98:0", 2024-05-09T10:51:35.593 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 3652209367 2024-05-09T10:51:35.593 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.593 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-05-09T10:51:35.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.594 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "volumes", 2024-05-09T10:51:35.594 INFO:teuthology.orchestra.run.smithi098.stdout: "addrvec": [ 2024-05-09T10:51:35.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.594 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v2", 2024-05-09T10:51:35.594 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "172.21.15.98:0", 2024-05-09T10:51:35.594 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 3894799170 2024-05-09T10:51:35.594 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.594 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-05-09T10:51:35.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.594 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "volumes", 2024-05-09T10:51:35.594 INFO:teuthology.orchestra.run.smithi098.stdout: "addrvec": [ 2024-05-09T10:51:35.594 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.594 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v2", 2024-05-09T10:51:35.595 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "172.21.15.98:0", 2024-05-09T10:51:35.595 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 714801283 2024-05-09T10:51:35.595 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.595 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-05-09T10:51:35.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.595 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "volumes", 2024-05-09T10:51:35.595 INFO:teuthology.orchestra.run.smithi098.stdout: "addrvec": [ 2024-05-09T10:51:35.595 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-05-09T10:51:35.595 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v2", 2024-05-09T10:51:35.595 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "172.21.15.98:0", 2024-05-09T10:51:35.595 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 3351163483 2024-05-09T10:51:35.595 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.595 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-05-09T10:51:35.595 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.595 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-05-09T10:51:35.596 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-05-09T10:51:35.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.841+0000 7f0916eee640 1 -- 172.21.15.98:0/2634875794 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f08fc06b680 msgr2=0x7f08fc06db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:35.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.841+0000 7f0916eee640 1 --2- 172.21.15.98:0/2634875794 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f08fc06b680 0x7f08fc06db40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f090c007f50 tx=0x7f090c0074a0 comp rx=0 tx=0).stop 2024-05-09T10:51:35.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.841+0000 7f0916eee640 1 -- 172.21.15.98:0/2634875794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09100ce2c0 msgr2=0x7f09100b0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:35.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.841+0000 7f0916eee640 1 --2- 172.21.15.98:0/2634875794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09100ce2c0 0x7f09100b0260 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f090802a140 tx=0x7f09080046c0 comp rx=0 tx=0).stop 2024-05-09T10:51:35.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.841+0000 7f0916eee640 1 -- 172.21.15.98:0/2634875794 shutdown_connections 2024-05-09T10:51:35.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.841+0000 7f0916eee640 1 --2- 172.21.15.98:0/2634875794 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f08fc06b680 0x7f08fc06db40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:35.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.841+0000 7f0916eee640 1 --2- 172.21.15.98:0/2634875794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09100ce2c0 0x7f09100b0260 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:35.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.841+0000 7f0916eee640 1 --2- 172.21.15.98:0/2634875794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0910072a20 0x7f09101e4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:35.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.841+0000 7f0916eee640 1 --2- 172.21.15.98:0/2634875794 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f091006bbf0 0x7f09100c3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:35.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.841+0000 7f0916eee640 1 -- 172.21.15.98:0/2634875794 >> 172.21.15.98:0/2634875794 conn(0x7f09100ccaa0 msgr2=0x7f09100af7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:35.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.841+0000 7f0916eee640 1 -- 172.21.15.98:0/2634875794 shutdown_connections 2024-05-09T10:51:35.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:34.841+0000 7f0916eee640 1 -- 172.21.15.98:0/2634875794 wait complete. 2024-05-09T10:51:35.602 DEBUG:tasks.check_counter:Getting stats from y 2024-05-09T10:51:35.602 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mgr.y.asok perf dump 2024-05-09T10:51:35.707 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-05-09T10:51:35.708 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T10:51:35.708 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 570, 2024-05-09T10:51:35.708 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 369, 2024-05-09T10:51:35.708 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 4225152, 2024-05-09T10:51:35.708 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 2596976, 2024-05-09T10:51:35.708 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 67, 2024-05-09T10:51:35.708 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 15, 2024-05-09T10:51:35.708 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.086230991, 2024-05-09T10:51:35.708 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.021221639, 2024-05-09T10:51:35.708 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.076763522, 2024-05-09T10:51:35.708 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000017079, 2024-05-09T10:51:35.708 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T10:51:35.708 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 348, 2024-05-09T10:51:35.709 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.020840660, 2024-05-09T10:51:35.709 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000059886 2024-05-09T10:51:35.709 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.709 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-05-09T10:51:35.709 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 62, 2024-05-09T10:51:35.709 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000007621, 2024-05-09T10:51:35.709 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000122 2024-05-09T10:51:35.709 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.709 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 4225152, 2024-05-09T10:51:35.709 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 2596976 2024-05-09T10:51:35.709 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.709 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T10:51:35.709 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 401, 2024-05-09T10:51:35.709 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 235, 2024-05-09T10:51:35.710 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 2262192, 2024-05-09T10:51:35.710 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 1432688, 2024-05-09T10:51:35.710 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 132, 2024-05-09T10:51:35.710 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 24, 2024-05-09T10:51:35.710 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.070173349, 2024-05-09T10:51:35.710 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.007903732, 2024-05-09T10:51:35.710 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.064581292, 2024-05-09T10:51:35.710 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000053250, 2024-05-09T10:51:35.710 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T10:51:35.710 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 216, 2024-05-09T10:51:35.710 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.020266869, 2024-05-09T10:51:35.710 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000093828 2024-05-09T10:51:35.710 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.710 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-05-09T10:51:35.710 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 282, 2024-05-09T10:51:35.711 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000028889, 2024-05-09T10:51:35.711 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000102 2024-05-09T10:51:35.711 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.711 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 2262192, 2024-05-09T10:51:35.711 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 1432688 2024-05-09T10:51:35.711 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.711 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T10:51:35.711 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 454, 2024-05-09T10:51:35.711 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 299, 2024-05-09T10:51:35.711 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 2901314, 2024-05-09T10:51:35.711 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 3492389, 2024-05-09T10:51:35.711 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 148, 2024-05-09T10:51:35.711 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 18, 2024-05-09T10:51:35.711 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.088917484, 2024-05-09T10:51:35.712 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.008318767, 2024-05-09T10:51:35.712 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.087605338, 2024-05-09T10:51:35.712 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.002755497, 2024-05-09T10:51:35.712 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T10:51:35.712 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 287, 2024-05-09T10:51:35.712 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.217222905, 2024-05-09T10:51:35.712 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000756874 2024-05-09T10:51:35.712 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.712 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-05-09T10:51:35.712 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 143, 2024-05-09T10:51:35.712 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000021591, 2024-05-09T10:51:35.712 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000150 2024-05-09T10:51:35.712 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.712 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 2900432, 2024-05-09T10:51:35.712 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 3491880 2024-05-09T10:51:35.712 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.713 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-05-09T10:51:35.713 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 0, 2024-05-09T10:51:35.713 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-05-09T10:51:35.713 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.713 INFO:teuthology.orchestra.run.smithi098.stdout: "client": { 2024-05-09T10:51:35.713 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": { 2024-05-09T10:51:35.713 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.713 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.713 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.713 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.713 INFO:teuthology.orchestra.run.smithi098.stdout: "lat": { 2024-05-09T10:51:35.713 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.713 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.713 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.713 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.713 INFO:teuthology.orchestra.run.smithi098.stdout: "wrlat": { 2024-05-09T10:51:35.713 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.714 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.714 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.714 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.714 INFO:teuthology.orchestra.run.smithi098.stdout: "rdlat": { 2024-05-09T10:51:35.714 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.714 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.714 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.714 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.714 INFO:teuthology.orchestra.run.smithi098.stdout: "fsync": { 2024-05-09T10:51:35.714 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.714 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.714 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.714 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.714 INFO:teuthology.orchestra.run.smithi098.stdout: "mdavg": 0.000000000, 2024-05-09T10:51:35.714 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsqsum": 0, 2024-05-09T10:51:35.714 INFO:teuthology.orchestra.run.smithi098.stdout: "mdops": 0, 2024-05-09T10:51:35.715 INFO:teuthology.orchestra.run.smithi098.stdout: "readavg": 0.000000000, 2024-05-09T10:51:35.715 INFO:teuthology.orchestra.run.smithi098.stdout: "readsqsum": 0, 2024-05-09T10:51:35.715 INFO:teuthology.orchestra.run.smithi098.stdout: "rdops": 0, 2024-05-09T10:51:35.715 INFO:teuthology.orchestra.run.smithi098.stdout: "writeavg": 0.000000000, 2024-05-09T10:51:35.715 INFO:teuthology.orchestra.run.smithi098.stdout: "writesqsum": 0, 2024-05-09T10:51:35.715 INFO:teuthology.orchestra.run.smithi098.stdout: "rdops": 0 2024-05-09T10:51:35.715 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.715 INFO:teuthology.orchestra.run.smithi098.stdout: "client-0x55fbaf7c65a0": { 2024-05-09T10:51:35.715 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": { 2024-05-09T10:51:35.715 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 20, 2024-05-09T10:51:35.715 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.324123323, 2024-05-09T10:51:35.715 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.016206166 2024-05-09T10:51:35.715 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.715 INFO:teuthology.orchestra.run.smithi098.stdout: "lat": { 2024-05-09T10:51:35.715 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 20, 2024-05-09T10:51:35.716 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.324123323, 2024-05-09T10:51:35.716 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.016206166 2024-05-09T10:51:35.716 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.716 INFO:teuthology.orchestra.run.smithi098.stdout: "wrlat": { 2024-05-09T10:51:35.716 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.716 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.716 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.716 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.716 INFO:teuthology.orchestra.run.smithi098.stdout: "rdlat": { 2024-05-09T10:51:35.716 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.716 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.716 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.716 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.716 INFO:teuthology.orchestra.run.smithi098.stdout: "fsync": { 2024-05-09T10:51:35.717 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.717 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.717 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.717 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.717 INFO:teuthology.orchestra.run.smithi098.stdout: "mdavg": 0.016206162, 2024-05-09T10:51:35.717 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsqsum": 91242217817581024, 2024-05-09T10:51:35.717 INFO:teuthology.orchestra.run.smithi098.stdout: "mdops": 20, 2024-05-09T10:51:35.717 INFO:teuthology.orchestra.run.smithi098.stdout: "readavg": 0.000000000, 2024-05-09T10:51:35.717 INFO:teuthology.orchestra.run.smithi098.stdout: "readsqsum": 0, 2024-05-09T10:51:35.717 INFO:teuthology.orchestra.run.smithi098.stdout: "rdops": 0, 2024-05-09T10:51:35.717 INFO:teuthology.orchestra.run.smithi098.stdout: "writeavg": 0.000000000, 2024-05-09T10:51:35.717 INFO:teuthology.orchestra.run.smithi098.stdout: "writesqsum": 0, 2024-05-09T10:51:35.717 INFO:teuthology.orchestra.run.smithi098.stdout: "rdops": 0 2024-05-09T10:51:35.717 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.717 INFO:teuthology.orchestra.run.smithi098.stdout: "client-0x55fbaf7c6a00": { 2024-05-09T10:51:35.717 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": { 2024-05-09T10:51:35.718 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 70, 2024-05-09T10:51:35.718 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.086094205, 2024-05-09T10:51:35.718 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001229917 2024-05-09T10:51:35.718 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.718 INFO:teuthology.orchestra.run.smithi098.stdout: "lat": { 2024-05-09T10:51:35.718 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 70, 2024-05-09T10:51:35.718 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.086094205, 2024-05-09T10:51:35.718 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001229917 2024-05-09T10:51:35.718 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.718 INFO:teuthology.orchestra.run.smithi098.stdout: "wrlat": { 2024-05-09T10:51:35.718 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 18, 2024-05-09T10:51:35.718 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000329051, 2024-05-09T10:51:35.718 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000018280 2024-05-09T10:51:35.718 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.719 INFO:teuthology.orchestra.run.smithi098.stdout: "rdlat": { 2024-05-09T10:51:35.719 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 18, 2024-05-09T10:51:35.719 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000246706, 2024-05-09T10:51:35.719 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000013705 2024-05-09T10:51:35.719 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.719 INFO:teuthology.orchestra.run.smithi098.stdout: "fsync": { 2024-05-09T10:51:35.719 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3, 2024-05-09T10:51:35.719 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.018306393, 2024-05-09T10:51:35.719 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.006102131 2024-05-09T10:51:35.719 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.719 INFO:teuthology.orchestra.run.smithi098.stdout: "mdavg": 0.001229902, 2024-05-09T10:51:35.719 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsqsum": 195963976310504, 2024-05-09T10:51:35.719 INFO:teuthology.orchestra.run.smithi098.stdout: "mdops": 70, 2024-05-09T10:51:35.719 INFO:teuthology.orchestra.run.smithi098.stdout: "readavg": 0.000013702, 2024-05-09T10:51:35.720 INFO:teuthology.orchestra.run.smithi098.stdout: "readsqsum": 343119669, 2024-05-09T10:51:35.720 INFO:teuthology.orchestra.run.smithi098.stdout: "rdops": 18, 2024-05-09T10:51:35.720 INFO:teuthology.orchestra.run.smithi098.stdout: "writeavg": 0.000018276, 2024-05-09T10:51:35.720 INFO:teuthology.orchestra.run.smithi098.stdout: "writesqsum": 1212289753, 2024-05-09T10:51:35.720 INFO:teuthology.orchestra.run.smithi098.stdout: "rdops": 18 2024-05-09T10:51:35.720 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.720 INFO:teuthology.orchestra.run.smithi098.stdout: "client-0x55fbaf7c7180": { 2024-05-09T10:51:35.720 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": { 2024-05-09T10:51:35.720 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 56, 2024-05-09T10:51:35.720 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.064800236, 2024-05-09T10:51:35.720 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001157147 2024-05-09T10:51:35.720 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.720 INFO:teuthology.orchestra.run.smithi098.stdout: "lat": { 2024-05-09T10:51:35.720 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 56, 2024-05-09T10:51:35.720 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.064800236, 2024-05-09T10:51:35.721 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001157147 2024-05-09T10:51:35.721 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.721 INFO:teuthology.orchestra.run.smithi098.stdout: "wrlat": { 2024-05-09T10:51:35.721 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.721 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.721 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.721 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.721 INFO:teuthology.orchestra.run.smithi098.stdout: "rdlat": { 2024-05-09T10:51:35.721 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.721 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.721 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.721 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.721 INFO:teuthology.orchestra.run.smithi098.stdout: "fsync": { 2024-05-09T10:51:35.721 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.721 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.721 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.721 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.722 INFO:teuthology.orchestra.run.smithi098.stdout: "mdavg": 0.001157134, 2024-05-09T10:51:35.722 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsqsum": 115328274786701, 2024-05-09T10:51:35.722 INFO:teuthology.orchestra.run.smithi098.stdout: "mdops": 56, 2024-05-09T10:51:35.722 INFO:teuthology.orchestra.run.smithi098.stdout: "readavg": 0.000000000, 2024-05-09T10:51:35.722 INFO:teuthology.orchestra.run.smithi098.stdout: "readsqsum": 0, 2024-05-09T10:51:35.722 INFO:teuthology.orchestra.run.smithi098.stdout: "rdops": 0, 2024-05-09T10:51:35.722 INFO:teuthology.orchestra.run.smithi098.stdout: "writeavg": 0.000000000, 2024-05-09T10:51:35.722 INFO:teuthology.orchestra.run.smithi098.stdout: "writesqsum": 0, 2024-05-09T10:51:35.722 INFO:teuthology.orchestra.run.smithi098.stdout: "rdops": 0 2024-05-09T10:51:35.722 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.722 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-Mgr": { 2024-05-09T10:51:35.722 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-05-09T10:51:35.722 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-05-09T10:51:35.722 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-05-09T10:51:35.722 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.031523945, 2024-05-09T10:51:35.723 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.015761972 2024-05-09T10:51:35.723 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.723 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.723 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MgrStandby": { 2024-05-09T10:51:35.723 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-05-09T10:51:35.723 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-05-09T10:51:35.723 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.723 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.723 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.723 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.723 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.723 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-balancer": { 2024-05-09T10:51:35.723 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-05-09T10:51:35.723 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-05-09T10:51:35.723 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.723 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.724 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.724 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.724 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.724 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-cmd_finisher": { 2024-05-09T10:51:35.724 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-05-09T10:51:35.724 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-05-09T10:51:35.724 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 17, 2024-05-09T10:51:35.724 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000661482, 2024-05-09T10:51:35.724 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000038910 2024-05-09T10:51:35.724 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.724 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.724 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-crash": { 2024-05-09T10:51:35.724 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-05-09T10:51:35.724 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-05-09T10:51:35.725 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.725 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.725 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.725 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.725 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.725 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-devicehealth": { 2024-05-09T10:51:35.725 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-05-09T10:51:35.725 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-05-09T10:51:35.725 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.725 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.725 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.725 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.725 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.725 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-iostat": { 2024-05-09T10:51:35.725 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-05-09T10:51:35.726 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-05-09T10:51:35.726 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.726 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.726 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.726 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.726 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.726 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-nfs": { 2024-05-09T10:51:35.726 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-05-09T10:51:35.726 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-05-09T10:51:35.726 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.726 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.726 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.726 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.726 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.727 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-orchestrator": { 2024-05-09T10:51:35.727 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-05-09T10:51:35.727 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-05-09T10:51:35.727 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.727 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.727 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.727 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.727 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.727 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-pg_autoscaler": { 2024-05-09T10:51:35.727 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-05-09T10:51:35.727 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-05-09T10:51:35.727 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.727 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.727 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.727 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.727 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.727 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-progress": { 2024-05-09T10:51:35.728 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-05-09T10:51:35.728 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-05-09T10:51:35.728 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.728 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.728 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.728 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.728 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.728 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-rbd_support": { 2024-05-09T10:51:35.728 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-05-09T10:51:35.728 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-05-09T10:51:35.728 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.728 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.728 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.728 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.728 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.729 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-restful": { 2024-05-09T10:51:35.729 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-05-09T10:51:35.729 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-05-09T10:51:35.729 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 17, 2024-05-09T10:51:35.729 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.002401535, 2024-05-09T10:51:35.729 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000141266 2024-05-09T10:51:35.729 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.729 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.729 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-status": { 2024-05-09T10:51:35.729 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-05-09T10:51:35.729 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-05-09T10:51:35.729 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.729 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.729 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.729 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.730 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.730 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-telemetry": { 2024-05-09T10:51:35.730 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-05-09T10:51:35.730 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-05-09T10:51:35.730 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.730 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.730 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.730 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.730 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.730 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-volumes": { 2024-05-09T10:51:35.730 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-05-09T10:51:35.730 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-05-09T10:51:35.730 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 9, 2024-05-09T10:51:35.730 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.140289992, 2024-05-09T10:51:35.731 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.015587776 2024-05-09T10:51:35.731 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.731 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.731 INFO:teuthology.orchestra.run.smithi098.stdout: "libcephsqlite_striper": { 2024-05-09T10:51:35.731 INFO:teuthology.orchestra.run.smithi098.stdout: "update_metadata": 8, 2024-05-09T10:51:35.731 INFO:teuthology.orchestra.run.smithi098.stdout: "update_allocated": 2, 2024-05-09T10:51:35.731 INFO:teuthology.orchestra.run.smithi098.stdout: "update_size": 6, 2024-05-09T10:51:35.731 INFO:teuthology.orchestra.run.smithi098.stdout: "update_version": 8, 2024-05-09T10:51:35.731 INFO:teuthology.orchestra.run.smithi098.stdout: "shrink": 0, 2024-05-09T10:51:35.731 INFO:teuthology.orchestra.run.smithi098.stdout: "shrink_bytes": 0, 2024-05-09T10:51:35.731 INFO:teuthology.orchestra.run.smithi098.stdout: "lock": 11, 2024-05-09T10:51:35.731 INFO:teuthology.orchestra.run.smithi098.stdout: "unlock": 11 2024-05-09T10:51:35.731 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.731 INFO:teuthology.orchestra.run.smithi098.stdout: "libcephsqlite_vfs": { 2024-05-09T10:51:35.731 INFO:teuthology.orchestra.run.smithi098.stdout: "op_open": { 2024-05-09T10:51:35.731 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-05-09T10:51:35.731 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.009999889, 2024-05-09T10:51:35.732 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.004999944 2024-05-09T10:51:35.732 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.732 INFO:teuthology.orchestra.run.smithi098.stdout: "op_delete": { 2024-05-09T10:51:35.732 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.732 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.732 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.732 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.732 INFO:teuthology.orchestra.run.smithi098.stdout: "op_access": { 2024-05-09T10:51:35.732 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.732 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.732 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.732 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.732 INFO:teuthology.orchestra.run.smithi098.stdout: "op_fullpathname": { 2024-05-09T10:51:35.732 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1, 2024-05-09T10:51:35.732 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000999989, 2024-05-09T10:51:35.733 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000999989 2024-05-09T10:51:35.733 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.733 INFO:teuthology.orchestra.run.smithi098.stdout: "op_currenttime": { 2024-05-09T10:51:35.733 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 4, 2024-05-09T10:51:35.733 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.733 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.733 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.733 INFO:teuthology.orchestra.run.smithi098.stdout: "opf_close": { 2024-05-09T10:51:35.733 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.733 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.733 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.733 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.733 INFO:teuthology.orchestra.run.smithi098.stdout: "opf_read": { 2024-05-09T10:51:35.733 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 20, 2024-05-09T10:51:35.734 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.007999911, 2024-05-09T10:51:35.734 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000399995 2024-05-09T10:51:35.734 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.734 INFO:teuthology.orchestra.run.smithi098.stdout: "opf_write": { 2024-05-09T10:51:35.734 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 85, 2024-05-09T10:51:35.734 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.004999944, 2024-05-09T10:51:35.734 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000058822 2024-05-09T10:51:35.734 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.734 INFO:teuthology.orchestra.run.smithi098.stdout: "opf_truncate": { 2024-05-09T10:51:35.734 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.734 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.734 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.734 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.734 INFO:teuthology.orchestra.run.smithi098.stdout: "opf_sync": { 2024-05-09T10:51:35.735 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 24, 2024-05-09T10:51:35.735 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.108998792, 2024-05-09T10:51:35.735 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.004541616 2024-05-09T10:51:35.735 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.735 INFO:teuthology.orchestra.run.smithi098.stdout: "opf_filesize": { 2024-05-09T10:51:35.735 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 20, 2024-05-09T10:51:35.735 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.735 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.735 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.735 INFO:teuthology.orchestra.run.smithi098.stdout: "opf_lock": { 2024-05-09T10:51:35.735 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 27, 2024-05-09T10:51:35.735 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.024999722, 2024-05-09T10:51:35.735 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000925915 2024-05-09T10:51:35.735 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.735 INFO:teuthology.orchestra.run.smithi098.stdout: "opf_unlock": { 2024-05-09T10:51:35.735 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 19, 2024-05-09T10:51:35.735 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.015999823, 2024-05-09T10:51:35.736 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000842095 2024-05-09T10:51:35.736 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.736 INFO:teuthology.orchestra.run.smithi098.stdout: "opf_checkreservedlock": { 2024-05-09T10:51:35.736 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 9, 2024-05-09T10:51:35.736 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.736 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.736 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.736 INFO:teuthology.orchestra.run.smithi098.stdout: "opf_filecontrol": { 2024-05-09T10:51:35.736 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 26, 2024-05-09T10:51:35.736 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.736 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.736 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.736 INFO:teuthology.orchestra.run.smithi098.stdout: "opf_sectorsize": { 2024-05-09T10:51:35.736 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.736 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.736 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.736 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.737 INFO:teuthology.orchestra.run.smithi098.stdout: "opf_devicecharacteristics": { 2024-05-09T10:51:35.737 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 30, 2024-05-09T10:51:35.737 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.737 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.737 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.737 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.737 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-05-09T10:51:35.737 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-05-09T10:51:35.737 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-05-09T10:51:35.737 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T10:51:35.737 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-05-09T10:51:35.737 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T10:51:35.737 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-05-09T10:51:35.737 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T10:51:35.737 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T10:51:35.737 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T10:51:35.737 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T10:51:35.738 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T10:51:35.738 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-05-09T10:51:35.738 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T10:51:35.738 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T10:51:35.738 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-05-09T10:51:35.738 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-05-09T10:51:35.738 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-05-09T10:51:35.738 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-05-09T10:51:35.738 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T10:51:35.738 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T10:51:35.738 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T10:51:35.738 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T10:51:35.738 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T10:51:35.738 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-05-09T10:51:35.739 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-05-09T10:51:35.739 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-05-09T10:51:35.739 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T10:51:35.739 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T10:51:35.739 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-05-09T10:51:35.739 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-05-09T10:51:35.739 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-05-09T10:51:35.739 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-05-09T10:51:35.739 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T10:51:35.739 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-05-09T10:51:35.739 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T10:51:35.739 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-05-09T10:51:35.739 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 1646274, 2024-05-09T10:51:35.739 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 340, 2024-05-09T10:51:35.740 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 1496, 2024-05-09T10:51:35.740 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 17, 2024-05-09T10:51:35.740 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-05-09T10:51:35.740 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-05-09T10:51:35.740 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-05-09T10:51:35.740 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-05-09T10:51:35.740 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-05-09T10:51:35.740 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-05-09T10:51:35.740 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 47096, 2024-05-09T10:51:35.740 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 828, 2024-05-09T10:51:35.740 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T10:51:35.740 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-05-09T10:51:35.740 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 12880, 2024-05-09T10:51:35.740 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 308, 2024-05-09T10:51:35.740 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 0, 2024-05-09T10:51:35.740 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 0, 2024-05-09T10:51:35.740 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-05-09T10:51:35.741 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-05-09T10:51:35.741 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-05-09T10:51:35.741 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-05-09T10:51:35.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.741 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": { 2024-05-09T10:51:35.741 INFO:teuthology.orchestra.run.smithi098.stdout: "cache_hit": 0, 2024-05-09T10:51:35.741 INFO:teuthology.orchestra.run.smithi098.stdout: "cache_miss": 0 2024-05-09T10:51:35.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.741 INFO:teuthology.orchestra.run.smithi098.stdout: "objectcacher-libcephfs": { 2024-05-09T10:51:35.741 INFO:teuthology.orchestra.run.smithi098.stdout: "cache_ops_hit": 0, 2024-05-09T10:51:35.741 INFO:teuthology.orchestra.run.smithi098.stdout: "cache_ops_miss": 0, 2024-05-09T10:51:35.741 INFO:teuthology.orchestra.run.smithi098.stdout: "cache_bytes_hit": 0, 2024-05-09T10:51:35.741 INFO:teuthology.orchestra.run.smithi098.stdout: "cache_bytes_miss": 0, 2024-05-09T10:51:35.741 INFO:teuthology.orchestra.run.smithi098.stdout: "data_read": 0, 2024-05-09T10:51:35.741 INFO:teuthology.orchestra.run.smithi098.stdout: "data_written": 0, 2024-05-09T10:51:35.741 INFO:teuthology.orchestra.run.smithi098.stdout: "data_flushed": 0, 2024-05-09T10:51:35.742 INFO:teuthology.orchestra.run.smithi098.stdout: "data_overwritten_while_flushing": 0, 2024-05-09T10:51:35.742 INFO:teuthology.orchestra.run.smithi098.stdout: "write_ops_blocked": 0, 2024-05-09T10:51:35.742 INFO:teuthology.orchestra.run.smithi098.stdout: "write_bytes_blocked": 0, 2024-05-09T10:51:35.742 INFO:teuthology.orchestra.run.smithi098.stdout: "write_time_blocked": 0.000000000 2024-05-09T10:51:35.742 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.742 INFO:teuthology.orchestra.run.smithi098.stdout: "objectcacher-libcephfs-0x55fbaf7c61e0": { 2024-05-09T10:51:35.742 INFO:teuthology.orchestra.run.smithi098.stdout: "cache_ops_hit": 0, 2024-05-09T10:51:35.742 INFO:teuthology.orchestra.run.smithi098.stdout: "cache_ops_miss": 0, 2024-05-09T10:51:35.742 INFO:teuthology.orchestra.run.smithi098.stdout: "cache_bytes_hit": 0, 2024-05-09T10:51:35.742 INFO:teuthology.orchestra.run.smithi098.stdout: "cache_bytes_miss": 0, 2024-05-09T10:51:35.742 INFO:teuthology.orchestra.run.smithi098.stdout: "data_read": 0, 2024-05-09T10:51:35.742 INFO:teuthology.orchestra.run.smithi098.stdout: "data_written": 0, 2024-05-09T10:51:35.742 INFO:teuthology.orchestra.run.smithi098.stdout: "data_flushed": 0, 2024-05-09T10:51:35.742 INFO:teuthology.orchestra.run.smithi098.stdout: "data_overwritten_while_flushing": 0, 2024-05-09T10:51:35.742 INFO:teuthology.orchestra.run.smithi098.stdout: "write_ops_blocked": 0, 2024-05-09T10:51:35.743 INFO:teuthology.orchestra.run.smithi098.stdout: "write_bytes_blocked": 0, 2024-05-09T10:51:35.743 INFO:teuthology.orchestra.run.smithi098.stdout: "write_time_blocked": 0.000000000 2024-05-09T10:51:35.743 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.743 INFO:teuthology.orchestra.run.smithi098.stdout: "objectcacher-libcephfs-0x55fbaf7c6320": { 2024-05-09T10:51:35.743 INFO:teuthology.orchestra.run.smithi098.stdout: "cache_ops_hit": 0, 2024-05-09T10:51:35.743 INFO:teuthology.orchestra.run.smithi098.stdout: "cache_ops_miss": 0, 2024-05-09T10:51:35.743 INFO:teuthology.orchestra.run.smithi098.stdout: "cache_bytes_hit": 0, 2024-05-09T10:51:35.743 INFO:teuthology.orchestra.run.smithi098.stdout: "cache_bytes_miss": 0, 2024-05-09T10:51:35.743 INFO:teuthology.orchestra.run.smithi098.stdout: "data_read": 0, 2024-05-09T10:51:35.743 INFO:teuthology.orchestra.run.smithi098.stdout: "data_written": 0, 2024-05-09T10:51:35.743 INFO:teuthology.orchestra.run.smithi098.stdout: "data_flushed": 0, 2024-05-09T10:51:35.743 INFO:teuthology.orchestra.run.smithi098.stdout: "data_overwritten_while_flushing": 0, 2024-05-09T10:51:35.743 INFO:teuthology.orchestra.run.smithi098.stdout: "write_ops_blocked": 0, 2024-05-09T10:51:35.743 INFO:teuthology.orchestra.run.smithi098.stdout: "write_bytes_blocked": 0, 2024-05-09T10:51:35.743 INFO:teuthology.orchestra.run.smithi098.stdout: "write_time_blocked": 0.000000000 2024-05-09T10:51:35.743 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.743 INFO:teuthology.orchestra.run.smithi098.stdout: "objectcacher-libcephfs-0x55fbaf7c68c0": { 2024-05-09T10:51:35.744 INFO:teuthology.orchestra.run.smithi098.stdout: "cache_ops_hit": 9, 2024-05-09T10:51:35.744 INFO:teuthology.orchestra.run.smithi098.stdout: "cache_ops_miss": 0, 2024-05-09T10:51:35.744 INFO:teuthology.orchestra.run.smithi098.stdout: "cache_bytes_hit": 1233, 2024-05-09T10:51:35.744 INFO:teuthology.orchestra.run.smithi098.stdout: "cache_bytes_miss": 0, 2024-05-09T10:51:35.744 INFO:teuthology.orchestra.run.smithi098.stdout: "data_read": 1233, 2024-05-09T10:51:35.744 INFO:teuthology.orchestra.run.smithi098.stdout: "data_written": 411, 2024-05-09T10:51:35.744 INFO:teuthology.orchestra.run.smithi098.stdout: "data_flushed": 411, 2024-05-09T10:51:35.744 INFO:teuthology.orchestra.run.smithi098.stdout: "data_overwritten_while_flushing": 0, 2024-05-09T10:51:35.744 INFO:teuthology.orchestra.run.smithi098.stdout: "write_ops_blocked": 0, 2024-05-09T10:51:35.744 INFO:teuthology.orchestra.run.smithi098.stdout: "write_bytes_blocked": 0, 2024-05-09T10:51:35.744 INFO:teuthology.orchestra.run.smithi098.stdout: "write_time_blocked": 0.000000000 2024-05-09T10:51:35.744 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.744 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2024-05-09T10:51:35.744 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-05-09T10:51:35.744 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-05-09T10:51:35.744 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 0, 2024-05-09T10:51:35.745 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 0, 2024-05-09T10:51:35.745 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 0, 2024-05-09T10:51:35.745 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 0, 2024-05-09T10:51:35.745 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-05-09T10:51:35.745 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.745 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.745 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.745 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.745 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-05-09T10:51:35.745 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-05-09T10:51:35.745 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.745 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0 2024-05-09T10:51:35.745 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.745 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 0, 2024-05-09T10:51:35.745 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 0, 2024-05-09T10:51:35.746 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 0, 2024-05-09T10:51:35.746 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-05-09T10:51:35.746 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-05-09T10:51:35.746 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 0, 2024-05-09T10:51:35.746 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 0, 2024-05-09T10:51:35.746 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 0, 2024-05-09T10:51:35.746 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 0, 2024-05-09T10:51:35.746 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 0, 2024-05-09T10:51:35.746 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-05-09T10:51:35.746 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-05-09T10:51:35.746 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-05-09T10:51:35.746 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-05-09T10:51:35.746 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 0, 2024-05-09T10:51:35.746 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-05-09T10:51:35.747 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-05-09T10:51:35.747 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-05-09T10:51:35.747 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 0, 2024-05-09T10:51:35.747 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 0, 2024-05-09T10:51:35.747 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-05-09T10:51:35.747 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-05-09T10:51:35.747 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-05-09T10:51:35.747 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-05-09T10:51:35.747 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-05-09T10:51:35.747 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-05-09T10:51:35.747 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T10:51:35.747 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-05-09T10:51:35.747 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-05-09T10:51:35.747 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-05-09T10:51:35.747 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-05-09T10:51:35.747 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-05-09T10:51:35.748 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-05-09T10:51:35.748 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-05-09T10:51:35.748 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-05-09T10:51:35.748 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-05-09T10:51:35.748 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-05-09T10:51:35.748 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-05-09T10:51:35.748 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-05-09T10:51:35.748 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-05-09T10:51:35.748 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-05-09T10:51:35.748 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-05-09T10:51:35.748 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-05-09T10:51:35.748 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-05-09T10:51:35.748 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 8, 2024-05-09T10:51:35.748 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-05-09T10:51:35.749 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 34, 2024-05-09T10:51:35.749 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-05-09T10:51:35.749 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 33, 2024-05-09T10:51:35.749 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 8, 2024-05-09T10:51:35.749 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 8, 2024-05-09T10:51:35.749 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2024-05-09T10:51:35.749 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-05-09T10:51:35.749 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 0, 2024-05-09T10:51:35.749 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 0, 2024-05-09T10:51:35.749 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 0 2024-05-09T10:51:35.749 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.749 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter-0x55fbaecf2d20": { 2024-05-09T10:51:35.749 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-05-09T10:51:35.749 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-05-09T10:51:35.750 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 163, 2024-05-09T10:51:35.750 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 2434716, 2024-05-09T10:51:35.750 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 0, 2024-05-09T10:51:35.750 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 163, 2024-05-09T10:51:35.750 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-05-09T10:51:35.750 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 163, 2024-05-09T10:51:35.750 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.507994371, 2024-05-09T10:51:35.750 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.003116529 2024-05-09T10:51:35.750 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.750 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-05-09T10:51:35.750 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-05-09T10:51:35.750 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 163, 2024-05-09T10:51:35.750 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 307 2024-05-09T10:51:35.750 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.751 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 163, 2024-05-09T10:51:35.751 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 46, 2024-05-09T10:51:35.751 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 117, 2024-05-09T10:51:35.751 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-05-09T10:51:35.751 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-05-09T10:51:35.751 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 0, 2024-05-09T10:51:35.751 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 2, 2024-05-09T10:51:35.751 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 20, 2024-05-09T10:51:35.751 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 85, 2024-05-09T10:51:35.751 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 0, 2024-05-09T10:51:35.751 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-05-09T10:51:35.751 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-05-09T10:51:35.751 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-05-09T10:51:35.751 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-05-09T10:51:35.751 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 0, 2024-05-09T10:51:35.751 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-05-09T10:51:35.751 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-05-09T10:51:35.752 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-05-09T10:51:35.752 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 104, 2024-05-09T10:51:35.752 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 52, 2024-05-09T10:51:35.752 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 22, 2024-05-09T10:51:35.752 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-05-09T10:51:35.752 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-05-09T10:51:35.752 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 22, 2024-05-09T10:51:35.752 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-05-09T10:51:35.752 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-05-09T10:51:35.752 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T10:51:35.752 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-05-09T10:51:35.752 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-05-09T10:51:35.752 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-05-09T10:51:35.752 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-05-09T10:51:35.752 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-05-09T10:51:35.752 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-05-09T10:51:35.753 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-05-09T10:51:35.753 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-05-09T10:51:35.753 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-05-09T10:51:35.753 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-05-09T10:51:35.753 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-05-09T10:51:35.753 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-05-09T10:51:35.753 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-05-09T10:51:35.753 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-05-09T10:51:35.753 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-05-09T10:51:35.753 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-05-09T10:51:35.753 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-05-09T10:51:35.753 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-05-09T10:51:35.753 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-05-09T10:51:35.753 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 16, 2024-05-09T10:51:35.753 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-05-09T10:51:35.754 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 6, 2024-05-09T10:51:35.754 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 1, 2024-05-09T10:51:35.754 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 1, 2024-05-09T10:51:35.754 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2024-05-09T10:51:35.754 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-05-09T10:51:35.754 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 0, 2024-05-09T10:51:35.754 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 0, 2024-05-09T10:51:35.754 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 0 2024-05-09T10:51:35.754 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.754 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter-0x55fbaecf3040": { 2024-05-09T10:51:35.754 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-05-09T10:51:35.754 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-05-09T10:51:35.754 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 0, 2024-05-09T10:51:35.754 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 0, 2024-05-09T10:51:35.754 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 0, 2024-05-09T10:51:35.755 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 0, 2024-05-09T10:51:35.755 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-05-09T10:51:35.755 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.755 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.755 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.755 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.755 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-05-09T10:51:35.755 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-05-09T10:51:35.755 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.755 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0 2024-05-09T10:51:35.755 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.755 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 0, 2024-05-09T10:51:35.755 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 0, 2024-05-09T10:51:35.755 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 0, 2024-05-09T10:51:35.755 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-05-09T10:51:35.755 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-05-09T10:51:35.755 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 0, 2024-05-09T10:51:35.756 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 0, 2024-05-09T10:51:35.756 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 0, 2024-05-09T10:51:35.756 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 0, 2024-05-09T10:51:35.756 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 0, 2024-05-09T10:51:35.756 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-05-09T10:51:35.756 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-05-09T10:51:35.756 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-05-09T10:51:35.756 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-05-09T10:51:35.756 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 0, 2024-05-09T10:51:35.756 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-05-09T10:51:35.756 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-05-09T10:51:35.756 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-05-09T10:51:35.756 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 0, 2024-05-09T10:51:35.756 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 0, 2024-05-09T10:51:35.756 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-05-09T10:51:35.756 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-05-09T10:51:35.756 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-05-09T10:51:35.757 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-05-09T10:51:35.757 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-05-09T10:51:35.757 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-05-09T10:51:35.757 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T10:51:35.757 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-05-09T10:51:35.757 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-05-09T10:51:35.757 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-05-09T10:51:35.757 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-05-09T10:51:35.757 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-05-09T10:51:35.757 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-05-09T10:51:35.757 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-05-09T10:51:35.757 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-05-09T10:51:35.757 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-05-09T10:51:35.757 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-05-09T10:51:35.757 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-05-09T10:51:35.758 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-05-09T10:51:35.758 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-05-09T10:51:35.758 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-05-09T10:51:35.758 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-05-09T10:51:35.758 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-05-09T10:51:35.758 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-05-09T10:51:35.758 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-05-09T10:51:35.758 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-05-09T10:51:35.758 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 0, 2024-05-09T10:51:35.758 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-05-09T10:51:35.758 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 0, 2024-05-09T10:51:35.758 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 0, 2024-05-09T10:51:35.758 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 0, 2024-05-09T10:51:35.758 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2024-05-09T10:51:35.759 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-05-09T10:51:35.759 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 0, 2024-05-09T10:51:35.759 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 0, 2024-05-09T10:51:35.759 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 0 2024-05-09T10:51:35.759 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.759 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter-0x55fbaecf32c0": { 2024-05-09T10:51:35.759 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-05-09T10:51:35.759 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-05-09T10:51:35.759 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 0, 2024-05-09T10:51:35.759 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 0, 2024-05-09T10:51:35.759 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 0, 2024-05-09T10:51:35.759 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 0, 2024-05-09T10:51:35.759 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-05-09T10:51:35.759 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.759 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.760 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.760 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.760 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-05-09T10:51:35.760 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-05-09T10:51:35.760 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.760 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0 2024-05-09T10:51:35.760 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.760 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 0, 2024-05-09T10:51:35.760 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 0, 2024-05-09T10:51:35.760 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 0, 2024-05-09T10:51:35.760 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-05-09T10:51:35.760 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-05-09T10:51:35.760 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 0, 2024-05-09T10:51:35.760 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 0, 2024-05-09T10:51:35.760 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 0, 2024-05-09T10:51:35.760 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 0, 2024-05-09T10:51:35.761 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 0, 2024-05-09T10:51:35.761 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-05-09T10:51:35.761 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-05-09T10:51:35.761 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-05-09T10:51:35.761 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-05-09T10:51:35.761 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 0, 2024-05-09T10:51:35.761 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-05-09T10:51:35.761 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-05-09T10:51:35.761 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-05-09T10:51:35.761 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 0, 2024-05-09T10:51:35.761 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 0, 2024-05-09T10:51:35.761 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-05-09T10:51:35.761 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-05-09T10:51:35.761 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-05-09T10:51:35.761 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-05-09T10:51:35.761 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-05-09T10:51:35.761 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-05-09T10:51:35.762 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T10:51:35.762 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-05-09T10:51:35.762 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-05-09T10:51:35.762 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-05-09T10:51:35.762 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-05-09T10:51:35.762 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-05-09T10:51:35.762 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-05-09T10:51:35.762 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-05-09T10:51:35.762 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-05-09T10:51:35.762 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-05-09T10:51:35.762 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-05-09T10:51:35.762 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-05-09T10:51:35.762 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-05-09T10:51:35.762 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-05-09T10:51:35.762 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-05-09T10:51:35.763 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-05-09T10:51:35.763 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-05-09T10:51:35.763 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-05-09T10:51:35.763 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-05-09T10:51:35.763 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-05-09T10:51:35.763 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 0, 2024-05-09T10:51:35.763 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-05-09T10:51:35.763 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 0, 2024-05-09T10:51:35.763 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 0, 2024-05-09T10:51:35.763 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 0, 2024-05-09T10:51:35.763 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2024-05-09T10:51:35.763 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-05-09T10:51:35.763 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 0, 2024-05-09T10:51:35.763 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 0, 2024-05-09T10:51:35.763 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 0 2024-05-09T10:51:35.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.763 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter-0x55fbaf410280": { 2024-05-09T10:51:35.764 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-05-09T10:51:35.764 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-05-09T10:51:35.764 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 1, 2024-05-09T10:51:35.764 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 39, 2024-05-09T10:51:35.764 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 0, 2024-05-09T10:51:35.764 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 1, 2024-05-09T10:51:35.764 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-05-09T10:51:35.764 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1, 2024-05-09T10:51:35.764 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.001999978, 2024-05-09T10:51:35.764 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001999978 2024-05-09T10:51:35.764 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.764 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-05-09T10:51:35.764 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-05-09T10:51:35.764 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1, 2024-05-09T10:51:35.764 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 1 2024-05-09T10:51:35.764 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.764 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 1, 2024-05-09T10:51:35.765 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 1, 2024-05-09T10:51:35.765 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 0, 2024-05-09T10:51:35.765 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-05-09T10:51:35.765 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-05-09T10:51:35.765 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 0, 2024-05-09T10:51:35.765 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 0, 2024-05-09T10:51:35.765 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 0, 2024-05-09T10:51:35.765 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 0, 2024-05-09T10:51:35.765 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 0, 2024-05-09T10:51:35.765 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-05-09T10:51:35.765 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-05-09T10:51:35.765 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-05-09T10:51:35.765 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-05-09T10:51:35.765 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 0, 2024-05-09T10:51:35.765 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-05-09T10:51:35.765 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-05-09T10:51:35.765 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-05-09T10:51:35.766 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 0, 2024-05-09T10:51:35.766 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 0, 2024-05-09T10:51:35.766 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-05-09T10:51:35.766 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-05-09T10:51:35.766 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-05-09T10:51:35.766 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-05-09T10:51:35.766 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-05-09T10:51:35.766 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-05-09T10:51:35.766 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T10:51:35.766 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-05-09T10:51:35.766 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-05-09T10:51:35.766 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 1, 2024-05-09T10:51:35.766 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-05-09T10:51:35.766 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-05-09T10:51:35.766 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-05-09T10:51:35.766 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-05-09T10:51:35.767 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-05-09T10:51:35.767 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-05-09T10:51:35.767 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-05-09T10:51:35.767 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-05-09T10:51:35.767 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-05-09T10:51:35.767 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-05-09T10:51:35.767 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-05-09T10:51:35.767 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-05-09T10:51:35.767 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-05-09T10:51:35.767 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-05-09T10:51:35.767 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-05-09T10:51:35.767 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-05-09T10:51:35.767 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 0, 2024-05-09T10:51:35.767 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-05-09T10:51:35.767 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 0, 2024-05-09T10:51:35.767 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 1, 2024-05-09T10:51:35.768 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 1, 2024-05-09T10:51:35.768 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2024-05-09T10:51:35.768 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-05-09T10:51:35.768 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 0, 2024-05-09T10:51:35.768 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 0, 2024-05-09T10:51:35.768 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 0 2024-05-09T10:51:35.768 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.768 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter-0x55fbaf7c6140": { 2024-05-09T10:51:35.768 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-05-09T10:51:35.768 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-05-09T10:51:35.768 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 0, 2024-05-09T10:51:35.768 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 0, 2024-05-09T10:51:35.768 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 0, 2024-05-09T10:51:35.768 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 0, 2024-05-09T10:51:35.768 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-05-09T10:51:35.768 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.768 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.769 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.769 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-05-09T10:51:35.769 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-05-09T10:51:35.769 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.769 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0 2024-05-09T10:51:35.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.769 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 0, 2024-05-09T10:51:35.769 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 0, 2024-05-09T10:51:35.769 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 0, 2024-05-09T10:51:35.769 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-05-09T10:51:35.769 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-05-09T10:51:35.769 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 0, 2024-05-09T10:51:35.769 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 0, 2024-05-09T10:51:35.770 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 0, 2024-05-09T10:51:35.770 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 0, 2024-05-09T10:51:35.770 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 0, 2024-05-09T10:51:35.770 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-05-09T10:51:35.770 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-05-09T10:51:35.770 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-05-09T10:51:35.770 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-05-09T10:51:35.770 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 0, 2024-05-09T10:51:35.770 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-05-09T10:51:35.770 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-05-09T10:51:35.770 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-05-09T10:51:35.770 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 0, 2024-05-09T10:51:35.770 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 0, 2024-05-09T10:51:35.770 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-05-09T10:51:35.770 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-05-09T10:51:35.771 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-05-09T10:51:35.771 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-05-09T10:51:35.771 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-05-09T10:51:35.771 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-05-09T10:51:35.771 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T10:51:35.771 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-05-09T10:51:35.771 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-05-09T10:51:35.771 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-05-09T10:51:35.771 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-05-09T10:51:35.771 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-05-09T10:51:35.771 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-05-09T10:51:35.771 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-05-09T10:51:35.771 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-05-09T10:51:35.771 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-05-09T10:51:35.771 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-05-09T10:51:35.772 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-05-09T10:51:35.772 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-05-09T10:51:35.772 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-05-09T10:51:35.772 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-05-09T10:51:35.772 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-05-09T10:51:35.772 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-05-09T10:51:35.772 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-05-09T10:51:35.772 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-05-09T10:51:35.772 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-05-09T10:51:35.772 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 0, 2024-05-09T10:51:35.772 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-05-09T10:51:35.772 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 0, 2024-05-09T10:51:35.772 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 0, 2024-05-09T10:51:35.772 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 0, 2024-05-09T10:51:35.772 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2024-05-09T10:51:35.772 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-05-09T10:51:35.772 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 0, 2024-05-09T10:51:35.773 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 0, 2024-05-09T10:51:35.773 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 0 2024-05-09T10:51:35.773 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.773 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter-0x55fbaf7c6960": { 2024-05-09T10:51:35.773 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-05-09T10:51:35.773 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-05-09T10:51:35.773 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 3, 2024-05-09T10:51:35.773 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 411, 2024-05-09T10:51:35.773 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 0, 2024-05-09T10:51:35.773 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 3, 2024-05-09T10:51:35.773 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-05-09T10:51:35.773 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3, 2024-05-09T10:51:35.773 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.005999933, 2024-05-09T10:51:35.773 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001999977 2024-05-09T10:51:35.773 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.773 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-05-09T10:51:35.773 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-05-09T10:51:35.774 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3, 2024-05-09T10:51:35.774 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 3 2024-05-09T10:51:35.774 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.774 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 3, 2024-05-09T10:51:35.774 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 0, 2024-05-09T10:51:35.774 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 3, 2024-05-09T10:51:35.774 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-05-09T10:51:35.774 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-05-09T10:51:35.774 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 0, 2024-05-09T10:51:35.774 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 0, 2024-05-09T10:51:35.774 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 0, 2024-05-09T10:51:35.774 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 3, 2024-05-09T10:51:35.774 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 0, 2024-05-09T10:51:35.774 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-05-09T10:51:35.774 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-05-09T10:51:35.774 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-05-09T10:51:35.775 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-05-09T10:51:35.775 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 0, 2024-05-09T10:51:35.775 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-05-09T10:51:35.775 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-05-09T10:51:35.775 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-05-09T10:51:35.775 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 0, 2024-05-09T10:51:35.775 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 0, 2024-05-09T10:51:35.775 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-05-09T10:51:35.775 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-05-09T10:51:35.775 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-05-09T10:51:35.775 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-05-09T10:51:35.775 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-05-09T10:51:35.775 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-05-09T10:51:35.775 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T10:51:35.775 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-05-09T10:51:35.776 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-05-09T10:51:35.776 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-05-09T10:51:35.776 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-05-09T10:51:35.776 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-05-09T10:51:35.776 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-05-09T10:51:35.776 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-05-09T10:51:35.776 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-05-09T10:51:35.776 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-05-09T10:51:35.776 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-05-09T10:51:35.776 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-05-09T10:51:35.776 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-05-09T10:51:35.776 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-05-09T10:51:35.776 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-05-09T10:51:35.776 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-05-09T10:51:35.776 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-05-09T10:51:35.776 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-05-09T10:51:35.777 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-05-09T10:51:35.777 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-05-09T10:51:35.777 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 0, 2024-05-09T10:51:35.777 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-05-09T10:51:35.777 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 0, 2024-05-09T10:51:35.777 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 2, 2024-05-09T10:51:35.777 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 2, 2024-05-09T10:51:35.777 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2024-05-09T10:51:35.777 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-05-09T10:51:35.777 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 0, 2024-05-09T10:51:35.777 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 0, 2024-05-09T10:51:35.777 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 0 2024-05-09T10:51:35.777 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.777 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter-0x55fbaf7c7220": { 2024-05-09T10:51:35.777 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-05-09T10:51:35.777 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-05-09T10:51:35.777 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 0, 2024-05-09T10:51:35.778 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 0, 2024-05-09T10:51:35.778 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 0, 2024-05-09T10:51:35.778 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 0, 2024-05-09T10:51:35.778 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-05-09T10:51:35.778 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.778 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.778 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.778 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.778 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-05-09T10:51:35.778 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-05-09T10:51:35.778 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.778 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0 2024-05-09T10:51:35.778 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.778 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 0, 2024-05-09T10:51:35.778 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 0, 2024-05-09T10:51:35.779 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 0, 2024-05-09T10:51:35.779 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-05-09T10:51:35.779 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-05-09T10:51:35.779 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 0, 2024-05-09T10:51:35.779 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 0, 2024-05-09T10:51:35.779 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 0, 2024-05-09T10:51:35.779 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 0, 2024-05-09T10:51:35.779 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 0, 2024-05-09T10:51:35.779 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-05-09T10:51:35.779 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-05-09T10:51:35.779 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-05-09T10:51:35.779 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-05-09T10:51:35.779 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 0, 2024-05-09T10:51:35.779 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-05-09T10:51:35.779 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-05-09T10:51:35.779 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-05-09T10:51:35.779 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 0, 2024-05-09T10:51:35.780 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 0, 2024-05-09T10:51:35.780 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-05-09T10:51:35.780 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-05-09T10:51:35.780 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-05-09T10:51:35.780 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-05-09T10:51:35.780 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-05-09T10:51:35.780 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-05-09T10:51:35.780 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T10:51:35.780 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-05-09T10:51:35.780 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-05-09T10:51:35.780 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-05-09T10:51:35.780 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-05-09T10:51:35.780 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-05-09T10:51:35.780 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-05-09T10:51:35.780 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-05-09T10:51:35.780 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-05-09T10:51:35.780 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-05-09T10:51:35.781 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-05-09T10:51:35.781 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-05-09T10:51:35.781 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-05-09T10:51:35.781 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-05-09T10:51:35.781 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-05-09T10:51:35.781 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-05-09T10:51:35.781 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-05-09T10:51:35.781 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-05-09T10:51:35.781 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-05-09T10:51:35.781 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-05-09T10:51:35.781 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 0, 2024-05-09T10:51:35.781 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-05-09T10:51:35.781 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 0, 2024-05-09T10:51:35.781 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 0, 2024-05-09T10:51:35.781 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 0, 2024-05-09T10:51:35.781 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 0, 2024-05-09T10:51:35.781 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-05-09T10:51:35.782 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 0, 2024-05-09T10:51:35.782 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 0, 2024-05-09T10:51:35.782 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 0 2024-05-09T10:51:35.782 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.782 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-mgr_client_bytes": { 2024-05-09T10:51:35.782 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.782 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 134217728, 2024-05-09T10:51:35.782 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.782 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 60, 2024-05-09T10:51:35.782 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 169728, 2024-05-09T10:51:35.782 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.782 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 60, 2024-05-09T10:51:35.782 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.782 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.782 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 60, 2024-05-09T10:51:35.783 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 169728, 2024-05-09T10:51:35.783 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.783 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.783 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.783 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.783 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.783 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.783 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-mgr_client_messages": { 2024-05-09T10:51:35.783 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.783 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 512, 2024-05-09T10:51:35.783 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.783 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 60, 2024-05-09T10:51:35.783 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 60, 2024-05-09T10:51:35.783 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.783 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 60, 2024-05-09T10:51:35.784 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.784 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.784 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 60, 2024-05-09T10:51:35.784 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 60, 2024-05-09T10:51:35.784 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.784 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.784 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.784 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.784 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.784 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.784 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-mgr_mds_bytes": { 2024-05-09T10:51:35.784 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.784 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 134217728, 2024-05-09T10:51:35.784 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.785 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 78, 2024-05-09T10:51:35.785 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 813482, 2024-05-09T10:51:35.785 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.785 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 78, 2024-05-09T10:51:35.785 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.785 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.785 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 78, 2024-05-09T10:51:35.785 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 813482, 2024-05-09T10:51:35.785 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.785 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.785 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.785 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.785 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.785 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.785 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-mgr_mds_messsages": { 2024-05-09T10:51:35.786 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.786 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 128, 2024-05-09T10:51:35.786 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.786 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 78, 2024-05-09T10:51:35.786 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 78, 2024-05-09T10:51:35.786 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.786 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 78, 2024-05-09T10:51:35.786 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.786 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.786 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 78, 2024-05-09T10:51:35.786 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 78, 2024-05-09T10:51:35.786 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.786 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.786 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.786 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.787 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.787 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.787 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-mgr_mon_bytes": { 2024-05-09T10:51:35.787 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.787 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 134217728, 2024-05-09T10:51:35.787 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.787 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 63, 2024-05-09T10:51:35.787 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 455513, 2024-05-09T10:51:35.787 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.787 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 63, 2024-05-09T10:51:35.787 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.787 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.787 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 63, 2024-05-09T10:51:35.787 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 455513, 2024-05-09T10:51:35.787 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.787 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.787 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.788 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.788 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.788 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.788 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-mgr_mon_messsages": { 2024-05-09T10:51:35.788 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.788 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 128, 2024-05-09T10:51:35.788 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.788 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 63, 2024-05-09T10:51:35.788 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 63, 2024-05-09T10:51:35.788 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.788 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 63, 2024-05-09T10:51:35.788 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.788 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.788 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 63, 2024-05-09T10:51:35.788 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 63, 2024-05-09T10:51:35.788 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.789 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.789 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.789 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.789 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.789 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.789 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-mgr_osd_bytes": { 2024-05-09T10:51:35.789 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.789 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 536870912, 2024-05-09T10:51:35.789 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.789 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 336, 2024-05-09T10:51:35.789 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 4877250, 2024-05-09T10:51:35.789 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.789 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 336, 2024-05-09T10:51:35.789 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.789 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.790 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 336, 2024-05-09T10:51:35.790 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 4877250, 2024-05-09T10:51:35.790 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.790 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.790 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.790 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.790 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.790 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.790 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-mgr_osd_messsages": { 2024-05-09T10:51:35.790 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.790 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 8192, 2024-05-09T10:51:35.790 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.790 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 336, 2024-05-09T10:51:35.790 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 336, 2024-05-09T10:51:35.790 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.791 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 336, 2024-05-09T10:51:35.791 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.791 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.791 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 336, 2024-05-09T10:51:35.791 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 336, 2024-05-09T10:51:35.791 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.791 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.791 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.791 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.791 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.791 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.791 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-client": { 2024-05-09T10:51:35.791 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.791 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-05-09T10:51:35.791 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.792 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 173, 2024-05-09T10:51:35.792 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 66691, 2024-05-09T10:51:35.792 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.792 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 173, 2024-05-09T10:51:35.792 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.792 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.792 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 173, 2024-05-09T10:51:35.792 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 66691, 2024-05-09T10:51:35.792 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.792 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.792 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.792 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.792 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.792 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.792 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-client-0x55fbaf7c6460": { 2024-05-09T10:51:35.792 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.793 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-05-09T10:51:35.793 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.793 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 86, 2024-05-09T10:51:35.793 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 45723, 2024-05-09T10:51:35.793 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.793 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 86, 2024-05-09T10:51:35.793 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.793 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.793 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 86, 2024-05-09T10:51:35.793 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 45723, 2024-05-09T10:51:35.793 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.793 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.793 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.793 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.793 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.793 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.794 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-client-0x55fbaf7c6dc0": { 2024-05-09T10:51:35.794 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.794 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-05-09T10:51:35.794 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.794 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 37, 2024-05-09T10:51:35.794 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 20954, 2024-05-09T10:51:35.794 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.794 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 37, 2024-05-09T10:51:35.794 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.794 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.794 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 37, 2024-05-09T10:51:35.794 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 20954, 2024-05-09T10:51:35.794 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.794 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.794 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.795 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.795 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.795 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.795 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mgr": { 2024-05-09T10:51:35.795 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.795 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-05-09T10:51:35.795 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.795 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 347, 2024-05-09T10:51:35.795 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 839430, 2024-05-09T10:51:35.795 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.795 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 347, 2024-05-09T10:51:35.795 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.795 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.795 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 347, 2024-05-09T10:51:35.795 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 839430, 2024-05-09T10:51:35.795 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.795 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.796 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.796 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.796 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.796 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.796 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mgr-0x55fba97e9d60": { 2024-05-09T10:51:35.796 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.796 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-05-09T10:51:35.796 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.796 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 537, 2024-05-09T10:51:35.796 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 6315973, 2024-05-09T10:51:35.796 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.796 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 537, 2024-05-09T10:51:35.796 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.796 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.796 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 537, 2024-05-09T10:51:35.796 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 6315973, 2024-05-09T10:51:35.796 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.797 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.797 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.797 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.797 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.797 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.797 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-radosclient": { 2024-05-09T10:51:35.797 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.797 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-05-09T10:51:35.797 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.797 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 177, 2024-05-09T10:51:35.797 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 539313, 2024-05-09T10:51:35.797 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.797 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 177, 2024-05-09T10:51:35.797 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.797 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.797 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 177, 2024-05-09T10:51:35.797 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 539313, 2024-05-09T10:51:35.798 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.798 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.798 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.798 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.798 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.798 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.798 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-radosclient-0x55fbaecf2500": { 2024-05-09T10:51:35.798 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.798 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-05-09T10:51:35.798 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.798 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 10, 2024-05-09T10:51:35.798 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 365342, 2024-05-09T10:51:35.798 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.798 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 10, 2024-05-09T10:51:35.798 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.799 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.799 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 10, 2024-05-09T10:51:35.799 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 365342, 2024-05-09T10:51:35.799 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.799 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.799 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.799 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.799 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.799 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.799 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-radosclient-0x55fbaecf3220": { 2024-05-09T10:51:35.799 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.799 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-05-09T10:51:35.799 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.799 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 11, 2024-05-09T10:51:35.799 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 496470, 2024-05-09T10:51:35.799 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.799 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 11, 2024-05-09T10:51:35.800 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.800 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.800 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 11, 2024-05-09T10:51:35.800 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 496470, 2024-05-09T10:51:35.800 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.800 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.800 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.800 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.800 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.800 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.800 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-radosclient-0x55fbaecf3c20": { 2024-05-09T10:51:35.800 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.800 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-05-09T10:51:35.800 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.800 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 12, 2024-05-09T10:51:35.800 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 496494, 2024-05-09T10:51:35.801 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.801 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 12, 2024-05-09T10:51:35.801 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.801 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.801 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 12, 2024-05-09T10:51:35.801 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 496494, 2024-05-09T10:51:35.801 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.801 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.801 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.801 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.801 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.801 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.801 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2024-05-09T10:51:35.801 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.801 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-05-09T10:51:35.801 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.801 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-05-09T10:51:35.802 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-05-09T10:51:35.802 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.802 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-05-09T10:51:35.802 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.802 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.802 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-05-09T10:51:35.802 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-05-09T10:51:35.802 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.802 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.802 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.802 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.802 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.803 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.803 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes-0x55fbaecf20a0": { 2024-05-09T10:51:35.803 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.803 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-05-09T10:51:35.803 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.803 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-05-09T10:51:35.803 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-05-09T10:51:35.803 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.803 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-05-09T10:51:35.803 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.803 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.803 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-05-09T10:51:35.803 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-05-09T10:51:35.803 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.803 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.804 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.804 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.804 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.804 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.804 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes-0x55fbaecf2dc0": { 2024-05-09T10:51:35.804 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.804 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-05-09T10:51:35.804 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.804 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 163, 2024-05-09T10:51:35.804 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 2433027, 2024-05-09T10:51:35.804 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.804 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 163, 2024-05-09T10:51:35.804 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.804 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.804 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 163, 2024-05-09T10:51:35.805 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 2433027, 2024-05-09T10:51:35.805 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.805 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.805 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.805 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.805 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.805 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.805 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes-0x55fbaecf3180": { 2024-05-09T10:51:35.805 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.805 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-05-09T10:51:35.805 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.805 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-05-09T10:51:35.805 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-05-09T10:51:35.805 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.805 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-05-09T10:51:35.806 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.806 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.806 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-05-09T10:51:35.806 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-05-09T10:51:35.806 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.806 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.806 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.806 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.806 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.806 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.806 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes-0x55fbaecf37c0": { 2024-05-09T10:51:35.806 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.806 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-05-09T10:51:35.806 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.807 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 1, 2024-05-09T10:51:35.807 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-05-09T10:51:35.807 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.807 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 1, 2024-05-09T10:51:35.807 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.807 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.807 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 1, 2024-05-09T10:51:35.807 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-05-09T10:51:35.807 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.807 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.807 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.807 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.807 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.807 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.807 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes-0x55fbaf4112c0": { 2024-05-09T10:51:35.807 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.808 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-05-09T10:51:35.808 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.808 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-05-09T10:51:35.808 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-05-09T10:51:35.808 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.808 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-05-09T10:51:35.808 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.808 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.808 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-05-09T10:51:35.808 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-05-09T10:51:35.808 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.808 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.808 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.808 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.808 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.809 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.809 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes-0x55fbaf7c6780": { 2024-05-09T10:51:35.809 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.809 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-05-09T10:51:35.809 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.809 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-05-09T10:51:35.809 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-05-09T10:51:35.809 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.809 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-05-09T10:51:35.809 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 3, 2024-05-09T10:51:35.809 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 411, 2024-05-09T10:51:35.809 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 3, 2024-05-09T10:51:35.809 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 411, 2024-05-09T10:51:35.809 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.810 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.810 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.810 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.810 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.810 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.810 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes-0x55fbaf7c6aa0": { 2024-05-09T10:51:35.810 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.810 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-05-09T10:51:35.810 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.810 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-05-09T10:51:35.810 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-05-09T10:51:35.810 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.810 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-05-09T10:51:35.810 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.811 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.811 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-05-09T10:51:35.811 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-05-09T10:51:35.811 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.811 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.811 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.811 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.811 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.811 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.811 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2024-05-09T10:51:35.811 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.811 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-05-09T10:51:35.811 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.811 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-05-09T10:51:35.811 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-05-09T10:51:35.811 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.812 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-05-09T10:51:35.812 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.812 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.812 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-05-09T10:51:35.812 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-05-09T10:51:35.812 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.812 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.812 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.812 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.812 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.812 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.812 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops-0x55fbaecf2e60": { 2024-05-09T10:51:35.812 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.812 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-05-09T10:51:35.812 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.812 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 163, 2024-05-09T10:51:35.813 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 163, 2024-05-09T10:51:35.813 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.813 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 163, 2024-05-09T10:51:35.813 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.813 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.813 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 163, 2024-05-09T10:51:35.813 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 163, 2024-05-09T10:51:35.813 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.813 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.813 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.813 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.813 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.813 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.813 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops-0x55fbaecf30e0": { 2024-05-09T10:51:35.813 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.813 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-05-09T10:51:35.813 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.814 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-05-09T10:51:35.814 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-05-09T10:51:35.814 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.814 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-05-09T10:51:35.814 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.814 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.814 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-05-09T10:51:35.814 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-05-09T10:51:35.814 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.814 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.814 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.814 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.814 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.814 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.814 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops-0x55fbaecf3360": { 2024-05-09T10:51:35.815 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.815 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-05-09T10:51:35.815 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.815 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-05-09T10:51:35.815 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-05-09T10:51:35.815 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.815 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-05-09T10:51:35.815 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.815 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.815 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-05-09T10:51:35.815 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-05-09T10:51:35.815 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.815 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.815 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.815 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.815 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.815 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.816 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops-0x55fbaf410140": { 2024-05-09T10:51:35.816 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.816 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-05-09T10:51:35.816 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.816 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 1, 2024-05-09T10:51:35.816 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 1, 2024-05-09T10:51:35.816 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.816 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 1, 2024-05-09T10:51:35.816 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.816 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.816 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 1, 2024-05-09T10:51:35.816 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 1, 2024-05-09T10:51:35.816 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.816 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.816 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.816 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.817 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.817 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.817 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops-0x55fbaf7c6280": { 2024-05-09T10:51:35.817 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.817 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-05-09T10:51:35.817 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.817 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-05-09T10:51:35.817 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-05-09T10:51:35.817 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.817 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-05-09T10:51:35.817 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.817 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.817 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-05-09T10:51:35.817 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-05-09T10:51:35.817 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.817 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.817 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.817 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.818 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.818 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.818 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops-0x55fbaf7c63c0": { 2024-05-09T10:51:35.818 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.818 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-05-09T10:51:35.818 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.818 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-05-09T10:51:35.818 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-05-09T10:51:35.818 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.818 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-05-09T10:51:35.818 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-05-09T10:51:35.818 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-05-09T10:51:35.818 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-05-09T10:51:35.818 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-05-09T10:51:35.818 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.818 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.819 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.819 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.819 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.819 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-05-09T10:51:35.819 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops-0x55fbaf7c6820": { 2024-05-09T10:51:35.819 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-05-09T10:51:35.819 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-05-09T10:51:35.819 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-05-09T10:51:35.819 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-05-09T10:51:35.819 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-05-09T10:51:35.819 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-05-09T10:51:35.819 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-05-09T10:51:35.819 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 3, 2024-05-09T10:51:35.819 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 3, 2024-05-09T10:51:35.819 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 3, 2024-05-09T10:51:35.819 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 3, 2024-05-09T10:51:35.820 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-05-09T10:51:35.820 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-05-09T10:51:35.820 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-05-09T10:51:35.820 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-05-09T10:51:35.820 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.820 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-05-09T10:51:35.820 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-05-09T10:51:35.822 INFO:tasks.check_counter:Daemon mgr.y finisher-volumes.complete_latency.avgcount=9 2024-05-09T10:51:35.822 INFO:tasks.check_counter:Daemon mgr.y finisher-volumes.queue_len=0 2024-05-09T10:51:35.822 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-09T10:51:35.831 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-09T10:51:35.832 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:51:35.832 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:51:35.848 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056333,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18772 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T10:51:35.848 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-05-09T10:51:35.848 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:51:35.905 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056333,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18772 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T10:51:35.905 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:51:35.905 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:51:35.925 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056332,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20554 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T10:51:35.926 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-05-09T10:51:35.926 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T10:51:35.984 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056332,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20554 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T10:51:35.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:35.984 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-05-09T10:51:36.000 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-09T10:51:36.000 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T10:51:36.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:36.001 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T10:51:36.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:36.067 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T10:51:36.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:36.132 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link delete ceph-brx 2024-05-09T10:51:36.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:36.230 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-05-09T10:51:36.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:36.246 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link delete ceph-brx 2024-05-09T10:51:36.313 INFO:teuthology.orchestra.run.smithi098.stderr:Cannot find device "ceph-brx" 2024-05-09T10:51:36.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T10:51:36.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:36.315 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-05-09T10:51:36.335 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-09T10:51:36.335 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-09T10:51:36.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:36.335 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-09T10:51:36.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:36.406 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T10:51:36.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:36.475 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2024-05-09T10:51:36.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:36.569 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-05-09T10:51:36.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T10:51:36.585 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2024-05-09T10:51:36.653 INFO:teuthology.orchestra.run.smithi105.stderr:Cannot find device "ceph-brx" 2024-05-09T10:51:36.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T10:51:36.655 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-09T10:51:36.664 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-09T10:51:36.665 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T10:51:36.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.773+0000 7f369da76640 1 Processor -- start 2024-05-09T10:51:36.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.773+0000 7f369da76640 1 -- start start 2024-05-09T10:51:36.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.774+0000 7f369da76640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3698143740 0x7f36980720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:36.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.774+0000 7f369da76640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f36981443e0 0x7f369815e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:36.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.774+0000 7f369da76640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3698143b20 con 0x7f3698143d70 2024-05-09T10:51:36.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.774+0000 7f369da76640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f369815f460 con 0x7f36981443e0 2024-05-09T10:51:36.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.774+0000 7f369da76640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36980b28c0 con 0x7f3698143740 2024-05-09T10:51:36.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.774+0000 7f3696ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f36981443e0 0x7f369815e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:36.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.774+0000 7f36977fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3698143740 0x7f36980720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:36.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.774+0000 7f3696ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f36981443e0 0x7f369815e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.98:42052/0 (socket says 172.21.15.98:42052) 2024-05-09T10:51:36.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.774+0000 7f3697fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3698143d70 0x7f369815ec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.98:43374/0 (socket says 172.21.15.98:43374) 2024-05-09T10:51:36.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.774+0000 7f3696ffd640 1 -- 172.21.15.98:0/2494503769 learned_addr learned my addr 172.21.15.98:0/2494503769 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:36.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.774+0000 7f3696ffd640 1 -- 172.21.15.98:0/2494503769 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3698143740 msgr2=0x7f36980720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:36.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.774+0000 7f3696ffd640 1 --2- 172.21.15.98:0/2494503769 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3698143740 0x7f36980720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:36.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.774+0000 7f3696ffd640 1 -- 172.21.15.98:0/2494503769 >> v1:172.21.15.105:6789/0 conn(0x7f3698143d70 legacy=0x7f369815ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:36.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.774+0000 7f3696ffd640 1 -- 172.21.15.98:0/2494503769 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36980b2ba0 con 0x7f36981443e0 2024-05-09T10:51:36.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.775+0000 7f3696ffd640 1 --2- 172.21.15.98:0/2494503769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36981443e0 0x7f369815e6e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f368c009870 tx=0x7f368c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=593667e820e764d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:36.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.775+0000 7f36967fc640 1 -- 172.21.15.98:0/2494503769 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c037070 con 0x7f36981443e0 2024-05-09T10:51:36.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.775+0000 7f36967fc640 1 -- 172.21.15.98:0/2494503769 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368c032440 con 0x7f36981443e0 2024-05-09T10:51:36.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.775+0000 7f36967fc640 1 -- 172.21.15.98:0/2494503769 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c032810 con 0x7f36981443e0 2024-05-09T10:51:36.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.776+0000 7f369da76640 1 -- 172.21.15.98:0/2494503769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36981443e0 msgr2=0x7f369815e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:36.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.776+0000 7f369da76640 1 --2- 172.21.15.98:0/2494503769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36981443e0 0x7f369815e6e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f368c009870 tx=0x7f368c02a1c0 comp rx=0 tx=0).stop 2024-05-09T10:51:36.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.776+0000 7f369da76640 1 -- 172.21.15.98:0/2494503769 shutdown_connections 2024-05-09T10:51:36.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.776+0000 7f369da76640 1 --2- 172.21.15.98:0/2494503769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36981443e0 0x7f369815e6e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:36.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.776+0000 7f369da76640 1 --2- 172.21.15.98:0/2494503769 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3698143740 0x7f36980720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:36.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.776+0000 7f369da76640 1 -- 172.21.15.98:0/2494503769 >> 172.21.15.98:0/2494503769 conn(0x7f369806abe0 msgr2=0x7f369806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:36.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.776+0000 7f369da76640 1 -- 172.21.15.98:0/2494503769 shutdown_connections 2024-05-09T10:51:36.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.776+0000 7f369da76640 1 -- 172.21.15.98:0/2494503769 wait complete. 2024-05-09T10:51:36.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.777+0000 7f369da76640 1 Processor -- start 2024-05-09T10:51:36.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.777+0000 7f369da76640 1 -- start start 2024-05-09T10:51:36.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.778+0000 7f369da76640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3698143740 0x7f36980b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:36.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.778+0000 7f369da76640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3698143d70 0x7f36980aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:36.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.778+0000 7f369da76640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36981443e0 0x7f369813a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:36.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.778+0000 7f369da76640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36980cc5b0 con 0x7f3698143d70 2024-05-09T10:51:36.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.778+0000 7f369da76640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36980cc430 con 0x7f3698143740 2024-05-09T10:51:36.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.778+0000 7f369da76640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36980cc730 con 0x7f36981443e0 2024-05-09T10:51:36.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.778+0000 7f3696ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3698143d70 0x7f36980aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:36.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.778+0000 7f3696ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3698143d70 0x7f36980aecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54434/0 (socket says 172.21.15.98:54434) 2024-05-09T10:51:36.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.778+0000 7f3696ffd640 1 -- 172.21.15.98:0/3562330609 learned_addr learned my addr 172.21.15.98:0/3562330609 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:36.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.778+0000 7f3697fff640 1 --2- 172.21.15.98:0/3562330609 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36981443e0 0x7f369813a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:36.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.778+0000 7f36977fe640 1 --2- 172.21.15.98:0/3562330609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3698143740 0x7f36980b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:36.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.778+0000 7f3696ffd640 1 -- 172.21.15.98:0/3562330609 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36981443e0 msgr2=0x7f369813a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:36.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.778+0000 7f3696ffd640 1 --2- 172.21.15.98:0/3562330609 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36981443e0 0x7f369813a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:36.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.778+0000 7f3696ffd640 1 -- 172.21.15.98:0/3562330609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3698143740 msgr2=0x7f36980b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:36.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.778+0000 7f3696ffd640 1 --2- 172.21.15.98:0/3562330609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3698143740 0x7f36980b0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:36.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.778+0000 7f3696ffd640 1 -- 172.21.15.98:0/3562330609 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36980b00a0 con 0x7f3698143d70 2024-05-09T10:51:36.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.778+0000 7f36977fe640 1 --2- 172.21.15.98:0/3562330609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3698143740 0x7f36980b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:36.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.779+0000 7f3696ffd640 1 --2- 172.21.15.98:0/3562330609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3698143d70 0x7f36980aecf0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f368c000c00 tx=0x7f368c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:36.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.779+0000 7f367ffff640 1 -- 172.21.15.98:0/3562330609 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c037040 con 0x7f3698143d70 2024-05-09T10:51:36.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.779+0000 7f367ffff640 1 -- 172.21.15.98:0/3562330609 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368c042070 con 0x7f3698143d70 2024-05-09T10:51:36.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.779+0000 7f367ffff640 1 -- 172.21.15.98:0/3562330609 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c030880 con 0x7f3698143d70 2024-05-09T10:51:36.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.779+0000 7f369da76640 1 -- 172.21.15.98:0/3562330609 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36980af490 con 0x7f3698143d70 2024-05-09T10:51:36.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.779+0000 7f369da76640 1 -- 172.21.15.98:0/3562330609 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36980afa50 con 0x7f3698143d70 2024-05-09T10:51:36.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.780+0000 7f369da76640 1 -- 172.21.15.98:0/3562330609 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3698142d70 con 0x7f3698143d70 2024-05-09T10:51:36.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.780+0000 7f367ffff640 1 -- 172.21.15.98:0/3562330609 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f368c02ad10 con 0x7f3698143d70 2024-05-09T10:51:36.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.781+0000 7f367ffff640 1 --2- 172.21.15.98:0/3562330609 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f368406b680 0x7f368406db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:36.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.781+0000 7f367ffff640 1 -- 172.21.15.98:0/3562330609 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f368c0aa810 con 0x7f3698143d70 2024-05-09T10:51:36.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.781+0000 7f36977fe640 1 --2- 172.21.15.98:0/3562330609 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f368406b680 0x7f368406db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:36.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.781+0000 7f36977fe640 1 --2- 172.21.15.98:0/3562330609 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f368406b680 0x7f368406db40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3688009ac0 tx=0x7f3688008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:36.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.785+0000 7f367ffff640 1 -- 172.21.15.98:0/3562330609 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3698142d70 con 0x7f3698143d70 2024-05-09T10:51:36.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.951+0000 7f369da76640 1 -- 172.21.15.98:0/3562330609 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3698069cb0 con 0x7f368406b680 2024-05-09T10:51:36.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.952+0000 7f367ffff640 1 -- 172.21.15.98:0/3562330609 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13467 (secure 0 0 0) 0x7f3698069cb0 con 0x7f368406b680 2024-05-09T10:51:36.945 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:36.946 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-05-09T10:51:36.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.954+0000 7f369da76640 1 -- 172.21.15.98:0/3562330609 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f368406b680 msgr2=0x7f368406db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:36.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.954+0000 7f369da76640 1 --2- 172.21.15.98:0/3562330609 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f368406b680 0x7f368406db40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3688009ac0 tx=0x7f3688008040 comp rx=0 tx=0).stop 2024-05-09T10:51:36.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.954+0000 7f369da76640 1 -- 172.21.15.98:0/3562330609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3698143d70 msgr2=0x7f36980aecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:36.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.954+0000 7f369da76640 1 --2- 172.21.15.98:0/3562330609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3698143d70 0x7f36980aecf0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f368c000c00 tx=0x7f368c0046c0 comp rx=0 tx=0).stop 2024-05-09T10:51:36.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.954+0000 7f369da76640 1 -- 172.21.15.98:0/3562330609 shutdown_connections 2024-05-09T10:51:36.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.954+0000 7f369da76640 1 --2- 172.21.15.98:0/3562330609 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f368406b680 0x7f368406db40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:36.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.954+0000 7f369da76640 1 --2- 172.21.15.98:0/3562330609 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f36981443e0 0x7f369813a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:36.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.954+0000 7f369da76640 1 --2- 172.21.15.98:0/3562330609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3698143d70 0x7f36980aecf0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:36.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.954+0000 7f369da76640 1 --2- 172.21.15.98:0/3562330609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3698143740 0x7f36980b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:36.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.954+0000 7f369da76640 1 -- 172.21.15.98:0/3562330609 >> 172.21.15.98:0/3562330609 conn(0x7f369806abe0 msgr2=0x7f36980b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:36.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.954+0000 7f369da76640 1 -- 172.21.15.98:0/3562330609 shutdown_connections 2024-05-09T10:51:36.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:36.954+0000 7f369da76640 1 -- 172.21.15.98:0/3562330609 wait complete. 2024-05-09T10:51:36.958 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-05-09T10:51:35.379507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220456,"kb_used_data":4296,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754264,"statfs":{"total":773094113280,"available":772868366336,"internally_reserved":0,"allocated":4399104,"data_stored":2568892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-133250,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":-104,"num_write_kb":-174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001105"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T10:51:30.134311+0000","last_change":"2024-05-09T10:50:08.410364+0000","last_active":"2024-05-09T10:51:30.134311+0000","last_peered":"2024-05-09T10:51:30.134311+0000","last_clean":"2024-05-09T10:51:30.134311+0000","last_became_active":"2024-05-09T10:50:08.409457+0000","last_became_peered":"2024-05-09T10:50:08.409457+0000","last_unstale":"2024-05-09T10:51:30.134311+0000","last_undegraded":"2024-05-09T10:51:30.134311+0000","last_fullsized":"2024-05-09T10:51:30.134311+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_clean_scrub_stamp":"2024-05-09T10:50:07.396510+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:03.566698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":958464,"data_stored":730665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":372736,"data_stored":141889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":372736,"data_stored":141321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":271671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":140277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":271671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":954368,"data_stored":730575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T10:51:36.959 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T10:51:37.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.066+0000 7f03fcb7e640 1 Processor -- start 2024-05-09T10:51:37.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.066+0000 7f03fcb7e640 1 -- start start 2024-05-09T10:51:37.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.067+0000 7f03fcb7e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03f80ce2c0 0x7f03f80ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:37.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.067+0000 7f03fcb7e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f03f8072310 0x7f03f80727a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:37.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.067+0000 7f03fcb7e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f80cbf50 con 0x7f03f806bbf0 2024-05-09T10:51:37.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.067+0000 7f03fcb7e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03f80cc1c0 con 0x7f03f8072310 2024-05-09T10:51:37.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.067+0000 7f03fcb7e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f806f2c0 con 0x7f03f80ce2c0 2024-05-09T10:51:37.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.067+0000 7f03effff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03f80ce2c0 0x7f03f80ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:37.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.067+0000 7f03f6d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f03f8072310 0x7f03f80727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:37.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.067+0000 7f03effff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03f80ce2c0 0x7f03f80ce720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.98:34064/0 (socket says 172.21.15.98:34064) 2024-05-09T10:51:37.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.067+0000 7f03effff640 1 -- 172.21.15.98:0/2923525048 learned_addr learned my addr 172.21.15.98:0/2923525048 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:37.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.067+0000 7f03f659c640 1 --1- 172.21.15.98:0/2923525048 >> v1:172.21.15.105:6789/0 conn(0x7f03f806bbf0 0x7f03f806bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.98:43390/0 (socket says 172.21.15.98:43390) 2024-05-09T10:51:37.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.067+0000 7f03effff640 1 -- 172.21.15.98:0/2923525048 >> v2:172.21.15.105:3300/0 conn(0x7f03f8072310 msgr2=0x7f03f80727a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:37.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.067+0000 7f03effff640 1 --2- 172.21.15.98:0/2923525048 >> v2:172.21.15.105:3300/0 conn(0x7f03f8072310 0x7f03f80727a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.067+0000 7f03effff640 1 -- 172.21.15.98:0/2923525048 >> v1:172.21.15.105:6789/0 conn(0x7f03f806bbf0 legacy=0x7f03f806bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:37.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.067+0000 7f03effff640 1 -- 172.21.15.98:0/2923525048 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f806ef50 con 0x7f03f80ce2c0 2024-05-09T10:51:37.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.068+0000 7f03effff640 1 --2- 172.21.15.98:0/2923525048 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03f80ce2c0 0x7f03f80ce720 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009870 tx=0x7f03e002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fba7d84c22377fe3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:37.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.068+0000 7f03f5d9b640 1 -- 172.21.15.98:0/2923525048 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0037070 con 0x7f03f80ce2c0 2024-05-09T10:51:37.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.068+0000 7f03f5d9b640 1 -- 172.21.15.98:0/2923525048 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e0032440 con 0x7f03f80ce2c0 2024-05-09T10:51:37.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.068+0000 7f03f5d9b640 1 -- 172.21.15.98:0/2923525048 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0032810 con 0x7f03f80ce2c0 2024-05-09T10:51:37.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.069+0000 7f03fcb7e640 1 -- 172.21.15.98:0/2923525048 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03f80ce2c0 msgr2=0x7f03f80ce720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:37.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.069+0000 7f03fcb7e640 1 --2- 172.21.15.98:0/2923525048 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03f80ce2c0 0x7f03f80ce720 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009870 tx=0x7f03e002a130 comp rx=0 tx=0).stop 2024-05-09T10:51:37.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.069+0000 7f03fcb7e640 1 -- 172.21.15.98:0/2923525048 shutdown_connections 2024-05-09T10:51:37.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.069+0000 7f03fcb7e640 1 --2- 172.21.15.98:0/2923525048 >> v2:172.21.15.105:3300/0 conn(0x7f03f8072310 0x7f03f80727a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.069+0000 7f03fcb7e640 1 --2- 172.21.15.98:0/2923525048 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03f80ce2c0 0x7f03f80ce720 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.069+0000 7f03fcb7e640 1 -- 172.21.15.98:0/2923525048 >> 172.21.15.98:0/2923525048 conn(0x7f03f80ccaa0 msgr2=0x7f03f80cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:37.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.069+0000 7f03fcb7e640 1 -- 172.21.15.98:0/2923525048 shutdown_connections 2024-05-09T10:51:37.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.069+0000 7f03fcb7e640 1 -- 172.21.15.98:0/2923525048 wait complete. 2024-05-09T10:51:37.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.070+0000 7f03fcb7e640 1 Processor -- start 2024-05-09T10:51:37.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.070+0000 7f03fcb7e640 1 -- start start 2024-05-09T10:51:37.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.071+0000 7f03fcb7e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f806bbf0 0x7f03f81e61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:37.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.071+0000 7f03fcb7e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03f8072310 0x7f03f809aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:37.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.071+0000 7f03fcb7e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03f80ce2c0 0x7f03f809f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:37.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.071+0000 7f03fcb7e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f80c3950 con 0x7f03f80ce2c0 2024-05-09T10:51:37.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.071+0000 7f03fcb7e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f80c37d0 con 0x7f03f806bbf0 2024-05-09T10:51:37.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.071+0000 7f03fcb7e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f80cc400 con 0x7f03f8072310 2024-05-09T10:51:37.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.071+0000 7f03f6d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03f80ce2c0 0x7f03f809f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:37.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.071+0000 7f03f6d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03f80ce2c0 0x7f03f809f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54448/0 (socket says 172.21.15.98:54448) 2024-05-09T10:51:37.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.071+0000 7f03f6d9d640 1 -- 172.21.15.98:0/2803879309 learned_addr learned my addr 172.21.15.98:0/2803879309 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:37.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.071+0000 7f03f659c640 1 --2- 172.21.15.98:0/2803879309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f806bbf0 0x7f03f81e61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:37.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.071+0000 7f03effff640 1 --2- 172.21.15.98:0/2803879309 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03f8072310 0x7f03f809aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:37.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.072+0000 7f03f6d9d640 1 -- 172.21.15.98:0/2803879309 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03f8072310 msgr2=0x7f03f809aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:37.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.072+0000 7f03f6d9d640 1 --2- 172.21.15.98:0/2803879309 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03f8072310 0x7f03f809aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.072+0000 7f03f6d9d640 1 -- 172.21.15.98:0/2803879309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f806bbf0 msgr2=0x7f03f81e61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:37.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.072+0000 7f03f6d9d640 1 --2- 172.21.15.98:0/2803879309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f806bbf0 0x7f03f81e61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.072+0000 7f03f6d9d640 1 -- 172.21.15.98:0/2803879309 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f809b870 con 0x7f03f80ce2c0 2024-05-09T10:51:37.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.072+0000 7f03effff640 1 --2- 172.21.15.98:0/2803879309 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03f8072310 0x7f03f809aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:37.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.072+0000 7f03f659c640 1 --2- 172.21.15.98:0/2803879309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f806bbf0 0x7f03f81e61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:37.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.072+0000 7f03f6d9d640 1 --2- 172.21.15.98:0/2803879309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03f80ce2c0 0x7f03f809f9b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009dd0 tx=0x7f03f000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:37.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.072+0000 7f03eeffd640 1 -- 172.21.15.98:0/2803879309 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f0018020 con 0x7f03f80ce2c0 2024-05-09T10:51:37.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.072+0000 7f03eeffd640 1 -- 172.21.15.98:0/2803879309 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f0014480 con 0x7f03f80ce2c0 2024-05-09T10:51:37.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.072+0000 7f03fcb7e640 1 -- 172.21.15.98:0/2803879309 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f809d280 con 0x7f03f80ce2c0 2024-05-09T10:51:37.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.072+0000 7f03eeffd640 1 -- 172.21.15.98:0/2803879309 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f0014870 con 0x7f03f80ce2c0 2024-05-09T10:51:37.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.072+0000 7f03fcb7e640 1 -- 172.21.15.98:0/2803879309 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f809d840 con 0x7f03f80ce2c0 2024-05-09T10:51:37.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.073+0000 7f03fcb7e640 1 -- 172.21.15.98:0/2803879309 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f8069cb0 con 0x7f03f80ce2c0 2024-05-09T10:51:37.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.073+0000 7f03eeffd640 1 -- 172.21.15.98:0/2803879309 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f03f0005d10 con 0x7f03f80ce2c0 2024-05-09T10:51:37.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.074+0000 7f03eeffd640 1 --2- 172.21.15.98:0/2803879309 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f03d806b680 0x7f03d806db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:37.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.074+0000 7f03eeffd640 1 -- 172.21.15.98:0/2803879309 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f03f008ae40 con 0x7f03f80ce2c0 2024-05-09T10:51:37.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.074+0000 7f03f659c640 1 --2- 172.21.15.98:0/2803879309 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f03d806b680 0x7f03d806db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:37.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.074+0000 7f03f659c640 1 --2- 172.21.15.98:0/2803879309 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f03d806b680 0x7f03d806db40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f03e8007fd0 tx=0x7f03e8007d60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:37.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.078+0000 7f03eeffd640 1 -- 172.21.15.98:0/2803879309 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03f0022070 con 0x7f03f80ce2c0 2024-05-09T10:51:37.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.244+0000 7f03fcb7e640 1 -- 172.21.15.98:0/2803879309 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f03f8066cc0 con 0x7f03d806b680 2024-05-09T10:51:37.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.245+0000 7f03eeffd640 1 -- 172.21.15.98:0/2803879309 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13467 (secure 0 0 0) 0x7f03f8066cc0 con 0x7f03d806b680 2024-05-09T10:51:37.239 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:37.239 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-05-09T10:51:37.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.247+0000 7f03fcb7e640 1 -- 172.21.15.98:0/2803879309 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f03d806b680 msgr2=0x7f03d806db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:37.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.247+0000 7f03fcb7e640 1 --2- 172.21.15.98:0/2803879309 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f03d806b680 0x7f03d806db40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f03e8007fd0 tx=0x7f03e8007d60 comp rx=0 tx=0).stop 2024-05-09T10:51:37.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.247+0000 7f03fcb7e640 1 -- 172.21.15.98:0/2803879309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03f80ce2c0 msgr2=0x7f03f809f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:37.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.247+0000 7f03fcb7e640 1 --2- 172.21.15.98:0/2803879309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03f80ce2c0 0x7f03f809f9b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009dd0 tx=0x7f03f000c490 comp rx=0 tx=0).stop 2024-05-09T10:51:37.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.247+0000 7f03fcb7e640 1 -- 172.21.15.98:0/2803879309 shutdown_connections 2024-05-09T10:51:37.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.247+0000 7f03fcb7e640 1 --2- 172.21.15.98:0/2803879309 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f03d806b680 0x7f03d806db40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.247+0000 7f03fcb7e640 1 --2- 172.21.15.98:0/2803879309 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03f80ce2c0 0x7f03f809f9b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.247+0000 7f03fcb7e640 1 --2- 172.21.15.98:0/2803879309 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f03f8072310 0x7f03f809aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.247+0000 7f03fcb7e640 1 --2- 172.21.15.98:0/2803879309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f806bbf0 0x7f03f81e61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.247+0000 7f03fcb7e640 1 -- 172.21.15.98:0/2803879309 >> 172.21.15.98:0/2803879309 conn(0x7f03f80ccaa0 msgr2=0x7f03f80b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:37.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.247+0000 7f03fcb7e640 1 -- 172.21.15.98:0/2803879309 shutdown_connections 2024-05-09T10:51:37.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.247+0000 7f03fcb7e640 1 -- 172.21.15.98:0/2803879309 wait complete. 2024-05-09T10:51:37.251 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-05-09T10:51:35.379507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220456,"kb_used_data":4296,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754264,"statfs":{"total":773094113280,"available":772868366336,"internally_reserved":0,"allocated":4399104,"data_stored":2568892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-133250,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":-104,"num_write_kb":-174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001105"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T10:51:30.134311+0000","last_change":"2024-05-09T10:50:08.410364+0000","last_active":"2024-05-09T10:51:30.134311+0000","last_peered":"2024-05-09T10:51:30.134311+0000","last_clean":"2024-05-09T10:51:30.134311+0000","last_became_active":"2024-05-09T10:50:08.409457+0000","last_became_peered":"2024-05-09T10:50:08.409457+0000","last_unstale":"2024-05-09T10:51:30.134311+0000","last_undegraded":"2024-05-09T10:51:30.134311+0000","last_fullsized":"2024-05-09T10:51:30.134311+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_clean_scrub_stamp":"2024-05-09T10:50:07.396510+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:03.566698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":958464,"data_stored":730665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":372736,"data_stored":141889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":372736,"data_stored":141321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":271671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":140277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":271671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":954368,"data_stored":730575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T10:51:37.252 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-09T10:51:37.252 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T10:51:37.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.358+0000 7f872686a640 1 Processor -- start 2024-05-09T10:51:37.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.358+0000 7f872686a640 1 -- start start 2024-05-09T10:51:37.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.359+0000 7f872686a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f872015df70 0x7f87200713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:37.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.359+0000 7f872686a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8720071910 0x7f872006abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:37.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.359+0000 7f872686a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87200cdb30 con 0x7f872015db90 2024-05-09T10:51:37.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.359+0000 7f872686a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8720071f40 con 0x7f872015df70 2024-05-09T10:51:37.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.359+0000 7f872686a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8720072220 con 0x7f8720071910 2024-05-09T10:51:37.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.359+0000 7f871ffff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f872015db90 0x7f872006d430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52726/0 (socket says 172.21.15.98:52726) 2024-05-09T10:51:37.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.359+0000 7f871ffff640 1 -- 172.21.15.98:0/3889200501 learned_addr learned my addr 172.21.15.98:0/3889200501 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:37.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.359+0000 7f8724e07640 1 --2- 172.21.15.98:0/3889200501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8720071910 0x7f872006abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:37.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.359+0000 7f871f7fe640 1 --2- 172.21.15.98:0/3889200501 >> v2:172.21.15.105:3300/0 conn(0x7f872015df70 0x7f87200713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:37.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.360+0000 7f871effd640 1 -- 172.21.15.98:0/3889200501 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4178785464 0 0) 0x7f8710001110 con 0x7f872015db90 2024-05-09T10:51:37.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.360+0000 7f871effd640 1 -- 172.21.15.98:0/3889200501 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8708003520 con 0x7f872015db90 2024-05-09T10:51:37.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.360+0000 7f871f7fe640 1 -- 172.21.15.98:0/3889200501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8720071910 msgr2=0x7f872006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:37.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.360+0000 7f871f7fe640 1 --2- 172.21.15.98:0/3889200501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8720071910 0x7f872006abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.360+0000 7f871f7fe640 1 -- 172.21.15.98:0/3889200501 >> v1:172.21.15.98:6789/0 conn(0x7f872015db90 legacy=0x7f872006d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:37.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.360+0000 7f871effd640 1 -- 172.21.15.98:0/3889200501 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3195714567 0 0) 0x7f8708003520 con 0x7f872015db90 2024-05-09T10:51:37.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.360+0000 7f871f7fe640 1 -- 172.21.15.98:0/3889200501 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8720072500 con 0x7f872015df70 2024-05-09T10:51:37.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.361+0000 7f871f7fe640 1 --2- 172.21.15.98:0/3889200501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f872015df70 0x7f87200713d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f87140097a0 tx=0x7f871402a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d43282552f52b02 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:37.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.361+0000 7f871effd640 1 -- 172.21.15.98:0/3889200501 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8714037070 con 0x7f872015df70 2024-05-09T10:51:37.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.361+0000 7f871effd640 1 -- 172.21.15.98:0/3889200501 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871402ab10 con 0x7f872015df70 2024-05-09T10:51:37.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.361+0000 7f871effd640 1 -- 172.21.15.98:0/3889200501 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8710001840 con 0x7f872015df70 2024-05-09T10:51:37.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.361+0000 7f872686a640 1 -- 172.21.15.98:0/3889200501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f872015df70 msgr2=0x7f87200713d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:37.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.361+0000 7f872686a640 1 --2- 172.21.15.98:0/3889200501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f872015df70 0x7f87200713d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f87140097a0 tx=0x7f871402a080 comp rx=0 tx=0).stop 2024-05-09T10:51:37.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.361+0000 7f872686a640 1 -- 172.21.15.98:0/3889200501 shutdown_connections 2024-05-09T10:51:37.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.361+0000 7f872686a640 1 --2- 172.21.15.98:0/3889200501 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8720071910 0x7f872006abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.361+0000 7f872686a640 1 --2- 172.21.15.98:0/3889200501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f872015df70 0x7f87200713d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.361+0000 7f872686a640 1 -- 172.21.15.98:0/3889200501 >> 172.21.15.98:0/3889200501 conn(0x7f8720066f20 msgr2=0x7f872006bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:37.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.361+0000 7f872686a640 1 -- 172.21.15.98:0/3889200501 shutdown_connections 2024-05-09T10:51:37.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.361+0000 7f872686a640 1 -- 172.21.15.98:0/3889200501 wait complete. 2024-05-09T10:51:37.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.362+0000 7f872686a640 1 Processor -- start 2024-05-09T10:51:37.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.363+0000 7f872686a640 1 -- start start 2024-05-09T10:51:37.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.363+0000 7f872686a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8720071910 0x7f87200cf840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:37.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.363+0000 7f872686a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f872015db90 0x7f87200cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:37.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.363+0000 7f872686a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f872015df70 0x7f87201f73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:37.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.363+0000 7f872686a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872006b120 con 0x7f872015df70 2024-05-09T10:51:37.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.363+0000 7f872686a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87200cefd0 con 0x7f872015db90 2024-05-09T10:51:37.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.363+0000 7f872686a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872006b2a0 con 0x7f8720071910 2024-05-09T10:51:37.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.364+0000 7f8724e07640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f872015df70 0x7f87201f73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:37.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.364+0000 7f8724e07640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f872015df70 0x7f87201f73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54450/0 (socket says 172.21.15.98:54450) 2024-05-09T10:51:37.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.364+0000 7f8724e07640 1 -- 172.21.15.98:0/3380945087 learned_addr learned my addr 172.21.15.98:0/3380945087 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:37.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.364+0000 7f871ffff640 1 --2- 172.21.15.98:0/3380945087 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8720071910 0x7f87200cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:37.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.364+0000 7f871f7fe640 1 --2- 172.21.15.98:0/3380945087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f872015db90 0x7f87200cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:37.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.364+0000 7f8724e07640 1 -- 172.21.15.98:0/3380945087 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8720071910 msgr2=0x7f87200cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:37.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.364+0000 7f8724e07640 1 --2- 172.21.15.98:0/3380945087 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8720071910 0x7f87200cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.364+0000 7f8724e07640 1 -- 172.21.15.98:0/3380945087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f872015db90 msgr2=0x7f87200cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:37.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.364+0000 7f8724e07640 1 --2- 172.21.15.98:0/3380945087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f872015db90 0x7f87200cfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.364+0000 7f8724e07640 1 -- 172.21.15.98:0/3380945087 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87201499b0 con 0x7f872015df70 2024-05-09T10:51:37.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.364+0000 7f8724e07640 1 --2- 172.21.15.98:0/3380945087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f872015df70 0x7f87201f73d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f871800efc0 tx=0x7f871800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:37.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.364+0000 7f871cff9640 1 -- 172.21.15.98:0/3380945087 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8718018020 con 0x7f872015df70 2024-05-09T10:51:37.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.364+0000 7f871cff9640 1 -- 172.21.15.98:0/3380945087 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8718014480 con 0x7f872015df70 2024-05-09T10:51:37.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.364+0000 7f872686a640 1 -- 172.21.15.98:0/3380945087 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8720157c00 con 0x7f872015df70 2024-05-09T10:51:37.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.364+0000 7f871cff9640 1 -- 172.21.15.98:0/3380945087 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8718014870 con 0x7f872015df70 2024-05-09T10:51:37.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.364+0000 7f872686a640 1 -- 172.21.15.98:0/3380945087 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87201581c0 con 0x7f872015df70 2024-05-09T10:51:37.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.365+0000 7f872686a640 1 -- 172.21.15.98:0/3380945087 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8720069cb0 con 0x7f872015df70 2024-05-09T10:51:37.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.366+0000 7f871cff9640 1 -- 172.21.15.98:0/3380945087 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f8718005d10 con 0x7f872015df70 2024-05-09T10:51:37.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.366+0000 7f871cff9640 1 --2- 172.21.15.98:0/3380945087 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f870c06b680 0x7f870c06db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:37.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.366+0000 7f871cff9640 1 -- 172.21.15.98:0/3380945087 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f871808afb0 con 0x7f872015df70 2024-05-09T10:51:37.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.366+0000 7f871ffff640 1 --2- 172.21.15.98:0/3380945087 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f870c06b680 0x7f870c06db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:37.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.367+0000 7f871ffff640 1 --2- 172.21.15.98:0/3380945087 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f870c06b680 0x7f870c06db40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8710005880 tx=0x7f87100012d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:37.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.371+0000 7f871cff9640 1 -- 172.21.15.98:0/3380945087 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8718022030 con 0x7f872015df70 2024-05-09T10:51:37.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.539+0000 7f872686a640 1 -- 172.21.15.98:0/3380945087 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f87200d05a0 con 0x7f870c06b680 2024-05-09T10:51:37.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.540+0000 7f871cff9640 1 -- 172.21.15.98:0/3380945087 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13467 (secure 0 0 0) 0x7f87200d05a0 con 0x7f870c06b680 2024-05-09T10:51:37.534 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:37.534 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-05-09T10:51:37.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.542+0000 7f872686a640 1 -- 172.21.15.98:0/3380945087 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f870c06b680 msgr2=0x7f870c06db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:37.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.542+0000 7f872686a640 1 --2- 172.21.15.98:0/3380945087 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f870c06b680 0x7f870c06db40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8710005880 tx=0x7f87100012d0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.542+0000 7f872686a640 1 -- 172.21.15.98:0/3380945087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f872015df70 msgr2=0x7f87201f73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:37.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.542+0000 7f872686a640 1 --2- 172.21.15.98:0/3380945087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f872015df70 0x7f87201f73d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f871800efc0 tx=0x7f871800c490 comp rx=0 tx=0).stop 2024-05-09T10:51:37.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.542+0000 7f872686a640 1 -- 172.21.15.98:0/3380945087 shutdown_connections 2024-05-09T10:51:37.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.542+0000 7f872686a640 1 --2- 172.21.15.98:0/3380945087 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f870c06b680 0x7f870c06db40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.542+0000 7f872686a640 1 --2- 172.21.15.98:0/3380945087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f872015df70 0x7f87201f73d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.542+0000 7f872686a640 1 --2- 172.21.15.98:0/3380945087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f872015db90 0x7f87200cfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.542+0000 7f872686a640 1 --2- 172.21.15.98:0/3380945087 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8720071910 0x7f87200cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.542+0000 7f872686a640 1 -- 172.21.15.98:0/3380945087 >> 172.21.15.98:0/3380945087 conn(0x7f8720066f20 msgr2=0x7f872006bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:37.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.542+0000 7f872686a640 1 -- 172.21.15.98:0/3380945087 shutdown_connections 2024-05-09T10:51:37.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.542+0000 7f872686a640 1 -- 172.21.15.98:0/3380945087 wait complete. 2024-05-09T10:51:37.546 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-05-09T10:51:37.379930+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220456,"kb_used_data":4296,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754264,"statfs":{"total":773094113280,"available":772868366336,"internally_reserved":0,"allocated":4399104,"data_stored":2568892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-133250,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":-104,"num_write_kb":-174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.514753"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T10:51:30.134311+0000","last_change":"2024-05-09T10:50:08.410364+0000","last_active":"2024-05-09T10:51:30.134311+0000","last_peered":"2024-05-09T10:51:30.134311+0000","last_clean":"2024-05-09T10:51:30.134311+0000","last_became_active":"2024-05-09T10:50:08.409457+0000","last_became_peered":"2024-05-09T10:50:08.409457+0000","last_unstale":"2024-05-09T10:51:30.134311+0000","last_undegraded":"2024-05-09T10:51:30.134311+0000","last_fullsized":"2024-05-09T10:51:30.134311+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_clean_scrub_stamp":"2024-05-09T10:50:07.396510+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:03.566698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":958464,"data_stored":730665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":140823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":372736,"data_stored":141889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":372736,"data_stored":141321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":271671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":140277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":271671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":954368,"data_stored":730575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T10:51:37.547 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T10:51:37.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.654+0000 7fd129c5d640 1 Processor -- start 2024-05-09T10:51:37.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.654+0000 7fd129c5d640 1 -- start start 2024-05-09T10:51:37.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.655+0000 7fd129c5d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1240ce2c0 0x7fd1240ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:37.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.655+0000 7fd129c5d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd124072310 0x7fd1240727a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:37.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.655+0000 7fd129c5d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1240cbf50 con 0x7fd12406bbf0 2024-05-09T10:51:37.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.655+0000 7fd129c5d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1240cc1c0 con 0x7fd124072310 2024-05-09T10:51:37.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.655+0000 7fd129c5d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12406f2c0 con 0x7fd1240ce2c0 2024-05-09T10:51:37.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.655+0000 7fd122ffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1240ce2c0 0x7fd1240ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:37.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.655+0000 7fd123fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd124072310 0x7fd1240727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:37.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.655+0000 7fd1237fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd12406bbf0 0x7fd12406bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.98:43400/0 (socket says 172.21.15.98:43400) 2024-05-09T10:51:37.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.655+0000 7fd1237fe640 1 -- 172.21.15.98:0/1210675974 learned_addr learned my addr 172.21.15.98:0/1210675974 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:37.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.655+0000 7fd1227fc640 1 -- 172.21.15.98:0/1210675974 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 783993711 0 0) 0x7fd1240cbf50 con 0x7fd12406bbf0 2024-05-09T10:51:37.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.655+0000 7fd1227fc640 1 -- 172.21.15.98:0/1210675974 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd10c003520 con 0x7fd12406bbf0 2024-05-09T10:51:37.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.656+0000 7fd123fff640 1 -- 172.21.15.98:0/1210675974 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1240ce2c0 msgr2=0x7fd1240ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:37.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.656+0000 7fd123fff640 1 --2- 172.21.15.98:0/1210675974 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1240ce2c0 0x7fd1240ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.656+0000 7fd123fff640 1 -- 172.21.15.98:0/1210675974 >> v1:172.21.15.105:6789/0 conn(0x7fd12406bbf0 legacy=0x7fd12406bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:37.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.656+0000 7fd123fff640 1 -- 172.21.15.98:0/1210675974 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12406ef50 con 0x7fd124072310 2024-05-09T10:51:37.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.656+0000 7fd122ffd640 1 --2- 172.21.15.98:0/1210675974 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1240ce2c0 0x7fd1240ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:51:37.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.656+0000 7fd123fff640 1 --2- 172.21.15.98:0/1210675974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd124072310 0x7fd1240727a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd11c0097a0 tx=0x7fd11c02a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37b821665451d795 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:37.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.656+0000 7fd1227fc640 1 -- 172.21.15.98:0/1210675974 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c037070 con 0x7fd124072310 2024-05-09T10:51:37.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.656+0000 7fd1227fc640 1 -- 172.21.15.98:0/1210675974 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11c02adb0 con 0x7fd124072310 2024-05-09T10:51:37.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.656+0000 7fd1227fc640 1 -- 172.21.15.98:0/1210675974 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c032640 con 0x7fd124072310 2024-05-09T10:51:37.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.657+0000 7fd129c5d640 1 -- 172.21.15.98:0/1210675974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd124072310 msgr2=0x7fd1240727a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:37.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.657+0000 7fd129c5d640 1 --2- 172.21.15.98:0/1210675974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd124072310 0x7fd1240727a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd11c0097a0 tx=0x7fd11c02a060 comp rx=0 tx=0).stop 2024-05-09T10:51:37.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.657+0000 7fd129c5d640 1 -- 172.21.15.98:0/1210675974 shutdown_connections 2024-05-09T10:51:37.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.657+0000 7fd129c5d640 1 --2- 172.21.15.98:0/1210675974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd124072310 0x7fd1240727a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.657+0000 7fd129c5d640 1 --2- 172.21.15.98:0/1210675974 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1240ce2c0 0x7fd1240ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.657+0000 7fd129c5d640 1 -- 172.21.15.98:0/1210675974 >> 172.21.15.98:0/1210675974 conn(0x7fd1240ccaa0 msgr2=0x7fd1240cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:37.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.657+0000 7fd129c5d640 1 -- 172.21.15.98:0/1210675974 shutdown_connections 2024-05-09T10:51:37.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.657+0000 7fd129c5d640 1 -- 172.21.15.98:0/1210675974 wait complete. 2024-05-09T10:51:37.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.658+0000 7fd129c5d640 1 Processor -- start 2024-05-09T10:51:37.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.658+0000 7fd129c5d640 1 -- start start 2024-05-09T10:51:37.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.659+0000 7fd129c5d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd12406bbf0 0x7fd1240c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:37.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.659+0000 7fd129c5d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd124072310 0x7fd1241f2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:37.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.659+0000 7fd129c5d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1240ce2c0 0x7fd1241f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:37.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.659+0000 7fd129c5d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1240c3950 con 0x7fd124072310 2024-05-09T10:51:37.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.659+0000 7fd129c5d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1240c37d0 con 0x7fd12406bbf0 2024-05-09T10:51:37.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.659+0000 7fd129c5d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1240cc400 con 0x7fd1240ce2c0 2024-05-09T10:51:37.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.659+0000 7fd122ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd124072310 0x7fd1241f2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:37.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.659+0000 7fd122ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd124072310 0x7fd1241f2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54462/0 (socket says 172.21.15.98:54462) 2024-05-09T10:51:37.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.659+0000 7fd122ffd640 1 -- 172.21.15.98:0/1401126468 learned_addr learned my addr 172.21.15.98:0/1401126468 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:37.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.659+0000 7fd123fff640 1 --2- 172.21.15.98:0/1401126468 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1240ce2c0 0x7fd1241f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:37.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.660+0000 7fd1237fe640 1 --2- 172.21.15.98:0/1401126468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd12406bbf0 0x7fd1240c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:37.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.660+0000 7fd122ffd640 1 -- 172.21.15.98:0/1401126468 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1240ce2c0 msgr2=0x7fd1241f2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:37.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.660+0000 7fd122ffd640 1 --2- 172.21.15.98:0/1401126468 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1240ce2c0 0x7fd1241f2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.660+0000 7fd122ffd640 1 -- 172.21.15.98:0/1401126468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd12406bbf0 msgr2=0x7fd1240c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:37.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.660+0000 7fd122ffd640 1 --2- 172.21.15.98:0/1401126468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd12406bbf0 0x7fd1240c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.660+0000 7fd122ffd640 1 -- 172.21.15.98:0/1401126468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1240c4e60 con 0x7fd124072310 2024-05-09T10:51:37.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.660+0000 7fd1237fe640 1 --2- 172.21.15.98:0/1401126468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd12406bbf0 0x7fd1240c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:37.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.660+0000 7fd122ffd640 1 --2- 172.21.15.98:0/1401126468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd124072310 0x7fd1241f2a40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd11800ebe0 tx=0x7fd11800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:37.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.660+0000 7fd10bfff640 1 -- 172.21.15.98:0/1401126468 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd118017070 con 0x7fd124072310 2024-05-09T10:51:37.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.660+0000 7fd10bfff640 1 -- 172.21.15.98:0/1401126468 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11800cc30 con 0x7fd124072310 2024-05-09T10:51:37.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.660+0000 7fd10bfff640 1 -- 172.21.15.98:0/1401126468 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd118013470 con 0x7fd124072310 2024-05-09T10:51:37.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.660+0000 7fd129c5d640 1 -- 172.21.15.98:0/1401126468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1240c5140 con 0x7fd124072310 2024-05-09T10:51:37.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.660+0000 7fd129c5d640 1 -- 172.21.15.98:0/1401126468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1240c5700 con 0x7fd124072310 2024-05-09T10:51:37.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.661+0000 7fd129c5d640 1 -- 172.21.15.98:0/1401126468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd124069cb0 con 0x7fd124072310 2024-05-09T10:51:37.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.661+0000 7fd10bfff640 1 -- 172.21.15.98:0/1401126468 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7fd11801a070 con 0x7fd124072310 2024-05-09T10:51:37.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.662+0000 7fd10bfff640 1 --2- 172.21.15.98:0/1401126468 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd11006b680 0x7fd11006db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:37.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.662+0000 7fd1237fe640 1 --2- 172.21.15.98:0/1401126468 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd11006b680 0x7fd11006db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:37.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.662+0000 7fd10bfff640 1 -- 172.21.15.98:0/1401126468 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd11808a510 con 0x7fd124072310 2024-05-09T10:51:37.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.662+0000 7fd1237fe640 1 --2- 172.21.15.98:0/1401126468 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd11006b680 0x7fd11006db40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd114005c00 tx=0x7fd114009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:37.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.666+0000 7fd10bfff640 1 -- 172.21.15.98:0/1401126468 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd118059410 con 0x7fd124072310 2024-05-09T10:51:37.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.836+0000 7fd129c5d640 1 -- 172.21.15.98:0/1401126468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd124066cc0 con 0x7fd124072310 2024-05-09T10:51:37.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.837+0000 7fd10bfff640 1 -- 172.21.15.98:0/1401126468 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+13018 (secure 0 0 0) 0x7fd118058d10 con 0x7fd124072310 2024-05-09T10:51:37.830 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:37.830 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":34,"fsid":"3f705980-2043-4fe5-8639-5415c9d3c944","created":"2024-05-09T10:49:55.185818+0000","modified":"2024-05-09T10:51:32.870839+0000","last_up_change":"2024-05-09T10:50:06.391445+0000","last_in_change":"2024-05-09T10:49:57.400886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T10:50:07.396423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce4e3fa1-1971-40e2-9b4f-2db997e7c8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6801","nonce":3255566819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6803","nonce":3255566819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6807","nonce":3255566819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3255566819},{"type":"v1","addr":"172.21.15.98:6805","nonce":3255566819}]},"public_addr":"172.21.15.98:6801/3255566819","cluster_addr":"172.21.15.98:6803/3255566819","heartbeat_back_addr":"172.21.15.98:6807/3255566819","heartbeat_front_addr":"172.21.15.98:6805/3255566819","state":["exists","up"]},{"osd":1,"uuid":"7ab030de-0196-40aa-a8f4-88b7ec5c6bd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6817","nonce":1902516290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6819","nonce":1902516290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6823","nonce":1902516290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":1902516290},{"type":"v1","addr":"172.21.15.98:6821","nonce":1902516290}]},"public_addr":"172.21.15.98:6817/1902516290","cluster_addr":"172.21.15.98:6819/1902516290","heartbeat_back_addr":"172.21.15.98:6823/1902516290","heartbeat_front_addr":"172.21.15.98:6821/1902516290","state":["exists","up"]},{"osd":2,"uuid":"a086d01b-45f0-4681-8691-143b9be38503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6825","nonce":4137049320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6827","nonce":4137049320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6831","nonce":4137049320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":4137049320},{"type":"v1","addr":"172.21.15.98:6829","nonce":4137049320}]},"public_addr":"172.21.15.98:6825/4137049320","cluster_addr":"172.21.15.98:6827/4137049320","heartbeat_back_addr":"172.21.15.98:6831/4137049320","heartbeat_front_addr":"172.21.15.98:6829/4137049320","state":["exists","up"]},{"osd":3,"uuid":"fbaae71d-5472-4203-9f21-bf9affa127f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6809","nonce":2633494785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6811","nonce":2633494785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6815","nonce":2633494785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2633494785},{"type":"v1","addr":"172.21.15.98:6813","nonce":2633494785}]},"public_addr":"172.21.15.98:6809/2633494785","cluster_addr":"172.21.15.98:6811/2633494785","heartbeat_back_addr":"172.21.15.98:6815/2633494785","heartbeat_front_addr":"172.21.15.98:6813/2633494785","state":["exists","up"]},{"osd":4,"uuid":"b996590f-1eac-4b34-9843-10c52c7bae73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6809","nonce":3698267721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6811","nonce":3698267721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6815","nonce":3698267721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3698267721},{"type":"v1","addr":"172.21.15.105:6813","nonce":3698267721}]},"public_addr":"172.21.15.105:6809/3698267721","cluster_addr":"172.21.15.105:6811/3698267721","heartbeat_back_addr":"172.21.15.105:6815/3698267721","heartbeat_front_addr":"172.21.15.105:6813/3698267721","state":["exists","up"]},{"osd":5,"uuid":"2a4fad49-a064-48cf-a77d-04105bb5dca5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6801","nonce":2110551679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6803","nonce":2110551679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6807","nonce":2110551679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2110551679},{"type":"v1","addr":"172.21.15.105:6805","nonce":2110551679}]},"public_addr":"172.21.15.105:6801/2110551679","cluster_addr":"172.21.15.105:6803/2110551679","heartbeat_back_addr":"172.21.15.105:6807/2110551679","heartbeat_front_addr":"172.21.15.105:6805/2110551679","state":["exists","up"]},{"osd":6,"uuid":"d4e69c47-342a-4f33-94fa-bad0fba92863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6817","nonce":721380547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6819","nonce":721380547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6823","nonce":721380547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":721380547},{"type":"v1","addr":"172.21.15.105:6821","nonce":721380547}]},"public_addr":"172.21.15.105:6817/721380547","cluster_addr":"172.21.15.105:6819/721380547","heartbeat_back_addr":"172.21.15.105:6823/721380547","heartbeat_front_addr":"172.21.15.105:6821/721380547","state":["exists","up"]},{"osd":7,"uuid":"5d0f0551-460f-4131-89b0-df7b1630a4e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6825","nonce":3129650451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6827","nonce":3129650451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6831","nonce":3129650451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3129650451},{"type":"v1","addr":"172.21.15.105:6829","nonce":3129650451}]},"public_addr":"172.21.15.105:6825/3129650451","cluster_addr":"172.21.15.105:6827/3129650451","heartbeat_back_addr":"172.21.15.105:6831/3129650451","heartbeat_front_addr":"172.21.15.105:6829/3129650451","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542275+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.593127+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.561691+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.558716+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.588488+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.534316+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.651606+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T10:50:02.542769+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.98:6837/164638690":"2024-05-10T10:51:29.862716+0000","172.21.15.98:6835/164638690":"2024-05-10T10:51:29.862716+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T10:51:37.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.839+0000 7fd129c5d640 1 -- 172.21.15.98:0/1401126468 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd11006b680 msgr2=0x7fd11006db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:37.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.839+0000 7fd129c5d640 1 --2- 172.21.15.98:0/1401126468 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd11006b680 0x7fd11006db40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd114005c00 tx=0x7fd114009040 comp rx=0 tx=0).stop 2024-05-09T10:51:37.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.839+0000 7fd129c5d640 1 -- 172.21.15.98:0/1401126468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd124072310 msgr2=0x7fd1241f2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:37.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.839+0000 7fd129c5d640 1 --2- 172.21.15.98:0/1401126468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd124072310 0x7fd1241f2a40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd11800ebe0 tx=0x7fd11800c520 comp rx=0 tx=0).stop 2024-05-09T10:51:37.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.839+0000 7fd129c5d640 1 -- 172.21.15.98:0/1401126468 shutdown_connections 2024-05-09T10:51:37.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.839+0000 7fd129c5d640 1 --2- 172.21.15.98:0/1401126468 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fd11006b680 0x7fd11006db40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.839+0000 7fd129c5d640 1 --2- 172.21.15.98:0/1401126468 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd1240ce2c0 0x7fd1241f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.839+0000 7fd129c5d640 1 --2- 172.21.15.98:0/1401126468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd124072310 0x7fd1241f2a40 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.839+0000 7fd129c5d640 1 --2- 172.21.15.98:0/1401126468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd12406bbf0 0x7fd1240c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:37.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.839+0000 7fd129c5d640 1 -- 172.21.15.98:0/1401126468 >> 172.21.15.98:0/1401126468 conn(0x7fd1240ccaa0 msgr2=0x7fd12409e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:37.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.839+0000 7fd129c5d640 1 -- 172.21.15.98:0/1401126468 shutdown_connections 2024-05-09T10:51:37.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:37.839+0000 7fd129c5d640 1 -- 172.21.15.98:0/1401126468 wait complete. 2024-05-09T10:51:38.844 INFO:tasks.ceph:Scrubbing osd.0 2024-05-09T10:51:38.844 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-09T10:51:38.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.965+0000 7f49878dd640 1 Processor -- start 2024-05-09T10:51:38.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.965+0000 7f49878dd640 1 -- start start 2024-05-09T10:51:38.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.966+0000 7f49878dd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49800b4d80 0x7f49800713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:38.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.966+0000 7f49878dd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4980071930 0x7f498006d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:38.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.966+0000 7f49878dd640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49800b01a0 con 0x7f49801584e0 2024-05-09T10:51:38.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.966+0000 7f49878dd640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49800b42a0 con 0x7f4980071930 2024-05-09T10:51:38.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.966+0000 7f49878dd640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49800b4580 con 0x7f49800b4d80 2024-05-09T10:51:38.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.966+0000 7f4985e7a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4980071930 0x7f498006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:38.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.967+0000 7f4985679640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f49801584e0 0x7f49800af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52732/0 (socket says 172.21.15.98:52732) 2024-05-09T10:51:38.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.967+0000 7f4985679640 1 -- 172.21.15.98:0/1995656239 learned_addr learned my addr 172.21.15.98:0/1995656239 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:38.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.967+0000 7f4985e7a640 1 --2- 172.21.15.98:0/1995656239 >> v2:172.21.15.98:3300/0 conn(0x7f4980071930 0x7f498006d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54474/0 (socket says 172.21.15.98:54474) 2024-05-09T10:51:38.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.967+0000 7f4984e78640 1 --2- 172.21.15.98:0/1995656239 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49800b4d80 0x7f49800713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:38.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.967+0000 7f4985e7a640 1 -- 172.21.15.98:0/1995656239 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49800b4d80 msgr2=0x7f49800713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:38.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.967+0000 7f4985e7a640 1 --2- 172.21.15.98:0/1995656239 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49800b4d80 0x7f49800713f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:38.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.967+0000 7f4985e7a640 1 -- 172.21.15.98:0/1995656239 >> v1:172.21.15.98:6789/0 conn(0x7f49801584e0 legacy=0x7f49800af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:38.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.967+0000 7f4985e7a640 1 -- 172.21.15.98:0/1995656239 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49800b4860 con 0x7f4980071930 2024-05-09T10:51:38.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.967+0000 7f497ffff640 1 -- 172.21.15.98:0/1995656239 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1286852534 0 0) 0x7f49800b01a0 con 0x7f49801584e0 2024-05-09T10:51:38.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.967+0000 7f4984e78640 1 --2- 172.21.15.98:0/1995656239 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49800b4d80 0x7f49800713f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:38.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.968+0000 7f4985e7a640 1 --2- 172.21.15.98:0/1995656239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4980071930 0x7f498006d390 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f49780097a0 tx=0x7f497802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4da2eec8d17f046 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:38.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.968+0000 7f497ffff640 1 -- 172.21.15.98:0/1995656239 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4978037070 con 0x7f4980071930 2024-05-09T10:51:38.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.968+0000 7f497ffff640 1 -- 172.21.15.98:0/1995656239 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497802ae50 con 0x7f4980071930 2024-05-09T10:51:38.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.968+0000 7f497ffff640 1 -- 172.21.15.98:0/1995656239 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4978032660 con 0x7f4980071930 2024-05-09T10:51:38.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.968+0000 7f49878dd640 1 -- 172.21.15.98:0/1995656239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4980071930 msgr2=0x7f498006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:38.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.968+0000 7f49878dd640 1 --2- 172.21.15.98:0/1995656239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4980071930 0x7f498006d390 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f49780097a0 tx=0x7f497802a100 comp rx=0 tx=0).stop 2024-05-09T10:51:38.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.968+0000 7f49878dd640 1 -- 172.21.15.98:0/1995656239 shutdown_connections 2024-05-09T10:51:38.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.968+0000 7f49878dd640 1 --2- 172.21.15.98:0/1995656239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4980071930 0x7f498006d390 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:38.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.968+0000 7f49878dd640 1 --2- 172.21.15.98:0/1995656239 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49800b4d80 0x7f49800713f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:38.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.968+0000 7f49878dd640 1 -- 172.21.15.98:0/1995656239 >> 172.21.15.98:0/1995656239 conn(0x7f49800d0860 msgr2=0x7f49800d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:38.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.968+0000 7f49878dd640 1 -- 172.21.15.98:0/1995656239 shutdown_connections 2024-05-09T10:51:38.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.968+0000 7f49878dd640 1 -- 172.21.15.98:0/1995656239 wait complete. 2024-05-09T10:51:38.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.970+0000 7f49878dd640 1 Processor -- start 2024-05-09T10:51:38.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.970+0000 7f49878dd640 1 -- start start 2024-05-09T10:51:38.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.970+0000 7f49878dd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4980071930 0x7f4980153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:38.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.970+0000 7f49878dd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49800b4d80 0x7f4980154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:38.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.970+0000 7f49878dd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49801584e0 0x7f49801eb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:38.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.970+0000 7f49878dd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49800afdf0 con 0x7f49800b4d80 2024-05-09T10:51:38.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.970+0000 7f49878dd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498015bde0 con 0x7f4980071930 2024-05-09T10:51:38.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.970+0000 7f49878dd640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4980152d10 con 0x7f49801584e0 2024-05-09T10:51:38.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.970+0000 7f4984e78640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49800b4d80 0x7f4980154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:38.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.970+0000 7f4984e78640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49800b4d80 0x7f4980154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54488/0 (socket says 172.21.15.98:54488) 2024-05-09T10:51:38.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.971+0000 7f4984e78640 1 -- 172.21.15.98:0/2418159797 learned_addr learned my addr 172.21.15.98:0/2418159797 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:38.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.971+0000 7f4985e7a640 1 --2- 172.21.15.98:0/2418159797 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49801584e0 0x7f49801eb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:38.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.971+0000 7f4985679640 1 --2- 172.21.15.98:0/2418159797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4980071930 0x7f4980153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:38.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.971+0000 7f4984e78640 1 -- 172.21.15.98:0/2418159797 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49801584e0 msgr2=0x7f49801eb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:38.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.971+0000 7f4984e78640 1 --2- 172.21.15.98:0/2418159797 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49801584e0 0x7f49801eb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:38.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.971+0000 7f4984e78640 1 -- 172.21.15.98:0/2418159797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4980071930 msgr2=0x7f4980153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:38.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.971+0000 7f4984e78640 1 --2- 172.21.15.98:0/2418159797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4980071930 0x7f4980153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:38.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.971+0000 7f4984e78640 1 -- 172.21.15.98:0/2418159797 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4980152ff0 con 0x7f49800b4d80 2024-05-09T10:51:38.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.971+0000 7f4984e78640 1 --2- 172.21.15.98:0/2418159797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49800b4d80 0x7f4980154550 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f497400c7f0 tx=0x7f497400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:38.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.971+0000 7f497dffb640 1 -- 172.21.15.98:0/2418159797 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4974015070 con 0x7f49800b4d80 2024-05-09T10:51:38.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.971+0000 7f497dffb640 1 -- 172.21.15.98:0/2418159797 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4974007a30 con 0x7f49800b4d80 2024-05-09T10:51:38.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.971+0000 7f49878dd640 1 -- 172.21.15.98:0/2418159797 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49801532d0 con 0x7f49800b4d80 2024-05-09T10:51:38.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.971+0000 7f497dffb640 1 -- 172.21.15.98:0/2418159797 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4974007e00 con 0x7f49800b4d80 2024-05-09T10:51:38.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.971+0000 7f49878dd640 1 -- 172.21.15.98:0/2418159797 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4980153890 con 0x7f49800b4d80 2024-05-09T10:51:38.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.972+0000 7f49878dd640 1 -- 172.21.15.98:0/2418159797 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4980072f30 con 0x7f49800b4d80 2024-05-09T10:51:38.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.973+0000 7f497dffb640 1 -- 172.21.15.98:0/2418159797 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f4974018070 con 0x7f49800b4d80 2024-05-09T10:51:38.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.974+0000 7f497dffb640 1 --2- 172.21.15.98:0/2418159797 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f496c06b680 0x7f496c06db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:38.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.974+0000 7f497dffb640 1 -- 172.21.15.98:0/2418159797 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f497408a840 con 0x7f49800b4d80 2024-05-09T10:51:38.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.974+0000 7f4985679640 1 --2- 172.21.15.98:0/2418159797 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f496c06b680 0x7f496c06db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:38.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.974+0000 7f497dffb640 1 --2- 172.21.15.98:0/2418159797 >> [v2:172.21.15.98:6800/3255566819,v1:172.21.15.98:6801/3255566819] conn(0x7f496c071540 0x7f496c073960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:38.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.974+0000 7f497dffb640 1 -- 172.21.15.98:0/2418159797 --> [v2:172.21.15.98:6800/3255566819,v1:172.21.15.98:6801/3255566819] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f496c074010 con 0x7f496c071540 2024-05-09T10:51:38.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.974+0000 7f497dffb640 1 -- 172.21.15.98:0/2418159797 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f497408acd0 con 0x7f49800b4d80 2024-05-09T10:51:38.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.974+0000 7f4985e7a640 1 --2- 172.21.15.98:0/2418159797 >> [v2:172.21.15.98:6800/3255566819,v1:172.21.15.98:6801/3255566819] conn(0x7f496c071540 0x7f496c073960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:38.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.974+0000 7f4985679640 1 --2- 172.21.15.98:0/2418159797 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f496c06b680 0x7f496c06db40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f49700012e0 tx=0x7f4970005eb0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:38.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.974+0000 7f4985e7a640 1 --2- 172.21.15.98:0/2418159797 >> [v2:172.21.15.98:6800/3255566819,v1:172.21.15.98:6801/3255566819] conn(0x7f496c071540 0x7f496c073960 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:38.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.975+0000 7f497dffb640 1 -- 172.21.15.98:0/2418159797 <== osd.0 v2:172.21.15.98:6800/3255566819 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f496c074010 con 0x7f496c071540 2024-05-09T10:51:38.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.988+0000 7f49878dd640 1 -- 172.21.15.98:0/2418159797 --> [v2:172.21.15.98:6800/3255566819,v1:172.21.15.98:6801/3255566819] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f498004f000 con 0x7f496c071540 2024-05-09T10:51:38.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.988+0000 7f497dffb640 1 -- 172.21.15.98:0/2418159797 <== osd.0 v2:172.21.15.98:6800/3255566819 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f498004f000 con 0x7f496c071540 2024-05-09T10:51:38.982 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-05-09T10:51:38.982 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T10:51:38.982 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-05-09T10:51:38.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.988+0000 7f49878dd640 1 -- 172.21.15.98:0/2418159797 >> [v2:172.21.15.98:6800/3255566819,v1:172.21.15.98:6801/3255566819] conn(0x7f496c071540 msgr2=0x7f496c073960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:38.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.988+0000 7f49878dd640 1 --2- 172.21.15.98:0/2418159797 >> [v2:172.21.15.98:6800/3255566819,v1:172.21.15.98:6801/3255566819] conn(0x7f496c071540 0x7f496c073960 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:38.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.989+0000 7f49878dd640 1 -- 172.21.15.98:0/2418159797 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f496c06b680 msgr2=0x7f496c06db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:38.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.989+0000 7f49878dd640 1 --2- 172.21.15.98:0/2418159797 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f496c06b680 0x7f496c06db40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f49700012e0 tx=0x7f4970005eb0 comp rx=0 tx=0).stop 2024-05-09T10:51:38.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.989+0000 7f49878dd640 1 -- 172.21.15.98:0/2418159797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49800b4d80 msgr2=0x7f4980154550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:38.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.989+0000 7f49878dd640 1 --2- 172.21.15.98:0/2418159797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49800b4d80 0x7f4980154550 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f497400c7f0 tx=0x7f497400cbf0 comp rx=0 tx=0).stop 2024-05-09T10:51:38.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.989+0000 7f4985e7a640 1 -- 172.21.15.98:0/2418159797 reap_dead start 2024-05-09T10:51:38.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.989+0000 7f49878dd640 1 -- 172.21.15.98:0/2418159797 shutdown_connections 2024-05-09T10:51:38.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.989+0000 7f49878dd640 1 -- 172.21.15.98:0/2418159797 >> 172.21.15.98:0/2418159797 conn(0x7f49800d0860 msgr2=0x7f498006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:38.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.989+0000 7f49878dd640 1 -- 172.21.15.98:0/2418159797 shutdown_connections 2024-05-09T10:51:38.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:38.989+0000 7f49878dd640 1 -- 172.21.15.98:0/2418159797 wait complete. 2024-05-09T10:51:38.993 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-09T10:51:39.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.099+0000 7f8e510a3640 1 Processor -- start 2024-05-09T10:51:39.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.099+0000 7f8e510a3640 1 -- start start 2024-05-09T10:51:39.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.100+0000 7f8e510a3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e4c0ceb70 0x7f8e4c0b4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.100+0000 7f8e510a3640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8e4c0b4d40 0x7f8e4c0cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.100+0000 7f8e510a3640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e4c0cc4c0 con 0x7f8e4c15bdc0 2024-05-09T10:51:39.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.100+0000 7f8e510a3640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e4c15c1a0 con 0x7f8e4c0b4d40 2024-05-09T10:51:39.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.100+0000 7f8e510a3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c0cf3c0 con 0x7f8e4c0ceb70 2024-05-09T10:51:39.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.100+0000 7f8e4ad9d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f8e4c15bdc0 0x7f8e4c0cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52744/0 (socket says 172.21.15.98:52744) 2024-05-09T10:51:39.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.100+0000 7f8e4ad9d640 1 -- 172.21.15.98:0/3910723593 learned_addr learned my addr 172.21.15.98:0/3910723593 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:39.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.100+0000 7f8e4b59e640 1 --2- 172.21.15.98:0/3910723593 >> v2:172.21.15.98:3300/0 conn(0x7f8e4c0b4d40 0x7f8e4c0cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.101+0000 7f8e4a59c640 1 --2- 172.21.15.98:0/3910723593 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e4c0ceb70 0x7f8e4c0b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.101+0000 7f8e4b59e640 1 -- 172.21.15.98:0/3910723593 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e4c0ceb70 msgr2=0x7f8e4c0b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:39.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.101+0000 7f8e4b59e640 1 --2- 172.21.15.98:0/3910723593 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e4c0ceb70 0x7f8e4c0b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.101+0000 7f8e4b59e640 1 -- 172.21.15.98:0/3910723593 >> v1:172.21.15.98:6789/0 conn(0x7f8e4c15bdc0 legacy=0x7f8e4c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.101+0000 7f8e4b59e640 1 -- 172.21.15.98:0/3910723593 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c0cf0e0 con 0x7f8e4c0b4d40 2024-05-09T10:51:39.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.101+0000 7f8e49d9b640 1 -- 172.21.15.98:0/3910723593 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3037450096 0 0) 0x7f8e4c0cc4c0 con 0x7f8e4c15bdc0 2024-05-09T10:51:39.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.101+0000 7f8e4b59e640 1 --2- 172.21.15.98:0/3910723593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e4c0b4d40 0x7f8e4c0cfa30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8e440097a0 tx=0x7f8e4402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5c7c6ee46926e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:39.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.101+0000 7f8e49d9b640 1 -- 172.21.15.98:0/3910723593 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44037070 con 0x7f8e4c0b4d40 2024-05-09T10:51:39.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.102+0000 7f8e49d9b640 1 -- 172.21.15.98:0/3910723593 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e44032440 con 0x7f8e4c0b4d40 2024-05-09T10:51:39.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.102+0000 7f8e49d9b640 1 -- 172.21.15.98:0/3910723593 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44032810 con 0x7f8e4c0b4d40 2024-05-09T10:51:39.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.102+0000 7f8e510a3640 1 -- 172.21.15.98:0/3910723593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e4c0b4d40 msgr2=0x7f8e4c0cfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.102+0000 7f8e510a3640 1 --2- 172.21.15.98:0/3910723593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e4c0b4d40 0x7f8e4c0cfa30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8e440097a0 tx=0x7f8e4402a180 comp rx=0 tx=0).stop 2024-05-09T10:51:39.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.102+0000 7f8e510a3640 1 -- 172.21.15.98:0/3910723593 shutdown_connections 2024-05-09T10:51:39.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.102+0000 7f8e510a3640 1 --2- 172.21.15.98:0/3910723593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e4c0b4d40 0x7f8e4c0cfa30 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.102+0000 7f8e510a3640 1 --2- 172.21.15.98:0/3910723593 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e4c0ceb70 0x7f8e4c0b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.102+0000 7f8e510a3640 1 -- 172.21.15.98:0/3910723593 >> 172.21.15.98:0/3910723593 conn(0x7f8e4c0b34e0 msgr2=0x7f8e4c0b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:39.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.102+0000 7f8e510a3640 1 -- 172.21.15.98:0/3910723593 shutdown_connections 2024-05-09T10:51:39.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.102+0000 7f8e510a3640 1 -- 172.21.15.98:0/3910723593 wait complete. 2024-05-09T10:51:39.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.104+0000 7f8e510a3640 1 Processor -- start 2024-05-09T10:51:39.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.104+0000 7f8e510a3640 1 -- start start 2024-05-09T10:51:39.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.104+0000 7f8e510a3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e4c0b4d40 0x7f8e4c1f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.104+0000 7f8e510a3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e4c0ceb70 0x7f8e4c1f5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.104+0000 7f8e510a3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e4c15bdc0 0x7f8e4c14acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.104+0000 7f8e510a3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c0ccd20 con 0x7f8e4c15bdc0 2024-05-09T10:51:39.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.104+0000 7f8e510a3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c0cca40 con 0x7f8e4c0ceb70 2024-05-09T10:51:39.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.104+0000 7f8e510a3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c0b51d0 con 0x7f8e4c0b4d40 2024-05-09T10:51:39.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.104+0000 7f8e4b59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e4c15bdc0 0x7f8e4c14acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.104+0000 7f8e4b59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e4c15bdc0 0x7f8e4c14acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54502/0 (socket says 172.21.15.98:54502) 2024-05-09T10:51:39.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.104+0000 7f8e4b59e640 1 -- 172.21.15.98:0/2657869701 learned_addr learned my addr 172.21.15.98:0/2657869701 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:39.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.105+0000 7f8e4ad9d640 1 --2- 172.21.15.98:0/2657869701 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e4c0b4d40 0x7f8e4c1f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.105+0000 7f8e4a59c640 1 --2- 172.21.15.98:0/2657869701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e4c0ceb70 0x7f8e4c1f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.105+0000 7f8e4b59e640 1 -- 172.21.15.98:0/2657869701 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e4c0b4d40 msgr2=0x7f8e4c1f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.105+0000 7f8e4b59e640 1 --2- 172.21.15.98:0/2657869701 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e4c0b4d40 0x7f8e4c1f5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.105+0000 7f8e4b59e640 1 -- 172.21.15.98:0/2657869701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e4c0ceb70 msgr2=0x7f8e4c1f5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.105+0000 7f8e4b59e640 1 --2- 172.21.15.98:0/2657869701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e4c0ceb70 0x7f8e4c1f5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.105+0000 7f8e4b59e640 1 -- 172.21.15.98:0/2657869701 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c0cb090 con 0x7f8e4c15bdc0 2024-05-09T10:51:39.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.105+0000 7f8e4a59c640 1 --2- 172.21.15.98:0/2657869701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e4c0ceb70 0x7f8e4c1f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:39.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.105+0000 7f8e4b59e640 1 --2- 172.21.15.98:0/2657869701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e4c15bdc0 0x7f8e4c14acb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8e440097a0 tx=0x7f8e440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:39.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.105+0000 7f8e437fe640 1 -- 172.21.15.98:0/2657869701 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44037040 con 0x7f8e4c15bdc0 2024-05-09T10:51:39.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.105+0000 7f8e437fe640 1 -- 172.21.15.98:0/2657869701 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e44004230 con 0x7f8e4c15bdc0 2024-05-09T10:51:39.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.105+0000 7f8e510a3640 1 -- 172.21.15.98:0/2657869701 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e4c0cb370 con 0x7f8e4c15bdc0 2024-05-09T10:51:39.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.105+0000 7f8e437fe640 1 -- 172.21.15.98:0/2657869701 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44032e20 con 0x7f8e4c15bdc0 2024-05-09T10:51:39.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.105+0000 7f8e510a3640 1 -- 172.21.15.98:0/2657869701 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4c0cb930 con 0x7f8e4c15bdc0 2024-05-09T10:51:39.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.106+0000 7f8e510a3640 1 -- 172.21.15.98:0/2657869701 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e4c069cb0 con 0x7f8e4c15bdc0 2024-05-09T10:51:39.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.106+0000 7f8e437fe640 1 -- 172.21.15.98:0/2657869701 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f8e440306f0 con 0x7f8e4c15bdc0 2024-05-09T10:51:39.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.107+0000 7f8e437fe640 1 --2- 172.21.15.98:0/2657869701 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8e2c06b680 0x7f8e2c06db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.107+0000 7f8e437fe640 1 -- 172.21.15.98:0/2657869701 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8e440ab3d0 con 0x7f8e4c15bdc0 2024-05-09T10:51:39.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.107+0000 7f8e4ad9d640 1 --2- 172.21.15.98:0/2657869701 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8e2c06b680 0x7f8e2c06db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.107+0000 7f8e4ad9d640 1 --2- 172.21.15.98:0/2657869701 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8e2c06b680 0x7f8e2c06db40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c0012e0 tx=0x7f8e3c007ed0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:39.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.111+0000 7f8e437fe640 1 -- 172.21.15.98:0/2657869701 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8e4407a3f0 con 0x7f8e4c15bdc0 2024-05-09T10:51:39.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.282+0000 7f8e510a3640 1 -- 172.21.15.98:0/2657869701 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e4c0b7930 con 0x7f8e2c06b680 2024-05-09T10:51:39.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.282+0000 7f8e437fe640 1 -- 172.21.15.98:0/2657869701 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8e4c0b7930 con 0x7f8e2c06b680 2024-05-09T10:51:39.276 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 0 to deep-scrub 2024-05-09T10:51:39.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.284+0000 7f8e510a3640 1 -- 172.21.15.98:0/2657869701 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8e2c06b680 msgr2=0x7f8e2c06db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.284+0000 7f8e510a3640 1 --2- 172.21.15.98:0/2657869701 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8e2c06b680 0x7f8e2c06db40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c0012e0 tx=0x7f8e3c007ed0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.284+0000 7f8e510a3640 1 -- 172.21.15.98:0/2657869701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e4c15bdc0 msgr2=0x7f8e4c14acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.284+0000 7f8e510a3640 1 --2- 172.21.15.98:0/2657869701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e4c15bdc0 0x7f8e4c14acb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8e440097a0 tx=0x7f8e440046c0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.284+0000 7f8e510a3640 1 -- 172.21.15.98:0/2657869701 shutdown_connections 2024-05-09T10:51:39.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.284+0000 7f8e510a3640 1 --2- 172.21.15.98:0/2657869701 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8e2c06b680 0x7f8e2c06db40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.284+0000 7f8e510a3640 1 --2- 172.21.15.98:0/2657869701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e4c15bdc0 0x7f8e4c14acb0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.284+0000 7f8e510a3640 1 --2- 172.21.15.98:0/2657869701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e4c0ceb70 0x7f8e4c1f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.284+0000 7f8e510a3640 1 --2- 172.21.15.98:0/2657869701 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e4c0b4d40 0x7f8e4c1f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.284+0000 7f8e510a3640 1 -- 172.21.15.98:0/2657869701 >> 172.21.15.98:0/2657869701 conn(0x7f8e4c0b34e0 msgr2=0x7f8e4c06e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:39.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.284+0000 7f8e510a3640 1 -- 172.21.15.98:0/2657869701 shutdown_connections 2024-05-09T10:51:39.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.284+0000 7f8e510a3640 1 -- 172.21.15.98:0/2657869701 wait complete. 2024-05-09T10:51:39.288 INFO:tasks.ceph:Scrubbing osd.1 2024-05-09T10:51:39.289 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-09T10:51:39.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.400+0000 7feee2e07640 1 Processor -- start 2024-05-09T10:51:39.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.400+0000 7feee2e07640 1 -- start start 2024-05-09T10:51:39.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.401+0000 7feee2e07640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feedc161450 0x7feedc0d4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.401+0000 7feee2e07640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7feedc0d2820 0x7feedc0b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.401+0000 7feee2e07640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feedc161830 con 0x7feedc0b2a30 2024-05-09T10:51:39.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.401+0000 7feee2e07640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feedc0b0760 con 0x7feedc0d2820 2024-05-09T10:51:39.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.401+0000 7feee2e07640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0cf2f0 con 0x7feedc161450 2024-05-09T10:51:39.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.401+0000 7feed3fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7feedc0b2a30 0x7feedc0d2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52754/0 (socket says 172.21.15.98:52754) 2024-05-09T10:51:39.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.401+0000 7feed3fff640 1 -- 172.21.15.98:0/1616920491 learned_addr learned my addr 172.21.15.98:0/1616920491 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:39.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feed37fe640 1 -- 172.21.15.98:0/1616920491 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 516338891 0 0) 0x7feecc001110 con 0x7feedc0b2a30 2024-05-09T10:51:39.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feed37fe640 1 -- 172.21.15.98:0/1616920491 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feec4003520 con 0x7feedc0b2a30 2024-05-09T10:51:39.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feee0ba3640 1 --2- 172.21.15.98:0/1616920491 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feedc161450 0x7feedc0d4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feee13a4640 1 --2- 172.21.15.98:0/1616920491 >> v2:172.21.15.105:3300/0 conn(0x7feedc0d2820 0x7feedc0b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feed37fe640 1 -- 172.21.15.98:0/1616920491 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2289662903 0 0) 0x7feec4003520 con 0x7feedc0b2a30 2024-05-09T10:51:39.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feed37fe640 1 -- 172.21.15.98:0/1616920491 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feedc161450 msgr2=0x7feedc0d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:39.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feed37fe640 1 --2- 172.21.15.98:0/1616920491 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feedc161450 0x7feedc0d4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feed37fe640 1 -- 172.21.15.98:0/1616920491 >> v2:172.21.15.105:3300/0 conn(0x7feedc0d2820 msgr2=0x7feedc0b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:39.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feed37fe640 1 --2- 172.21.15.98:0/1616920491 >> v2:172.21.15.105:3300/0 conn(0x7feedc0d2820 0x7feedc0b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feed37fe640 1 -- 172.21.15.98:0/1616920491 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc0cf5d0 con 0x7feedc0b2a30 2024-05-09T10:51:39.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feed37fe640 1 -- 172.21.15.98:0/1616920491 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7feecc001840 con 0x7feedc0b2a30 2024-05-09T10:51:39.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feee0ba3640 1 --2- 172.21.15.98:0/1616920491 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feedc161450 0x7feedc0d4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:39.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feee13a4640 1 --2- 172.21.15.98:0/1616920491 >> v2:172.21.15.105:3300/0 conn(0x7feedc0d2820 0x7feedc0b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:39.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feed37fe640 1 -- 172.21.15.98:0/1616920491 >> v1:172.21.15.98:6789/0 conn(0x7feedc0b2a30 legacy=0x7feedc0d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feed37fe640 1 --2- 172.21.15.98:0/1616920491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feec40039b0 0x7feec4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feed37fe640 1 --2- 172.21.15.98:0/1616920491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feec4004800 0x7feec4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feed37fe640 1 --2- 172.21.15.98:0/1616920491 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feedc161450 0x7feec40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feed37fe640 1 -- 172.21.15.98:0/1616920491 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0b0760 con 0x7feec40039b0 2024-05-09T10:51:39.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feed37fe640 1 -- 172.21.15.98:0/1616920491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0cf2f0 con 0x7feec4004800 2024-05-09T10:51:39.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feed37fe640 1 -- 172.21.15.98:0/1616920491 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec400baf0 con 0x7feedc161450 2024-05-09T10:51:39.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feee0ba3640 1 --2- 172.21.15.98:0/1616920491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feec40039b0 0x7feec4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feed3fff640 1 --2- 172.21.15.98:0/1616920491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feec4004800 0x7feec4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feee13a4640 1 --2- 172.21.15.98:0/1616920491 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feedc161450 0x7feec40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feee0ba3640 1 -- 172.21.15.98:0/1616920491 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feedc161450 msgr2=0x7feec40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:39.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feee0ba3640 1 --2- 172.21.15.98:0/1616920491 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feedc161450 0x7feec40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feee0ba3640 1 -- 172.21.15.98:0/1616920491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feec4004800 msgr2=0x7feec4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:39.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feee0ba3640 1 --2- 172.21.15.98:0/1616920491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feec4004800 0x7feec4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.402+0000 7feee0ba3640 1 -- 172.21.15.98:0/1616920491 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec4003520 con 0x7feec40039b0 2024-05-09T10:51:39.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.403+0000 7feee0ba3640 1 --2- 172.21.15.98:0/1616920491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feec40039b0 0x7feec4003dd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7feed8006fd0 tx=0x7feec4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d914a5fbd0eb5e01 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:39.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.403+0000 7feed37fe640 1 -- 172.21.15.98:0/1616920491 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed800f040 con 0x7feec40039b0 2024-05-09T10:51:39.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.403+0000 7feed37fe640 1 -- 172.21.15.98:0/1616920491 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed802ac70 con 0x7feec40039b0 2024-05-09T10:51:39.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.403+0000 7feed37fe640 1 -- 172.21.15.98:0/1616920491 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed8004250 con 0x7feec40039b0 2024-05-09T10:51:39.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.404+0000 7feee2e07640 1 -- 172.21.15.98:0/1616920491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feec40039b0 msgr2=0x7feec4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.404+0000 7feee2e07640 1 --2- 172.21.15.98:0/1616920491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feec40039b0 0x7feec4003dd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7feed8006fd0 tx=0x7feec4005830 comp rx=0 tx=0).stop 2024-05-09T10:51:39.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.404+0000 7feee13a4640 1 -- 172.21.15.98:0/1616920491 reap_dead start 2024-05-09T10:51:39.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.404+0000 7feee2e07640 1 -- 172.21.15.98:0/1616920491 shutdown_connections 2024-05-09T10:51:39.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.404+0000 7feee2e07640 1 -- 172.21.15.98:0/1616920491 >> 172.21.15.98:0/1616920491 conn(0x7feedc0d3b30 msgr2=0x7feedc0d3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:39.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.404+0000 7feee2e07640 1 -- 172.21.15.98:0/1616920491 shutdown_connections 2024-05-09T10:51:39.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.404+0000 7feee2e07640 1 -- 172.21.15.98:0/1616920491 wait complete. 2024-05-09T10:51:39.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.405+0000 7feee2e07640 1 Processor -- start 2024-05-09T10:51:39.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.405+0000 7feee2e07640 1 -- start start 2024-05-09T10:51:39.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.406+0000 7feee2e07640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feec40039b0 0x7feedc09d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.406+0000 7feee2e07640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feedc14f1a0 0x7feedc09e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.406+0000 7feee2e07640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feedc09e9e0 0x7feedc0a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.406+0000 7feee2e07640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0d4650 con 0x7feedc09e9e0 2024-05-09T10:51:39.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.406+0000 7feee2e07640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0b09b0 con 0x7feedc14f1a0 2024-05-09T10:51:39.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.406+0000 7feee2e07640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0d47d0 con 0x7feec40039b0 2024-05-09T10:51:39.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.406+0000 7feee13a4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feedc09e9e0 0x7feedc0a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.406+0000 7feee13a4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feedc09e9e0 0x7feedc0a0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54530/0 (socket says 172.21.15.98:54530) 2024-05-09T10:51:39.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.406+0000 7feee13a4640 1 -- 172.21.15.98:0/1595445358 learned_addr learned my addr 172.21.15.98:0/1595445358 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:39.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.406+0000 7feee0ba3640 1 --2- 172.21.15.98:0/1595445358 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feec40039b0 0x7feedc09d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.406+0000 7feed3fff640 1 --2- 172.21.15.98:0/1595445358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feedc14f1a0 0x7feedc09e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.406+0000 7feee13a4640 1 -- 172.21.15.98:0/1595445358 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feec40039b0 msgr2=0x7feedc09d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.406+0000 7feee13a4640 1 --2- 172.21.15.98:0/1595445358 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feec40039b0 0x7feedc09d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.406+0000 7feee13a4640 1 -- 172.21.15.98:0/1595445358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feedc14f1a0 msgr2=0x7feedc09e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.406+0000 7feee13a4640 1 --2- 172.21.15.98:0/1595445358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feedc14f1a0 0x7feedc09e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.406+0000 7feee13a4640 1 -- 172.21.15.98:0/1595445358 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc150460 con 0x7feedc09e9e0 2024-05-09T10:51:39.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.406+0000 7feee0ba3640 1 --2- 172.21.15.98:0/1595445358 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feec40039b0 0x7feedc09d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:39.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.406+0000 7feee13a4640 1 --2- 172.21.15.98:0/1595445358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feedc09e9e0 0x7feedc0a0510 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7feed40045e0 tx=0x7feed4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:39.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.406+0000 7feed17fa640 1 -- 172.21.15.98:0/1595445358 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed4003ea0 con 0x7feedc09e9e0 2024-05-09T10:51:39.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.406+0000 7feed17fa640 1 -- 172.21.15.98:0/1595445358 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed4019070 con 0x7feedc09e9e0 2024-05-09T10:51:39.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.406+0000 7feee2e07640 1 -- 172.21.15.98:0/1595445358 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feedc150740 con 0x7feedc09e9e0 2024-05-09T10:51:39.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.406+0000 7feed17fa640 1 -- 172.21.15.98:0/1595445358 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed4014630 con 0x7feedc09e9e0 2024-05-09T10:51:39.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.407+0000 7feee2e07640 1 -- 172.21.15.98:0/1595445358 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feedc150d00 con 0x7feedc09e9e0 2024-05-09T10:51:39.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.407+0000 7feee2e07640 1 -- 172.21.15.98:0/1595445358 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7feedc1510e0 con 0x7feedc09e9e0 2024-05-09T10:51:39.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.408+0000 7feed17fa640 1 -- 172.21.15.98:0/1595445358 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7feed4005d10 con 0x7feedc09e9e0 2024-05-09T10:51:39.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.408+0000 7feed17fa640 1 --2- 172.21.15.98:0/1595445358 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7feec806b680 0x7feec806db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.408+0000 7feee0ba3640 1 --2- 172.21.15.98:0/1595445358 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7feec806b680 0x7feec806db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.408+0000 7feed17fa640 1 -- 172.21.15.98:0/1595445358 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7feed408b220 con 0x7feedc09e9e0 2024-05-09T10:51:39.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.409+0000 7feed17fa640 1 --2- 172.21.15.98:0/1595445358 >> [v2:172.21.15.98:6816/1902516290,v1:172.21.15.98:6817/1902516290] conn(0x7feec8071540 0x7feec8073960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.409+0000 7feed17fa640 1 -- 172.21.15.98:0/1595445358 --> [v2:172.21.15.98:6816/1902516290,v1:172.21.15.98:6817/1902516290] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feed4025e80 con 0x7feec8071540 2024-05-09T10:51:39.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.409+0000 7feed17fa640 1 -- 172.21.15.98:0/1595445358 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7feed408b6b0 con 0x7feedc09e9e0 2024-05-09T10:51:39.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.409+0000 7feed3fff640 1 --2- 172.21.15.98:0/1595445358 >> [v2:172.21.15.98:6816/1902516290,v1:172.21.15.98:6817/1902516290] conn(0x7feec8071540 0x7feec8073960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.409+0000 7feee0ba3640 1 --2- 172.21.15.98:0/1595445358 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7feec806b680 0x7feec806db40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feed8033000 tx=0x7feed802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:39.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.409+0000 7feed3fff640 1 --2- 172.21.15.98:0/1595445358 >> [v2:172.21.15.98:6816/1902516290,v1:172.21.15.98:6817/1902516290] conn(0x7feec8071540 0x7feec8073960 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:39.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.410+0000 7feed17fa640 1 -- 172.21.15.98:0/1595445358 <== osd.1 v2:172.21.15.98:6816/1902516290 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7feed4025e80 con 0x7feec8071540 2024-05-09T10:51:39.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.423+0000 7feee2e07640 1 -- 172.21.15.98:0/1595445358 --> [v2:172.21.15.98:6816/1902516290,v1:172.21.15.98:6817/1902516290] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7feedc0a1ae0 con 0x7feec8071540 2024-05-09T10:51:39.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.424+0000 7feed17fa640 1 -- 172.21.15.98:0/1595445358 <== osd.1 v2:172.21.15.98:6816/1902516290 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7feedc0a1ae0 con 0x7feec8071540 2024-05-09T10:51:39.417 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-05-09T10:51:39.417 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T10:51:39.417 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-05-09T10:51:39.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.424+0000 7feee2e07640 1 -- 172.21.15.98:0/1595445358 >> [v2:172.21.15.98:6816/1902516290,v1:172.21.15.98:6817/1902516290] conn(0x7feec8071540 msgr2=0x7feec8073960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.424+0000 7feee2e07640 1 --2- 172.21.15.98:0/1595445358 >> [v2:172.21.15.98:6816/1902516290,v1:172.21.15.98:6817/1902516290] conn(0x7feec8071540 0x7feec8073960 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.424+0000 7feee2e07640 1 -- 172.21.15.98:0/1595445358 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7feec806b680 msgr2=0x7feec806db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.424+0000 7feee2e07640 1 --2- 172.21.15.98:0/1595445358 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7feec806b680 0x7feec806db40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feed8033000 tx=0x7feed802e000 comp rx=0 tx=0).stop 2024-05-09T10:51:39.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.424+0000 7feee2e07640 1 -- 172.21.15.98:0/1595445358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feedc09e9e0 msgr2=0x7feedc0a0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.424+0000 7feee2e07640 1 --2- 172.21.15.98:0/1595445358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feedc09e9e0 0x7feedc0a0510 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7feed40045e0 tx=0x7feed4007700 comp rx=0 tx=0).stop 2024-05-09T10:51:39.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.424+0000 7feee13a4640 1 -- 172.21.15.98:0/1595445358 reap_dead start 2024-05-09T10:51:39.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.424+0000 7feee2e07640 1 -- 172.21.15.98:0/1595445358 shutdown_connections 2024-05-09T10:51:39.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.424+0000 7feee2e07640 1 -- 172.21.15.98:0/1595445358 >> 172.21.15.98:0/1595445358 conn(0x7feedc0d3b30 msgr2=0x7feedc160150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:39.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.424+0000 7feee2e07640 1 -- 172.21.15.98:0/1595445358 shutdown_connections 2024-05-09T10:51:39.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.424+0000 7feee2e07640 1 -- 172.21.15.98:0/1595445358 wait complete. 2024-05-09T10:51:39.428 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-09T10:51:39.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.534+0000 7f4ed651d640 1 Processor -- start 2024-05-09T10:51:39.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.534+0000 7f4ed651d640 1 -- start start 2024-05-09T10:51:39.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.535+0000 7f4ed651d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4ed00d0b80 0x7f4ed00b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.535+0000 7f4ed651d640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed00d2c30 con 0x7f4ed00d27a0 2024-05-09T10:51:39.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.535+0000 7f4ed651d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed015e220 con 0x7f4ed015de40 2024-05-09T10:51:39.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.535+0000 7f4ed651d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ed00d1790 con 0x7f4ed00d0b80 2024-05-09T10:51:39.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.535+0000 7f4ed4aba640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4ed00d0b80 0x7f4ed00b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.535+0000 7f4ecf7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f4ed00d27a0 0x7f4ed00d0470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52770/0 (socket says 172.21.15.98:52770) 2024-05-09T10:51:39.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.535+0000 7f4ecf7fe640 1 -- 172.21.15.98:0/546002554 learned_addr learned my addr 172.21.15.98:0/546002554 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:39.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.535+0000 7f4ed4aba640 1 -- 172.21.15.98:0/546002554 >> v1:172.21.15.105:6789/0 conn(0x7f4ed015de40 legacy=0x7f4ed00cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:39.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.535+0000 7f4eceffd640 1 -- 172.21.15.98:0/546002554 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812882918 0 0) 0x7f4ed00d2c30 con 0x7f4ed00d27a0 2024-05-09T10:51:39.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.535+0000 7f4ed4aba640 1 -- 172.21.15.98:0/546002554 >> v1:172.21.15.98:6789/0 conn(0x7f4ed00d27a0 legacy=0x7f4ed00d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.535+0000 7f4ed4aba640 1 -- 172.21.15.98:0/546002554 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed00d19e0 con 0x7f4ed00d0b80 2024-05-09T10:51:39.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.536+0000 7f4ed4aba640 1 --2- 172.21.15.98:0/546002554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ed00d0b80 0x7f4ed00b6a10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009d90 tx=0x7f4ec802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c42dd1782779ffe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:39.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.536+0000 7f4eceffd640 1 -- 172.21.15.98:0/546002554 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8007740 con 0x7f4ed00d0b80 2024-05-09T10:51:39.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.536+0000 7f4eceffd640 1 -- 172.21.15.98:0/546002554 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec80078e0 con 0x7f4ed00d0b80 2024-05-09T10:51:39.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.536+0000 7f4eceffd640 1 -- 172.21.15.98:0/546002554 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8007cb0 con 0x7f4ed00d0b80 2024-05-09T10:51:39.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.537+0000 7f4ed651d640 1 -- 172.21.15.98:0/546002554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ed00d0b80 msgr2=0x7f4ed00b6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.537+0000 7f4ed651d640 1 --2- 172.21.15.98:0/546002554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ed00d0b80 0x7f4ed00b6a10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009d90 tx=0x7f4ec802a6a0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.537+0000 7f4ed651d640 1 -- 172.21.15.98:0/546002554 shutdown_connections 2024-05-09T10:51:39.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.537+0000 7f4ed651d640 1 --2- 172.21.15.98:0/546002554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ed00d0b80 0x7f4ed00b6a10 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.537+0000 7f4ed651d640 1 -- 172.21.15.98:0/546002554 >> 172.21.15.98:0/546002554 conn(0x7f4ed009ce90 msgr2=0x7f4ed00d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:39.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.537+0000 7f4ed651d640 1 -- 172.21.15.98:0/546002554 shutdown_connections 2024-05-09T10:51:39.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.537+0000 7f4ed651d640 1 -- 172.21.15.98:0/546002554 wait complete. 2024-05-09T10:51:39.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.538+0000 7f4ed651d640 1 Processor -- start 2024-05-09T10:51:39.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.538+0000 7f4ed651d640 1 -- start start 2024-05-09T10:51:39.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.539+0000 7f4ed651d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ed00d0b80 0x7f4ed01f3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.539+0000 7f4ed651d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4ed00d27a0 0x7f4ed00ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.539+0000 7f4ed651d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ed015de40 0x7f4ed0157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.539+0000 7f4ed651d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed00d2280 con 0x7f4ed00d0b80 2024-05-09T10:51:39.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.539+0000 7f4ed651d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed00d1fa0 con 0x7f4ed015de40 2024-05-09T10:51:39.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.539+0000 7f4ed651d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed00cf960 con 0x7f4ed00d27a0 2024-05-09T10:51:39.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.539+0000 7f4ecffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ed00d0b80 0x7f4ed01f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.539+0000 7f4ecffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ed00d0b80 0x7f4ed01f3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54550/0 (socket says 172.21.15.98:54550) 2024-05-09T10:51:39.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.539+0000 7f4ecffff640 1 -- 172.21.15.98:0/2053970234 learned_addr learned my addr 172.21.15.98:0/2053970234 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:39.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.539+0000 7f4ecf7fe640 1 --2- 172.21.15.98:0/2053970234 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4ed00d27a0 0x7f4ed00ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.539+0000 7f4ed4aba640 1 --2- 172.21.15.98:0/2053970234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ed015de40 0x7f4ed0157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.539+0000 7f4ecffff640 1 -- 172.21.15.98:0/2053970234 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4ed00d27a0 msgr2=0x7f4ed00ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.539+0000 7f4ecffff640 1 --2- 172.21.15.98:0/2053970234 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4ed00d27a0 0x7f4ed00ccaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.539+0000 7f4ecffff640 1 -- 172.21.15.98:0/2053970234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ed015de40 msgr2=0x7f4ed0157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.539+0000 7f4ecffff640 1 --2- 172.21.15.98:0/2053970234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ed015de40 0x7f4ed0157a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.539+0000 7f4ecffff640 1 -- 172.21.15.98:0/2053970234 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed01f4190 con 0x7f4ed00d0b80 2024-05-09T10:51:39.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.539+0000 7f4ecffff640 1 --2- 172.21.15.98:0/2053970234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ed00d0b80 0x7f4ed01f3960 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000b3f0 tx=0x7f4ec000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:39.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.540+0000 7f4eccff9640 1 -- 172.21.15.98:0/2053970234 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0015020 con 0x7f4ed00d0b80 2024-05-09T10:51:39.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.540+0000 7f4eccff9640 1 -- 172.21.15.98:0/2053970234 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec0011830 con 0x7f4ed00d0b80 2024-05-09T10:51:39.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.540+0000 7f4ed651d640 1 -- 172.21.15.98:0/2053970234 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed01f4470 con 0x7f4ed00d0b80 2024-05-09T10:51:39.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.540+0000 7f4eccff9640 1 -- 172.21.15.98:0/2053970234 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0011c20 con 0x7f4ed00d0b80 2024-05-09T10:51:39.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.540+0000 7f4ed651d640 1 -- 172.21.15.98:0/2053970234 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed0157570 con 0x7f4ed00d0b80 2024-05-09T10:51:39.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.541+0000 7f4ed651d640 1 -- 172.21.15.98:0/2053970234 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed0157770 con 0x7f4ed00d0b80 2024-05-09T10:51:39.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.541+0000 7f4eccff9640 1 -- 172.21.15.98:0/2053970234 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f4ec0018020 con 0x7f4ed00d0b80 2024-05-09T10:51:39.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.542+0000 7f4eccff9640 1 --2- 172.21.15.98:0/2053970234 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f4ebc06b680 0x7f4ebc06db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.542+0000 7f4eccff9640 1 -- 172.21.15.98:0/2053970234 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4ec0089da0 con 0x7f4ed00d0b80 2024-05-09T10:51:39.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.542+0000 7f4ecf7fe640 1 --2- 172.21.15.98:0/2053970234 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f4ebc06b680 0x7f4ebc06db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.542+0000 7f4ecf7fe640 1 --2- 172.21.15.98:0/2053970234 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f4ebc06b680 0x7f4ebc06db40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4000f60 tx=0x7f4ec4007b50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:39.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.546+0000 7f4eccff9640 1 -- 172.21.15.98:0/2053970234 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ec001f030 con 0x7f4ed00d0b80 2024-05-09T10:51:39.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.714+0000 7f4ed651d640 1 -- 172.21.15.98:0/2053970234 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ed0157360 con 0x7f4ebc06b680 2024-05-09T10:51:39.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.715+0000 7f4eccff9640 1 -- 172.21.15.98:0/2053970234 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4ed0157360 con 0x7f4ebc06b680 2024-05-09T10:51:39.710 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 1 to deep-scrub 2024-05-09T10:51:39.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.716+0000 7f4ed651d640 1 -- 172.21.15.98:0/2053970234 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f4ebc06b680 msgr2=0x7f4ebc06db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.716+0000 7f4ed651d640 1 --2- 172.21.15.98:0/2053970234 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f4ebc06b680 0x7f4ebc06db40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4000f60 tx=0x7f4ec4007b50 comp rx=0 tx=0).stop 2024-05-09T10:51:39.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.716+0000 7f4ed651d640 1 -- 172.21.15.98:0/2053970234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ed00d0b80 msgr2=0x7f4ed01f3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.716+0000 7f4ed651d640 1 --2- 172.21.15.98:0/2053970234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ed00d0b80 0x7f4ed01f3960 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000b3f0 tx=0x7f4ec000b8b0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.716+0000 7f4ed651d640 1 -- 172.21.15.98:0/2053970234 shutdown_connections 2024-05-09T10:51:39.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.716+0000 7f4ed651d640 1 --2- 172.21.15.98:0/2053970234 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f4ebc06b680 0x7f4ebc06db40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.716+0000 7f4ed651d640 1 --2- 172.21.15.98:0/2053970234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ed015de40 0x7f4ed0157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.716+0000 7f4ed651d640 1 --2- 172.21.15.98:0/2053970234 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4ed00d27a0 0x7f4ed00ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.716+0000 7f4ed651d640 1 --2- 172.21.15.98:0/2053970234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ed00d0b80 0x7f4ed01f3960 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.716+0000 7f4ed651d640 1 -- 172.21.15.98:0/2053970234 >> 172.21.15.98:0/2053970234 conn(0x7f4ed009ce90 msgr2=0x7f4ed006b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:39.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.717+0000 7f4ed651d640 1 -- 172.21.15.98:0/2053970234 shutdown_connections 2024-05-09T10:51:39.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.717+0000 7f4ed651d640 1 -- 172.21.15.98:0/2053970234 wait complete. 2024-05-09T10:51:39.721 INFO:tasks.ceph:Scrubbing osd.2 2024-05-09T10:51:39.721 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-09T10:51:39.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.831+0000 7f192eebc640 1 Processor -- start 2024-05-09T10:51:39.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.831+0000 7f192eebc640 1 -- start start 2024-05-09T10:51:39.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.832+0000 7f192eebc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f19280cfbb0 0x7f1928161b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.832+0000 7f192eebc640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19280d0090 con 0x7f1928162070 2024-05-09T10:51:39.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.832+0000 7f192eebc640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19280d0370 con 0x7f19280b2d40 2024-05-09T10:51:39.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.832+0000 7f192eebc640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19281624a0 con 0x7f19280cfbb0 2024-05-09T10:51:39.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.832+0000 7f1927fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f19280cfbb0 0x7f1928161b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.832+0000 7f1927fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f19280cfbb0 0x7f1928161b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54558/0 (socket says 172.21.15.98:54558) 2024-05-09T10:51:39.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.832+0000 7f192d459640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f1928162070 0x7f192815d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52778/0 (socket says 172.21.15.98:52778) 2024-05-09T10:51:39.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.832+0000 7f1927fff640 1 -- 172.21.15.98:0/600659478 learned_addr learned my addr 172.21.15.98:0/600659478 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:39.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.833+0000 7f1927fff640 1 -- 172.21.15.98:0/600659478 >> v1:172.21.15.105:6789/0 conn(0x7f19280b2d40 legacy=0x7f192806c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.833+0000 7f1927fff640 1 -- 172.21.15.98:0/600659478 >> v1:172.21.15.98:6789/0 conn(0x7f1928162070 legacy=0x7f192815d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.833+0000 7f19277fe640 1 -- 172.21.15.98:0/600659478 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 274669782 0 0) 0x7f19280d0090 con 0x7f1928162070 2024-05-09T10:51:39.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.833+0000 7f1927fff640 1 -- 172.21.15.98:0/600659478 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19280d0650 con 0x7f19280cfbb0 2024-05-09T10:51:39.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.833+0000 7f1927fff640 1 --2- 172.21.15.98:0/600659478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19280cfbb0 0x7f1928161b30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f191c009870 tx=0x7f191c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302588f91ff4cb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:39.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.833+0000 7f19277fe640 1 -- 172.21.15.98:0/600659478 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c037070 con 0x7f19280cfbb0 2024-05-09T10:51:39.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.834+0000 7f19277fe640 1 -- 172.21.15.98:0/600659478 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191c032440 con 0x7f19280cfbb0 2024-05-09T10:51:39.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.834+0000 7f19277fe640 1 -- 172.21.15.98:0/600659478 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c032810 con 0x7f19280cfbb0 2024-05-09T10:51:39.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.834+0000 7f192eebc640 1 -- 172.21.15.98:0/600659478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19280cfbb0 msgr2=0x7f1928161b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.834+0000 7f192eebc640 1 --2- 172.21.15.98:0/600659478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19280cfbb0 0x7f1928161b30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f191c009870 tx=0x7f191c02a270 comp rx=0 tx=0).stop 2024-05-09T10:51:39.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.834+0000 7f192eebc640 1 -- 172.21.15.98:0/600659478 shutdown_connections 2024-05-09T10:51:39.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.834+0000 7f192eebc640 1 --2- 172.21.15.98:0/600659478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19280cfbb0 0x7f1928161b30 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.834+0000 7f192eebc640 1 -- 172.21.15.98:0/600659478 >> 172.21.15.98:0/600659478 conn(0x7f19280d1970 msgr2=0x7f19280d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:39.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.834+0000 7f192eebc640 1 -- 172.21.15.98:0/600659478 shutdown_connections 2024-05-09T10:51:39.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.834+0000 7f192eebc640 1 -- 172.21.15.98:0/600659478 wait complete. 2024-05-09T10:51:39.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.835+0000 7f192eebc640 1 Processor -- start 2024-05-09T10:51:39.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.835+0000 7f192eebc640 1 -- start start 2024-05-09T10:51:39.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.836+0000 7f192eebc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19280b2d40 0x7f19280b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.836+0000 7f192eebc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19280cfbb0 0x7f1928159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.836+0000 7f192eebc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1928162070 0x7f19280b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.836+0000 7f192eebc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192806d020 con 0x7f19280b2d40 2024-05-09T10:51:39.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.836+0000 7f192eebc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19280d0c10 con 0x7f19280cfbb0 2024-05-09T10:51:39.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.836+0000 7f192eebc640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19280b3120 con 0x7f1928162070 2024-05-09T10:51:39.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.836+0000 7f192cc58640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19280b2d40 0x7f19280b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.836+0000 7f192cc58640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19280b2d40 0x7f19280b04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54570/0 (socket says 172.21.15.98:54570) 2024-05-09T10:51:39.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.836+0000 7f192cc58640 1 -- 172.21.15.98:0/559965625 learned_addr learned my addr 172.21.15.98:0/559965625 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:39.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.836+0000 7f192d459640 1 --2- 172.21.15.98:0/559965625 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1928162070 0x7f19280b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.837+0000 7f1927fff640 1 --2- 172.21.15.98:0/559965625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19280cfbb0 0x7f1928159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.837+0000 7f192cc58640 1 -- 172.21.15.98:0/559965625 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1928162070 msgr2=0x7f19280b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.837+0000 7f192cc58640 1 --2- 172.21.15.98:0/559965625 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1928162070 0x7f19280b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.837+0000 7f192cc58640 1 -- 172.21.15.98:0/559965625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19280cfbb0 msgr2=0x7f1928159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.837+0000 7f192cc58640 1 --2- 172.21.15.98:0/559965625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19280cfbb0 0x7f1928159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.837+0000 7f192cc58640 1 -- 172.21.15.98:0/559965625 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192815c3d0 con 0x7f19280b2d40 2024-05-09T10:51:39.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.837+0000 7f192d459640 1 --2- 172.21.15.98:0/559965625 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1928162070 0x7f19280b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:39.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.837+0000 7f1927fff640 1 --2- 172.21.15.98:0/559965625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19280cfbb0 0x7f1928159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:39.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.837+0000 7f192cc58640 1 --2- 172.21.15.98:0/559965625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19280b2d40 0x7f19280b04e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f191800c400 tx=0x7f191800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:39.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.837+0000 7f19257fa640 1 -- 172.21.15.98:0/559965625 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1918017070 con 0x7f19280b2d40 2024-05-09T10:51:39.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.837+0000 7f19257fa640 1 -- 172.21.15.98:0/559965625 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19180052d0 con 0x7f19280b2d40 2024-05-09T10:51:39.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.837+0000 7f192eebc640 1 -- 172.21.15.98:0/559965625 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f192815c6b0 con 0x7f19280b2d40 2024-05-09T10:51:39.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.837+0000 7f19257fa640 1 -- 172.21.15.98:0/559965625 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1918003830 con 0x7f19280b2d40 2024-05-09T10:51:39.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.837+0000 7f192eebc640 1 -- 172.21.15.98:0/559965625 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19280b0a20 con 0x7f19280b2d40 2024-05-09T10:51:39.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.838+0000 7f192eebc640 1 -- 172.21.15.98:0/559965625 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f192815cd70 con 0x7f19280b2d40 2024-05-09T10:51:39.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.838+0000 7f19257fa640 1 -- 172.21.15.98:0/559965625 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f191801a070 con 0x7f19280b2d40 2024-05-09T10:51:39.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.839+0000 7f19257fa640 1 --2- 172.21.15.98:0/559965625 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f191406b680 0x7f191406db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.839+0000 7f19257fa640 1 -- 172.21.15.98:0/559965625 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1918089510 con 0x7f19280b2d40 2024-05-09T10:51:39.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.839+0000 7f1927fff640 1 --2- 172.21.15.98:0/559965625 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f191406b680 0x7f191406db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.839+0000 7f19257fa640 1 --2- 172.21.15.98:0/559965625 >> [v2:172.21.15.98:6824/4137049320,v1:172.21.15.98:6825/4137049320] conn(0x7f1914071540 0x7f1914073960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.839+0000 7f19257fa640 1 -- 172.21.15.98:0/559965625 --> [v2:172.21.15.98:6824/4137049320,v1:172.21.15.98:6825/4137049320] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1914074010 con 0x7f1914071540 2024-05-09T10:51:39.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.839+0000 7f19257fa640 1 -- 172.21.15.98:0/559965625 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1918090050 con 0x7f19280b2d40 2024-05-09T10:51:39.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.839+0000 7f192d459640 1 --2- 172.21.15.98:0/559965625 >> [v2:172.21.15.98:6824/4137049320,v1:172.21.15.98:6825/4137049320] conn(0x7f1914071540 0x7f1914073960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.839+0000 7f1927fff640 1 --2- 172.21.15.98:0/559965625 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f191406b680 0x7f191406db40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f191c034000 tx=0x7f191c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:39.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.840+0000 7f192d459640 1 --2- 172.21.15.98:0/559965625 >> [v2:172.21.15.98:6824/4137049320,v1:172.21.15.98:6825/4137049320] conn(0x7f1914071540 0x7f1914073960 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:39.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.840+0000 7f19257fa640 1 -- 172.21.15.98:0/559965625 <== osd.2 v2:172.21.15.98:6824/4137049320 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1914074010 con 0x7f1914071540 2024-05-09T10:51:39.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.853+0000 7f192eebc640 1 -- 172.21.15.98:0/559965625 --> [v2:172.21.15.98:6824/4137049320,v1:172.21.15.98:6825/4137049320] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f192809fa40 con 0x7f1914071540 2024-05-09T10:51:39.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.853+0000 7f19257fa640 1 -- 172.21.15.98:0/559965625 <== osd.2 v2:172.21.15.98:6824/4137049320 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f192809fa40 con 0x7f1914071540 2024-05-09T10:51:39.847 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-05-09T10:51:39.847 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T10:51:39.847 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-05-09T10:51:39.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.854+0000 7f192eebc640 1 -- 172.21.15.98:0/559965625 >> [v2:172.21.15.98:6824/4137049320,v1:172.21.15.98:6825/4137049320] conn(0x7f1914071540 msgr2=0x7f1914073960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.854+0000 7f192eebc640 1 --2- 172.21.15.98:0/559965625 >> [v2:172.21.15.98:6824/4137049320,v1:172.21.15.98:6825/4137049320] conn(0x7f1914071540 0x7f1914073960 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.854+0000 7f192eebc640 1 -- 172.21.15.98:0/559965625 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f191406b680 msgr2=0x7f191406db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.854+0000 7f192eebc640 1 --2- 172.21.15.98:0/559965625 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f191406b680 0x7f191406db40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f191c034000 tx=0x7f191c02e000 comp rx=0 tx=0).stop 2024-05-09T10:51:39.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.854+0000 7f192eebc640 1 -- 172.21.15.98:0/559965625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19280b2d40 msgr2=0x7f19280b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.854+0000 7f192eebc640 1 --2- 172.21.15.98:0/559965625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19280b2d40 0x7f19280b04e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f191800c400 tx=0x7f191800c8c0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.854+0000 7f192d459640 1 -- 172.21.15.98:0/559965625 reap_dead start 2024-05-09T10:51:39.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.854+0000 7f192eebc640 1 -- 172.21.15.98:0/559965625 shutdown_connections 2024-05-09T10:51:39.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.854+0000 7f192eebc640 1 -- 172.21.15.98:0/559965625 >> 172.21.15.98:0/559965625 conn(0x7f19280d1970 msgr2=0x7f19280c7060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:39.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.854+0000 7f192eebc640 1 -- 172.21.15.98:0/559965625 shutdown_connections 2024-05-09T10:51:39.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.854+0000 7f192eebc640 1 -- 172.21.15.98:0/559965625 wait complete. 2024-05-09T10:51:39.858 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-09T10:51:39.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.965+0000 7f2af16b6640 1 Processor -- start 2024-05-09T10:51:39.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.965+0000 7f2af16b6640 1 -- start start 2024-05-09T10:51:39.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.966+0000 7f2af16b6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2aec06d570 0x7f2aec0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.966+0000 7f2af16b6640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aec15ddc0 0x7f2aec15e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.966+0000 7f2af16b6640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aec0d07a0 con 0x7f2aec06eb30 2024-05-09T10:51:39.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.966+0000 7f2af16b6640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aec0d0db0 con 0x7f2aec06d570 2024-05-09T10:51:39.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.966+0000 7f2af16b6640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec06f9b0 con 0x7f2aec15ddc0 2024-05-09T10:51:39.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.966+0000 7f2aeaffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2aec06eb30 0x7f2aec06ef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52794/0 (socket says 172.21.15.98:52794) 2024-05-09T10:51:39.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.966+0000 7f2aeaffd640 1 -- 172.21.15.98:0/1575989440 learned_addr learned my addr 172.21.15.98:0/1575989440 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:39.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2aea7fc640 1 --2- 172.21.15.98:0/1575989440 >> v2:172.21.15.105:3300/0 conn(0x7f2aec06d570 0x7f2aec0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2aeb7fe640 1 --2- 172.21.15.98:0/1575989440 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aec15ddc0 0x7f2aec15e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2ae9ffb640 1 -- 172.21.15.98:0/1575989440 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1852178241 0 0) 0x7f2aec0d07a0 con 0x7f2aec06eb30 2024-05-09T10:51:39.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2ae9ffb640 1 -- 172.21.15.98:0/1575989440 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ad4003520 con 0x7f2aec06eb30 2024-05-09T10:51:39.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2ae9ffb640 1 -- 172.21.15.98:0/1575989440 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2573104575 0 0) 0x7f2ad4003520 con 0x7f2aec06eb30 2024-05-09T10:51:39.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2ae9ffb640 1 -- 172.21.15.98:0/1575989440 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aec15ddc0 msgr2=0x7f2aec15e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:39.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2ae9ffb640 1 --2- 172.21.15.98:0/1575989440 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aec15ddc0 0x7f2aec15e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2ae9ffb640 1 -- 172.21.15.98:0/1575989440 >> v2:172.21.15.105:3300/0 conn(0x7f2aec06d570 msgr2=0x7f2aec0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:39.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2ae9ffb640 1 --2- 172.21.15.98:0/1575989440 >> v2:172.21.15.105:3300/0 conn(0x7f2aec06d570 0x7f2aec0b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2ae9ffb640 1 -- 172.21.15.98:0/1575989440 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec06f6d0 con 0x7f2aec06eb30 2024-05-09T10:51:39.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2ae9ffb640 1 -- 172.21.15.98:0/1575989440 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f2adc0019f0 con 0x7f2aec06eb30 2024-05-09T10:51:39.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2ae9ffb640 1 -- 172.21.15.98:0/1575989440 >> v1:172.21.15.98:6789/0 conn(0x7f2aec06eb30 legacy=0x7f2aec06ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2ae9ffb640 1 --2- 172.21.15.98:0/1575989440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ad40039b0 0x7f2ad4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2ae9ffb640 1 --2- 172.21.15.98:0/1575989440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ad4004800 0x7f2ad4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2ae9ffb640 1 --2- 172.21.15.98:0/1575989440 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aec15ddc0 0x7f2ad40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2ae9ffb640 1 -- 172.21.15.98:0/1575989440 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec0d0db0 con 0x7f2ad4004800 2024-05-09T10:51:39.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2ae9ffb640 1 -- 172.21.15.98:0/1575989440 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec06f9b0 con 0x7f2ad40039b0 2024-05-09T10:51:39.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2ae9ffb640 1 -- 172.21.15.98:0/1575989440 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad400ba10 con 0x7f2aec15ddc0 2024-05-09T10:51:39.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2aea7fc640 1 --2- 172.21.15.98:0/1575989440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ad4004800 0x7f2ad4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2aeaffd640 1 --2- 172.21.15.98:0/1575989440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ad40039b0 0x7f2ad4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2aeb7fe640 1 --2- 172.21.15.98:0/1575989440 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aec15ddc0 0x7f2ad40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2aea7fc640 1 -- 172.21.15.98:0/1575989440 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aec15ddc0 msgr2=0x7f2ad40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:39.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2aea7fc640 1 --2- 172.21.15.98:0/1575989440 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aec15ddc0 0x7f2ad40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2aea7fc640 1 -- 172.21.15.98:0/1575989440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ad40039b0 msgr2=0x7f2ad4003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:39.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2aea7fc640 1 --2- 172.21.15.98:0/1575989440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ad40039b0 0x7f2ad4003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.967+0000 7f2aea7fc640 1 -- 172.21.15.98:0/1575989440 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad4003520 con 0x7f2ad4004800 2024-05-09T10:51:39.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.968+0000 7f2aea7fc640 1 --2- 172.21.15.98:0/1575989440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ad4004800 0x7f2ad4004c80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0009ac0 tx=0x7f2ad4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27981dc64967590 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:39.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.968+0000 7f2ae9ffb640 1 -- 172.21.15.98:0/1575989440 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0036070 con 0x7f2ad4004800 2024-05-09T10:51:39.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.968+0000 7f2ae9ffb640 1 -- 172.21.15.98:0/1575989440 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae0031400 con 0x7f2ad4004800 2024-05-09T10:51:39.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.968+0000 7f2ae9ffb640 1 -- 172.21.15.98:0/1575989440 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0009690 con 0x7f2ad4004800 2024-05-09T10:51:39.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.969+0000 7f2af16b6640 1 -- 172.21.15.98:0/1575989440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ad4004800 msgr2=0x7f2ad4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.969+0000 7f2af16b6640 1 --2- 172.21.15.98:0/1575989440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ad4004800 0x7f2ad4004c80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0009ac0 tx=0x7f2ad4005830 comp rx=0 tx=0).stop 2024-05-09T10:51:39.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.969+0000 7f2aeb7fe640 1 -- 172.21.15.98:0/1575989440 reap_dead start 2024-05-09T10:51:39.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.969+0000 7f2af16b6640 1 -- 172.21.15.98:0/1575989440 shutdown_connections 2024-05-09T10:51:39.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.969+0000 7f2af16b6640 1 -- 172.21.15.98:0/1575989440 >> 172.21.15.98:0/1575989440 conn(0x7f2aec0b34e0 msgr2=0x7f2aec0b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:39.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.969+0000 7f2af16b6640 1 -- 172.21.15.98:0/1575989440 shutdown_connections 2024-05-09T10:51:39.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.969+0000 7f2af16b6640 1 -- 172.21.15.98:0/1575989440 wait complete. 2024-05-09T10:51:39.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.970+0000 7f2af16b6640 1 Processor -- start 2024-05-09T10:51:39.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.970+0000 7f2af16b6640 1 -- start start 2024-05-09T10:51:39.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.971+0000 7f2af16b6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ad4004800 0x7f2aec0ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.971+0000 7f2af16b6640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aec0b46d0 0x7f2aec0c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.971+0000 7f2af16b6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2aec1f3a60 0x7f2aec0c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.971+0000 7f2af16b6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec15e3f0 con 0x7f2aec1f3a60 2024-05-09T10:51:39.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.971+0000 7f2af16b6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec15e270 con 0x7f2ad4004800 2024-05-09T10:51:39.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.971+0000 7f2af16b6640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec15e570 con 0x7f2aec0b46d0 2024-05-09T10:51:39.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.971+0000 7f2aeb7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2aec1f3a60 0x7f2aec0c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.971+0000 7f2aeb7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2aec1f3a60 0x7f2aec0c67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54588/0 (socket says 172.21.15.98:54588) 2024-05-09T10:51:39.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.971+0000 7f2aeb7fe640 1 -- 172.21.15.98:0/458654264 learned_addr learned my addr 172.21.15.98:0/458654264 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:39.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.971+0000 7f2aea7fc640 1 --2- 172.21.15.98:0/458654264 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aec0b46d0 0x7f2aec0c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.971+0000 7f2aeb7fe640 1 -- 172.21.15.98:0/458654264 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aec0b46d0 msgr2=0x7f2aec0c5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.971+0000 7f2aeb7fe640 1 --2- 172.21.15.98:0/458654264 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aec0b46d0 0x7f2aec0c5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.971+0000 7f2aeb7fe640 1 -- 172.21.15.98:0/458654264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ad4004800 msgr2=0x7f2aec0ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:39.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.971+0000 7f2aeaffd640 1 --2- 172.21.15.98:0/458654264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ad4004800 0x7f2aec0ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.971+0000 7f2aeb7fe640 1 --2- 172.21.15.98:0/458654264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ad4004800 0x7f2aec0ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:39.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.971+0000 7f2aeb7fe640 1 -- 172.21.15.98:0/458654264 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec1f49e0 con 0x7f2aec1f3a60 2024-05-09T10:51:39.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.971+0000 7f2aeb7fe640 1 --2- 172.21.15.98:0/458654264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2aec1f3a60 0x7f2aec0c67c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2aec15e1a0 tx=0x7f2ae4011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:39.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.972+0000 7f2ad37fe640 1 -- 172.21.15.98:0/458654264 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4004240 con 0x7f2aec1f3a60 2024-05-09T10:51:39.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.972+0000 7f2ad37fe640 1 -- 172.21.15.98:0/458654264 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae401b070 con 0x7f2aec1f3a60 2024-05-09T10:51:39.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.972+0000 7f2af16b6640 1 -- 172.21.15.98:0/458654264 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aec1f4480 con 0x7f2aec1f3a60 2024-05-09T10:51:39.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.972+0000 7f2ad37fe640 1 -- 172.21.15.98:0/458654264 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae400f770 con 0x7f2aec1f3a60 2024-05-09T10:51:39.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.972+0000 7f2af16b6640 1 -- 172.21.15.98:0/458654264 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aec0c57b0 con 0x7f2aec1f3a60 2024-05-09T10:51:39.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.972+0000 7f2af16b6640 1 -- 172.21.15.98:0/458654264 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aec06d2e0 con 0x7f2aec1f3a60 2024-05-09T10:51:39.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.973+0000 7f2ad37fe640 1 -- 172.21.15.98:0/458654264 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f2ae4012040 con 0x7f2aec1f3a60 2024-05-09T10:51:39.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.973+0000 7f2ad37fe640 1 --2- 172.21.15.98:0/458654264 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2ad806b680 0x7f2ad806db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:39.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.973+0000 7f2ad37fe640 1 -- 172.21.15.98:0/458654264 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2ae4089fd0 con 0x7f2aec1f3a60 2024-05-09T10:51:39.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.973+0000 7f2aeaffd640 1 --2- 172.21.15.98:0/458654264 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2ad806b680 0x7f2ad806db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:39.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.974+0000 7f2aeaffd640 1 --2- 172.21.15.98:0/458654264 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2ad806b680 0x7f2ad806db40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2adc001b10 tx=0x7f2adc006040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:39.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:39.977+0000 7f2ad37fe640 1 -- 172.21.15.98:0/458654264 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2aec1f4480 con 0x7f2aec1f3a60 2024-05-09T10:51:40.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.146+0000 7f2af16b6640 1 -- 172.21.15.98:0/458654264 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2aec0b3fe0 con 0x7f2ad806b680 2024-05-09T10:51:40.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.146+0000 7f2ad37fe640 1 -- 172.21.15.98:0/458654264 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2aec0b3fe0 con 0x7f2ad806b680 2024-05-09T10:51:40.140 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 2 to deep-scrub 2024-05-09T10:51:40.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.148+0000 7f2af16b6640 1 -- 172.21.15.98:0/458654264 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2ad806b680 msgr2=0x7f2ad806db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.148+0000 7f2af16b6640 1 --2- 172.21.15.98:0/458654264 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2ad806b680 0x7f2ad806db40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2adc001b10 tx=0x7f2adc006040 comp rx=0 tx=0).stop 2024-05-09T10:51:40.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.148+0000 7f2af16b6640 1 -- 172.21.15.98:0/458654264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2aec1f3a60 msgr2=0x7f2aec0c67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.148+0000 7f2af16b6640 1 --2- 172.21.15.98:0/458654264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2aec1f3a60 0x7f2aec0c67c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2aec15e1a0 tx=0x7f2ae4011cf0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.148+0000 7f2af16b6640 1 -- 172.21.15.98:0/458654264 shutdown_connections 2024-05-09T10:51:40.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.148+0000 7f2af16b6640 1 --2- 172.21.15.98:0/458654264 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f2ad806b680 0x7f2ad806db40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.148+0000 7f2af16b6640 1 --2- 172.21.15.98:0/458654264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2aec1f3a60 0x7f2aec0c67c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.148+0000 7f2af16b6640 1 --2- 172.21.15.98:0/458654264 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2aec0b46d0 0x7f2aec0c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.148+0000 7f2af16b6640 1 --2- 172.21.15.98:0/458654264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ad4004800 0x7f2aec0ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.148+0000 7f2af16b6640 1 -- 172.21.15.98:0/458654264 >> 172.21.15.98:0/458654264 conn(0x7f2aec0b34e0 msgr2=0x7f2aec15d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:40.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.148+0000 7f2af16b6640 1 -- 172.21.15.98:0/458654264 shutdown_connections 2024-05-09T10:51:40.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.148+0000 7f2af16b6640 1 -- 172.21.15.98:0/458654264 wait complete. 2024-05-09T10:51:40.153 INFO:tasks.ceph:Scrubbing osd.3 2024-05-09T10:51:40.153 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-09T10:51:40.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.264+0000 7f4306542640 1 Processor -- start 2024-05-09T10:51:40.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.264+0000 7f4306542640 1 -- start start 2024-05-09T10:51:40.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.264+0000 7f4306542640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43000b1d90 0x7f43000cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.264+0000 7f4306542640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f43000b2170 0x7f43000d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.264+0000 7f4306542640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43000cc550 con 0x7f43000b2660 2024-05-09T10:51:40.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.264+0000 7f4306542640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43000cfa70 con 0x7f43000b2170 2024-05-09T10:51:40.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.264+0000 7f4306542640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43000cea90 con 0x7f43000b1d90 2024-05-09T10:51:40.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.265+0000 7f42f7fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f43000b2170 0x7f43000d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:40.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.265+0000 7f4304adf640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f43000b2660 0x7f430015fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52804/0 (socket says 172.21.15.98:52804) 2024-05-09T10:51:40.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.265+0000 7f4304adf640 1 -- 172.21.15.98:0/3252044733 learned_addr learned my addr 172.21.15.98:0/3252044733 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:40.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.265+0000 7f42f7fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f43000b2170 0x7f43000d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54592/0 (socket says 172.21.15.98:54592) 2024-05-09T10:51:40.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.265+0000 7f42fffff640 1 --2- 172.21.15.98:0/3252044733 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43000b1d90 0x7f43000cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:40.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.265+0000 7f42f7fff640 1 -- 172.21.15.98:0/3252044733 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43000b1d90 msgr2=0x7f43000cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:40.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.265+0000 7f42f7fff640 1 --2- 172.21.15.98:0/3252044733 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43000b1d90 0x7f43000cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.265+0000 7f42f7fff640 1 -- 172.21.15.98:0/3252044733 >> v1:172.21.15.98:6789/0 conn(0x7f43000b2660 legacy=0x7f430015fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.265+0000 7f42f7fff640 1 -- 172.21.15.98:0/3252044733 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43000ced70 con 0x7f43000b2170 2024-05-09T10:51:40.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.265+0000 7f42ff7fe640 1 -- 172.21.15.98:0/3252044733 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963325553 0 0) 0x7f42f8001490 con 0x7f43000b2660 2024-05-09T10:51:40.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.266+0000 7f42f7fff640 1 --2- 172.21.15.98:0/3252044733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43000b2170 0x7f43000d0460 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f42e80097a0 tx=0x7f42e802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584677f613c2f041 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:40.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.266+0000 7f42ff7fe640 1 -- 172.21.15.98:0/3252044733 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e8037070 con 0x7f43000b2170 2024-05-09T10:51:40.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.266+0000 7f42ff7fe640 1 -- 172.21.15.98:0/3252044733 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e8032440 con 0x7f43000b2170 2024-05-09T10:51:40.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.266+0000 7f42ff7fe640 1 -- 172.21.15.98:0/3252044733 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e8032810 con 0x7f43000b2170 2024-05-09T10:51:40.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.266+0000 7f4306542640 1 -- 172.21.15.98:0/3252044733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43000b2170 msgr2=0x7f43000d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.266+0000 7f4306542640 1 --2- 172.21.15.98:0/3252044733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43000b2170 0x7f43000d0460 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f42e80097a0 tx=0x7f42e802a180 comp rx=0 tx=0).stop 2024-05-09T10:51:40.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.266+0000 7f4306542640 1 -- 172.21.15.98:0/3252044733 shutdown_connections 2024-05-09T10:51:40.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.266+0000 7f4306542640 1 --2- 172.21.15.98:0/3252044733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43000b2170 0x7f43000d0460 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.266+0000 7f4306542640 1 --2- 172.21.15.98:0/3252044733 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43000b1d90 0x7f43000cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.266+0000 7f4306542640 1 -- 172.21.15.98:0/3252044733 >> 172.21.15.98:0/3252044733 conn(0x7f43000b3450 msgr2=0x7f43000b3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:40.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.266+0000 7f4306542640 1 -- 172.21.15.98:0/3252044733 shutdown_connections 2024-05-09T10:51:40.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.266+0000 7f4306542640 1 -- 172.21.15.98:0/3252044733 wait complete. 2024-05-09T10:51:40.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.268+0000 7f4306542640 1 Processor -- start 2024-05-09T10:51:40.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.268+0000 7f4306542640 1 -- start start 2024-05-09T10:51:40.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.268+0000 7f4306542640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43000b1d90 0x7f43001541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.268+0000 7f4306542640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43000b2170 0x7f43001f5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.268+0000 7f4306542640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43000b2660 0x7f43001f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.268+0000 7f4306542640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43000b4790 con 0x7f43000b2170 2024-05-09T10:51:40.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.268+0000 7f4306542640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43000cf720 con 0x7f43000b2660 2024-05-09T10:51:40.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.268+0000 7f4306542640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43000b4910 con 0x7f43000b1d90 2024-05-09T10:51:40.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.268+0000 7f42f7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43000b2170 0x7f43001f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:40.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.268+0000 7f42f7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43000b2170 0x7f43001f5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54598/0 (socket says 172.21.15.98:54598) 2024-05-09T10:51:40.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.268+0000 7f42f7fff640 1 -- 172.21.15.98:0/4159774552 learned_addr learned my addr 172.21.15.98:0/4159774552 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:40.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.269+0000 7f42f7fff640 1 -- 172.21.15.98:0/4159774552 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43000b1d90 msgr2=0x7f43001541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.269+0000 7f42f7fff640 1 --2- 172.21.15.98:0/4159774552 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f43000b1d90 0x7f43001541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.269+0000 7f42f7fff640 1 -- 172.21.15.98:0/4159774552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43000b2660 msgr2=0x7f43001f7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.269+0000 7f42f7fff640 1 --2- 172.21.15.98:0/4159774552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43000b2660 0x7f43001f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.269+0000 7f42f7fff640 1 -- 172.21.15.98:0/4159774552 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43001f5fd0 con 0x7f43000b2170 2024-05-09T10:51:40.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.269+0000 7f42f7fff640 1 --2- 172.21.15.98:0/4159774552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43000b2170 0x7f43001f5660 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f42e8035040 tx=0x7f42e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:40.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.269+0000 7f42fd7fa640 1 -- 172.21.15.98:0/4159774552 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e8037040 con 0x7f43000b2170 2024-05-09T10:51:40.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.269+0000 7f42fd7fa640 1 -- 172.21.15.98:0/4159774552 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e8030500 con 0x7f43000b2170 2024-05-09T10:51:40.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.269+0000 7f4306542640 1 -- 172.21.15.98:0/4159774552 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43001f62b0 con 0x7f43000b2170 2024-05-09T10:51:40.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.269+0000 7f42fd7fa640 1 -- 172.21.15.98:0/4159774552 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e8030910 con 0x7f43000b2170 2024-05-09T10:51:40.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.269+0000 7f4306542640 1 -- 172.21.15.98:0/4159774552 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43001f6870 con 0x7f43000b2170 2024-05-09T10:51:40.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.270+0000 7f4306542640 1 -- 172.21.15.98:0/4159774552 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f430009fab0 con 0x7f43000b2170 2024-05-09T10:51:40.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.270+0000 7f42fd7fa640 1 -- 172.21.15.98:0/4159774552 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f42e8044020 con 0x7f43000b2170 2024-05-09T10:51:40.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.271+0000 7f42fd7fa640 1 --2- 172.21.15.98:0/4159774552 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f42e006b6b0 0x7f42e006db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.271+0000 7f42fd7fa640 1 -- 172.21.15.98:0/4159774552 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f42e80aa4e0 con 0x7f43000b2170 2024-05-09T10:51:40.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.271+0000 7f42fffff640 1 --2- 172.21.15.98:0/4159774552 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f42e006b6b0 0x7f42e006db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:40.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.271+0000 7f42fd7fa640 1 --2- 172.21.15.98:0/4159774552 >> [v2:172.21.15.98:6808/2633494785,v1:172.21.15.98:6809/2633494785] conn(0x7f42e0071570 0x7f42e0073990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.271+0000 7f42fd7fa640 1 -- 172.21.15.98:0/4159774552 --> [v2:172.21.15.98:6808/2633494785,v1:172.21.15.98:6809/2633494785] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f42e8032440 con 0x7f42e0071570 2024-05-09T10:51:40.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.271+0000 7f42fd7fa640 1 -- 172.21.15.98:0/4159774552 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42e80b1050 con 0x7f43000b2170 2024-05-09T10:51:40.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.271+0000 7f4304adf640 1 --2- 172.21.15.98:0/4159774552 >> [v2:172.21.15.98:6808/2633494785,v1:172.21.15.98:6809/2633494785] conn(0x7f42e0071570 0x7f42e0073990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:40.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.271+0000 7f42fffff640 1 --2- 172.21.15.98:0/4159774552 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f42e006b6b0 0x7f42e006db70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f42f0006a90 tx=0x7f42f0006a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:40.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.271+0000 7f4304adf640 1 --2- 172.21.15.98:0/4159774552 >> [v2:172.21.15.98:6808/2633494785,v1:172.21.15.98:6809/2633494785] conn(0x7f42e0071570 0x7f42e0073990 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:40.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.272+0000 7f42fd7fa640 1 -- 172.21.15.98:0/4159774552 <== osd.3 v2:172.21.15.98:6808/2633494785 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f42e8032440 con 0x7f42e0071570 2024-05-09T10:51:40.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.285+0000 7f4306542640 1 -- 172.21.15.98:0/4159774552 --> [v2:172.21.15.98:6808/2633494785,v1:172.21.15.98:6809/2633494785] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f43000b39a0 con 0x7f42e0071570 2024-05-09T10:51:40.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.285+0000 7f42fd7fa640 1 -- 172.21.15.98:0/4159774552 <== osd.3 v2:172.21.15.98:6808/2633494785 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f43000b39a0 con 0x7f42e0071570 2024-05-09T10:51:40.278 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-05-09T10:51:40.279 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T10:51:40.279 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-05-09T10:51:40.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.285+0000 7f4306542640 1 -- 172.21.15.98:0/4159774552 >> [v2:172.21.15.98:6808/2633494785,v1:172.21.15.98:6809/2633494785] conn(0x7f42e0071570 msgr2=0x7f42e0073990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.285+0000 7f4306542640 1 --2- 172.21.15.98:0/4159774552 >> [v2:172.21.15.98:6808/2633494785,v1:172.21.15.98:6809/2633494785] conn(0x7f42e0071570 0x7f42e0073990 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.285+0000 7f4306542640 1 -- 172.21.15.98:0/4159774552 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f42e006b6b0 msgr2=0x7f42e006db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.285+0000 7f4306542640 1 --2- 172.21.15.98:0/4159774552 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f42e006b6b0 0x7f42e006db70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f42f0006a90 tx=0x7f42f0006a20 comp rx=0 tx=0).stop 2024-05-09T10:51:40.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.285+0000 7f4306542640 1 -- 172.21.15.98:0/4159774552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43000b2170 msgr2=0x7f43001f5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.285+0000 7f4306542640 1 --2- 172.21.15.98:0/4159774552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43000b2170 0x7f43001f5660 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f42e8035040 tx=0x7f42e80046c0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.286+0000 7f4304adf640 1 -- 172.21.15.98:0/4159774552 reap_dead start 2024-05-09T10:51:40.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.286+0000 7f4306542640 1 -- 172.21.15.98:0/4159774552 shutdown_connections 2024-05-09T10:51:40.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.286+0000 7f4306542640 1 -- 172.21.15.98:0/4159774552 >> 172.21.15.98:0/4159774552 conn(0x7f43000b3450 msgr2=0x7f430015bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:40.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.286+0000 7f4306542640 1 -- 172.21.15.98:0/4159774552 shutdown_connections 2024-05-09T10:51:40.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.286+0000 7f4306542640 1 -- 172.21.15.98:0/4159774552 wait complete. 2024-05-09T10:51:40.289 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-09T10:51:40.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.397+0000 7f68964ee640 1 Processor -- start 2024-05-09T10:51:40.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.397+0000 7f68964ee640 1 -- start start 2024-05-09T10:51:40.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.398+0000 7f68964ee640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f689015dce0 0x7f689006bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.398+0000 7f68964ee640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f689015e0c0 0x7f689006c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.398+0000 7f68964ee640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68900d0c70 con 0x7f68900d0700 2024-05-09T10:51:40.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.398+0000 7f68964ee640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68900b4350 con 0x7f689015dce0 2024-05-09T10:51:40.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.398+0000 7f68964ee640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68900d0f50 con 0x7f689015e0c0 2024-05-09T10:51:40.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.398+0000 7f688ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f689015dce0 0x7f689006bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:40.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.398+0000 7f688ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f689015dce0 0x7f689006bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54604/0 (socket says 172.21.15.98:54604) 2024-05-09T10:51:40.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.398+0000 7f688ffff640 1 -- 172.21.15.98:0/4093633666 learned_addr learned my addr 172.21.15.98:0/4093633666 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:40.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.398+0000 7f688f7fe640 1 --2- 172.21.15.98:0/4093633666 >> v2:172.21.15.105:3300/0 conn(0x7f689015e0c0 0x7f689006c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:40.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.398+0000 7f688ffff640 1 -- 172.21.15.98:0/4093633666 >> v2:172.21.15.105:3300/0 conn(0x7f689015e0c0 msgr2=0x7f689006c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:40.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.398+0000 7f688ffff640 1 --2- 172.21.15.98:0/4093633666 >> v2:172.21.15.105:3300/0 conn(0x7f689015e0c0 0x7f689006c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.398+0000 7f688ffff640 1 -- 172.21.15.98:0/4093633666 >> v1:172.21.15.105:6789/0 conn(0x7f68900d0700 legacy=0x7f6890161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:40.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.398+0000 7f688ffff640 1 -- 172.21.15.98:0/4093633666 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68900b0fd0 con 0x7f689015dce0 2024-05-09T10:51:40.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.399+0000 7f688ffff640 1 --2- 172.21.15.98:0/4093633666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f689015dce0 0x7f689006bcf0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f68900717d0 tx=0x7f688002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6965edfaa03fae44 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:40.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.399+0000 7f688effd640 1 -- 172.21.15.98:0/4093633666 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880037070 con 0x7f689015dce0 2024-05-09T10:51:40.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.399+0000 7f688effd640 1 -- 172.21.15.98:0/4093633666 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6880032440 con 0x7f689015dce0 2024-05-09T10:51:40.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.399+0000 7f688effd640 1 -- 172.21.15.98:0/4093633666 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880032810 con 0x7f689015dce0 2024-05-09T10:51:40.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.400+0000 7f68964ee640 1 -- 172.21.15.98:0/4093633666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f689015dce0 msgr2=0x7f689006bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.400+0000 7f68964ee640 1 --2- 172.21.15.98:0/4093633666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f689015dce0 0x7f689006bcf0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f68900717d0 tx=0x7f688002a2c0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.400+0000 7f68964ee640 1 -- 172.21.15.98:0/4093633666 shutdown_connections 2024-05-09T10:51:40.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.400+0000 7f68964ee640 1 --2- 172.21.15.98:0/4093633666 >> v2:172.21.15.105:3300/0 conn(0x7f689015e0c0 0x7f689006c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.400+0000 7f68964ee640 1 --2- 172.21.15.98:0/4093633666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f689015dce0 0x7f689006bcf0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.400+0000 7f68964ee640 1 -- 172.21.15.98:0/4093633666 >> 172.21.15.98:0/4093633666 conn(0x7f68900b34e0 msgr2=0x7f68900b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:40.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.400+0000 7f68964ee640 1 -- 172.21.15.98:0/4093633666 shutdown_connections 2024-05-09T10:51:40.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.400+0000 7f68964ee640 1 -- 172.21.15.98:0/4093633666 wait complete. 2024-05-09T10:51:40.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.401+0000 7f68964ee640 1 Processor -- start 2024-05-09T10:51:40.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.401+0000 7f68964ee640 1 -- start start 2024-05-09T10:51:40.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.402+0000 7f68964ee640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68900d0700 0x7f689015c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.402+0000 7f68964ee640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f689015dce0 0x7f68900c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.402+0000 7f68964ee640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f689015e0c0 0x7f68901f7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.402+0000 7f68964ee640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68900b2ac0 con 0x7f689015e0c0 2024-05-09T10:51:40.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.402+0000 7f68964ee640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68900b1590 con 0x7f68900d0700 2024-05-09T10:51:40.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.402+0000 7f68964ee640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68900b2c40 con 0x7f689015dce0 2024-05-09T10:51:40.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.402+0000 7f6894a8b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f689015e0c0 0x7f68901f7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:40.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.402+0000 7f6894a8b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f689015e0c0 0x7f68901f7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54606/0 (socket says 172.21.15.98:54606) 2024-05-09T10:51:40.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.402+0000 7f6894a8b640 1 -- 172.21.15.98:0/4036300896 learned_addr learned my addr 172.21.15.98:0/4036300896 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:40.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.402+0000 7f6894a8b640 1 -- 172.21.15.98:0/4036300896 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f689015dce0 msgr2=0x7f68900c7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.402+0000 7f688ffff640 1 --2- 172.21.15.98:0/4036300896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68900d0700 0x7f689015c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:40.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.402+0000 7f688f7fe640 1 --2- 172.21.15.98:0/4036300896 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f689015dce0 0x7f68900c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:40.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.402+0000 7f6894a8b640 1 --2- 172.21.15.98:0/4036300896 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f689015dce0 0x7f68900c7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.402+0000 7f6894a8b640 1 -- 172.21.15.98:0/4036300896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68900d0700 msgr2=0x7f689015c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.402+0000 7f6894a8b640 1 --2- 172.21.15.98:0/4036300896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68900d0700 0x7f689015c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.402+0000 7f6894a8b640 1 -- 172.21.15.98:0/4036300896 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68900c5730 con 0x7f689015e0c0 2024-05-09T10:51:40.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.403+0000 7f6894a8b640 1 --2- 172.21.15.98:0/4036300896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f689015e0c0 0x7f68901f7700 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f688800c780 tx=0x7f688800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:40.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.403+0000 7f688cff9640 1 -- 172.21.15.98:0/4036300896 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888004b60 con 0x7f689015e0c0 2024-05-09T10:51:40.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.403+0000 7f688cff9640 1 -- 172.21.15.98:0/4036300896 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6888004d00 con 0x7f689015e0c0 2024-05-09T10:51:40.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.403+0000 7f688cff9640 1 -- 172.21.15.98:0/4036300896 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688800aa90 con 0x7f689015e0c0 2024-05-09T10:51:40.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.403+0000 7f68964ee640 1 -- 172.21.15.98:0/4036300896 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68900c5a10 con 0x7f689015e0c0 2024-05-09T10:51:40.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.403+0000 7f68964ee640 1 -- 172.21.15.98:0/4036300896 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68900c6d40 con 0x7f689015e0c0 2024-05-09T10:51:40.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.404+0000 7f68964ee640 1 -- 172.21.15.98:0/4036300896 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68900c6860 con 0x7f689015e0c0 2024-05-09T10:51:40.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.404+0000 7f688cff9640 1 -- 172.21.15.98:0/4036300896 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f688800ac30 con 0x7f689015e0c0 2024-05-09T10:51:40.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.405+0000 7f688cff9640 1 --2- 172.21.15.98:0/4036300896 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f687c06b680 0x7f687c06db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.405+0000 7f688cff9640 1 -- 172.21.15.98:0/4036300896 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f688808a560 con 0x7f689015e0c0 2024-05-09T10:51:40.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.405+0000 7f688ffff640 1 --2- 172.21.15.98:0/4036300896 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f687c06b680 0x7f687c06db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:40.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.405+0000 7f688ffff640 1 --2- 172.21.15.98:0/4036300896 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f687c06b680 0x7f687c06db40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6880034000 tx=0x7f688002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:40.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.409+0000 7f688cff9640 1 -- 172.21.15.98:0/4036300896 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6888020070 con 0x7f689015e0c0 2024-05-09T10:51:40.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.582+0000 7f68964ee640 1 -- 172.21.15.98:0/4036300896 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f6890069cb0 con 0x7f687c06b680 2024-05-09T10:51:40.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.582+0000 7f688cff9640 1 -- 172.21.15.98:0/4036300896 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6890069cb0 con 0x7f687c06b680 2024-05-09T10:51:40.575 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 3 to deep-scrub 2024-05-09T10:51:40.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.584+0000 7f68964ee640 1 -- 172.21.15.98:0/4036300896 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f687c06b680 msgr2=0x7f687c06db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.584+0000 7f68964ee640 1 --2- 172.21.15.98:0/4036300896 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f687c06b680 0x7f687c06db40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6880034000 tx=0x7f688002e000 comp rx=0 tx=0).stop 2024-05-09T10:51:40.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.584+0000 7f68964ee640 1 -- 172.21.15.98:0/4036300896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f689015e0c0 msgr2=0x7f68901f7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.584+0000 7f68964ee640 1 --2- 172.21.15.98:0/4036300896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f689015e0c0 0x7f68901f7700 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f688800c780 tx=0x7f688800cc40 comp rx=0 tx=0).stop 2024-05-09T10:51:40.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.584+0000 7f68964ee640 1 -- 172.21.15.98:0/4036300896 shutdown_connections 2024-05-09T10:51:40.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.584+0000 7f68964ee640 1 --2- 172.21.15.98:0/4036300896 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f687c06b680 0x7f687c06db40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.584+0000 7f68964ee640 1 --2- 172.21.15.98:0/4036300896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f689015e0c0 0x7f68901f7700 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.584+0000 7f68964ee640 1 --2- 172.21.15.98:0/4036300896 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f689015dce0 0x7f68900c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.584+0000 7f68964ee640 1 --2- 172.21.15.98:0/4036300896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68900d0700 0x7f689015c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.584+0000 7f68964ee640 1 -- 172.21.15.98:0/4036300896 >> 172.21.15.98:0/4036300896 conn(0x7f68900b34e0 msgr2=0x7f6890159ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:40.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.584+0000 7f68964ee640 1 -- 172.21.15.98:0/4036300896 shutdown_connections 2024-05-09T10:51:40.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.584+0000 7f68964ee640 1 -- 172.21.15.98:0/4036300896 wait complete. 2024-05-09T10:51:40.588 INFO:tasks.ceph:Scrubbing osd.4 2024-05-09T10:51:40.588 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-09T10:51:40.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.698+0000 7f3f359b0640 1 Processor -- start 2024-05-09T10:51:40.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.699+0000 7f3f359b0640 1 -- start start 2024-05-09T10:51:40.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.699+0000 7f3f359b0640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3f300b20e0 0x7f3f300ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.699+0000 7f3f359b0640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f300d1710 con 0x7f3f300cc060 2024-05-09T10:51:40.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.699+0000 7f3f359b0640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f300b2e70 con 0x7f3f300b2780 2024-05-09T10:51:40.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.699+0000 7f3f359b0640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f300d05e0 con 0x7f3f300b20e0 2024-05-09T10:51:40.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.699+0000 7f3f2f7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f3f300cc060 0x7f3f300cc4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52810/0 (socket says 172.21.15.98:52810) 2024-05-09T10:51:40.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2f7fe640 1 -- 172.21.15.98:0/802349179 learned_addr learned my addr 172.21.15.98:0/802349179 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:40.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2effd640 1 --2- 172.21.15.98:0/802349179 >> v2:172.21.15.105:3300/0 conn(0x7f3f300b20e0 0x7f3f300ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:40.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2e7fc640 1 -- 172.21.15.98:0/802349179 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232961002 0 0) 0x7f3f300d1710 con 0x7f3f300cc060 2024-05-09T10:51:40.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2e7fc640 1 -- 172.21.15.98:0/802349179 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f1c003520 con 0x7f3f300cc060 2024-05-09T10:51:40.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2e7fc640 1 -- 172.21.15.98:0/802349179 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4046282727 0 0) 0x7f3f1c003520 con 0x7f3f300cc060 2024-05-09T10:51:40.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2e7fc640 1 -- 172.21.15.98:0/802349179 >> v2:172.21.15.105:3300/0 conn(0x7f3f300b20e0 msgr2=0x7f3f300ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:40.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2e7fc640 1 --2- 172.21.15.98:0/802349179 >> v2:172.21.15.105:3300/0 conn(0x7f3f300b20e0 0x7f3f300ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2e7fc640 1 -- 172.21.15.98:0/802349179 >> v1:172.21.15.105:6789/0 conn(0x7f3f300b2780 legacy=0x7f3f300ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2e7fc640 1 -- 172.21.15.98:0/802349179 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f300d08c0 con 0x7f3f300cc060 2024-05-09T10:51:40.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2effd640 1 --2- 172.21.15.98:0/802349179 >> v2:172.21.15.105:3300/0 conn(0x7f3f300b20e0 0x7f3f300ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T10:51:40.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2e7fc640 1 -- 172.21.15.98:0/802349179 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f3f280019f0 con 0x7f3f300cc060 2024-05-09T10:51:40.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2e7fc640 1 -- 172.21.15.98:0/802349179 >> v1:172.21.15.98:6789/0 conn(0x7f3f300cc060 legacy=0x7f3f300cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2e7fc640 1 --2- 172.21.15.98:0/802349179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f1c0039b0 0x7f3f1c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2e7fc640 1 --2- 172.21.15.98:0/802349179 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f1c004800 0x7f3f1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2e7fc640 1 --2- 172.21.15.98:0/802349179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f1c0051c0 0x7f3f1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2e7fc640 1 -- 172.21.15.98:0/802349179 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f300d05e0 con 0x7f3f1c0039b0 2024-05-09T10:51:40.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2e7fc640 1 -- 172.21.15.98:0/802349179 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c00c060 con 0x7f3f1c0051c0 2024-05-09T10:51:40.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2e7fc640 1 -- 172.21.15.98:0/802349179 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c00c260 con 0x7f3f1c004800 2024-05-09T10:51:40.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2effd640 1 --2- 172.21.15.98:0/802349179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f1c0039b0 0x7f3f1c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:40.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f267fc640 1 --2- 172.21.15.98:0/802349179 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f1c004800 0x7f3f1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:40.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2f7fe640 1 --2- 172.21.15.98:0/802349179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f1c0051c0 0x7f3f1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:40.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2effd640 1 -- 172.21.15.98:0/802349179 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f1c004800 msgr2=0x7f3f1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:40.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2effd640 1 --2- 172.21.15.98:0/802349179 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f1c004800 0x7f3f1c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2effd640 1 -- 172.21.15.98:0/802349179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f1c0051c0 msgr2=0x7f3f1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:40.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2effd640 1 --2- 172.21.15.98:0/802349179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f1c0051c0 0x7f3f1c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2effd640 1 -- 172.21.15.98:0/802349179 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c003520 con 0x7f3f1c0039b0 2024-05-09T10:51:40.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2f7fe640 1 --2- 172.21.15.98:0/802349179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f1c0051c0 0x7f3f1c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:51:40.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.700+0000 7f3f2f7fe640 1 -- 172.21.15.98:0/802349179 reap_dead start 2024-05-09T10:51:40.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.701+0000 7f3f2effd640 1 --2- 172.21.15.98:0/802349179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f1c0039b0 0x7f3f1c003dd0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009c90 tx=0x7f3f1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de57eebf449a766a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:40.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.701+0000 7f3f2e7fc640 1 -- 172.21.15.98:0/802349179 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f20036070 con 0x7f3f1c0039b0 2024-05-09T10:51:40.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.701+0000 7f3f2e7fc640 1 -- 172.21.15.98:0/802349179 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f20031400 con 0x7f3f1c0039b0 2024-05-09T10:51:40.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.701+0000 7f3f2e7fc640 1 -- 172.21.15.98:0/802349179 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f20009860 con 0x7f3f1c0039b0 2024-05-09T10:51:40.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.702+0000 7f3f359b0640 1 -- 172.21.15.98:0/802349179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f1c0039b0 msgr2=0x7f3f1c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.702+0000 7f3f359b0640 1 --2- 172.21.15.98:0/802349179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f1c0039b0 0x7f3f1c003dd0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009c90 tx=0x7f3f1c005830 comp rx=0 tx=0).stop 2024-05-09T10:51:40.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.702+0000 7f3f359b0640 1 -- 172.21.15.98:0/802349179 shutdown_connections 2024-05-09T10:51:40.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.702+0000 7f3f359b0640 1 --2- 172.21.15.98:0/802349179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f1c0039b0 0x7f3f1c003dd0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.702+0000 7f3f359b0640 1 -- 172.21.15.98:0/802349179 >> 172.21.15.98:0/802349179 conn(0x7f3f300b34e0 msgr2=0x7f3f300b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:40.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.702+0000 7f3f359b0640 1 -- 172.21.15.98:0/802349179 shutdown_connections 2024-05-09T10:51:40.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.702+0000 7f3f359b0640 1 -- 172.21.15.98:0/802349179 wait complete. 2024-05-09T10:51:40.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.703+0000 7f3f359b0640 1 Processor -- start 2024-05-09T10:51:40.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.703+0000 7f3f359b0640 1 -- start start 2024-05-09T10:51:40.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.704+0000 7f3f359b0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f1c0039b0 0x7f3f300c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.704+0000 7f3f359b0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f300c5d20 0x7f3f30153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.704+0000 7f3f359b0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f300ae7c0 0x7f3f3006d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.704+0000 7f3f359b0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f300ce190 con 0x7f3f300c5d20 2024-05-09T10:51:40.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.704+0000 7f3f359b0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f300b3150 con 0x7f3f300ae7c0 2024-05-09T10:51:40.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.704+0000 7f3f359b0640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f300ce310 con 0x7f3f1c0039b0 2024-05-09T10:51:40.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.704+0000 7f3f267fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f300c5d20 0x7f3f30153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:40.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.704+0000 7f3f267fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f300c5d20 0x7f3f30153b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54630/0 (socket says 172.21.15.98:54630) 2024-05-09T10:51:40.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.704+0000 7f3f267fc640 1 -- 172.21.15.98:0/810854299 learned_addr learned my addr 172.21.15.98:0/810854299 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:40.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.704+0000 7f3f267fc640 1 -- 172.21.15.98:0/810854299 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f1c0039b0 msgr2=0x7f3f300c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.704+0000 7f3f267fc640 1 --2- 172.21.15.98:0/810854299 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3f1c0039b0 0x7f3f300c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.704+0000 7f3f267fc640 1 -- 172.21.15.98:0/810854299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f300ae7c0 msgr2=0x7f3f3006d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.704+0000 7f3f267fc640 1 --2- 172.21.15.98:0/810854299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f300ae7c0 0x7f3f3006d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.704+0000 7f3f267fc640 1 -- 172.21.15.98:0/810854299 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f300c61b0 con 0x7f3f300c5d20 2024-05-09T10:51:40.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.705+0000 7f3f267fc640 1 --2- 172.21.15.98:0/810854299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f300c5d20 0x7f3f30153b10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800e4d0 tx=0x7f3f1800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:40.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.705+0000 7f3f2cff9640 1 -- 172.21.15.98:0/810854299 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f18019070 con 0x7f3f300c5d20 2024-05-09T10:51:40.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.705+0000 7f3f2cff9640 1 -- 172.21.15.98:0/810854299 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f180053a0 con 0x7f3f300c5d20 2024-05-09T10:51:40.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.705+0000 7f3f2cff9640 1 -- 172.21.15.98:0/810854299 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f18003810 con 0x7f3f300c5d20 2024-05-09T10:51:40.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.705+0000 7f3f359b0640 1 -- 172.21.15.98:0/810854299 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f3015f820 con 0x7f3f300c5d20 2024-05-09T10:51:40.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.705+0000 7f3f359b0640 1 -- 172.21.15.98:0/810854299 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3015fde0 con 0x7f3f300c5d20 2024-05-09T10:51:40.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.706+0000 7f3f359b0640 1 -- 172.21.15.98:0/810854299 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3f3009fb40 con 0x7f3f300c5d20 2024-05-09T10:51:40.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.707+0000 7f3f2cff9640 1 -- 172.21.15.98:0/810854299 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f3f1801c070 con 0x7f3f300c5d20 2024-05-09T10:51:40.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.707+0000 7f3f2cff9640 1 --2- 172.21.15.98:0/810854299 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3f1006b680 0x7f3f1006db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.707+0000 7f3f2cff9640 1 -- 172.21.15.98:0/810854299 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3f1808a2c0 con 0x7f3f300c5d20 2024-05-09T10:51:40.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.707+0000 7f3f2effd640 1 --2- 172.21.15.98:0/810854299 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3f1006b680 0x7f3f1006db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:40.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.707+0000 7f3f2cff9640 1 --2- 172.21.15.98:0/810854299 >> [v2:172.21.15.105:6808/3698267721,v1:172.21.15.105:6809/3698267721] conn(0x7f3f10071540 0x7f3f10073960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.707+0000 7f3f2cff9640 1 -- 172.21.15.98:0/810854299 --> [v2:172.21.15.105:6808/3698267721,v1:172.21.15.105:6809/3698267721] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3f18009e50 con 0x7f3f10071540 2024-05-09T10:51:40.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.707+0000 7f3f2cff9640 1 -- 172.21.15.98:0/810854299 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f1808a750 con 0x7f3f300c5d20 2024-05-09T10:51:40.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.707+0000 7f3f2f7fe640 1 --2- 172.21.15.98:0/810854299 >> [v2:172.21.15.105:6808/3698267721,v1:172.21.15.105:6809/3698267721] conn(0x7f3f10071540 0x7f3f10073960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:40.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.707+0000 7f3f2effd640 1 --2- 172.21.15.98:0/810854299 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3f1006b680 0x7f3f1006db40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3f20033000 tx=0x7f3f2002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:40.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.708+0000 7f3f2f7fe640 1 --2- 172.21.15.98:0/810854299 >> [v2:172.21.15.105:6808/3698267721,v1:172.21.15.105:6809/3698267721] conn(0x7f3f10071540 0x7f3f10073960 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:40.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.709+0000 7f3f2cff9640 1 -- 172.21.15.98:0/810854299 <== osd.4 v2:172.21.15.105:6808/3698267721 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3f18009e50 con 0x7f3f10071540 2024-05-09T10:51:40.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.721+0000 7f3f359b0640 1 -- 172.21.15.98:0/810854299 --> [v2:172.21.15.105:6808/3698267721,v1:172.21.15.105:6809/3698267721] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3f3004f000 con 0x7f3f10071540 2024-05-09T10:51:40.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.722+0000 7f3f2cff9640 1 -- 172.21.15.98:0/810854299 <== osd.4 v2:172.21.15.105:6808/3698267721 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f3f3004f000 con 0x7f3f10071540 2024-05-09T10:51:40.715 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-05-09T10:51:40.715 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T10:51:40.715 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-05-09T10:51:40.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.722+0000 7f3f359b0640 1 -- 172.21.15.98:0/810854299 >> [v2:172.21.15.105:6808/3698267721,v1:172.21.15.105:6809/3698267721] conn(0x7f3f10071540 msgr2=0x7f3f10073960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.722+0000 7f3f359b0640 1 --2- 172.21.15.98:0/810854299 >> [v2:172.21.15.105:6808/3698267721,v1:172.21.15.105:6809/3698267721] conn(0x7f3f10071540 0x7f3f10073960 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.722+0000 7f3f359b0640 1 -- 172.21.15.98:0/810854299 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3f1006b680 msgr2=0x7f3f1006db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.722+0000 7f3f359b0640 1 --2- 172.21.15.98:0/810854299 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3f1006b680 0x7f3f1006db40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3f20033000 tx=0x7f3f2002e000 comp rx=0 tx=0).stop 2024-05-09T10:51:40.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.722+0000 7f3f359b0640 1 -- 172.21.15.98:0/810854299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f300c5d20 msgr2=0x7f3f30153b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.722+0000 7f3f359b0640 1 --2- 172.21.15.98:0/810854299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f300c5d20 0x7f3f30153b10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800e4d0 tx=0x7f3f1800e990 comp rx=0 tx=0).stop 2024-05-09T10:51:40.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.722+0000 7f3f2f7fe640 1 -- 172.21.15.98:0/810854299 reap_dead start 2024-05-09T10:51:40.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.722+0000 7f3f359b0640 1 -- 172.21.15.98:0/810854299 shutdown_connections 2024-05-09T10:51:40.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.722+0000 7f3f359b0640 1 -- 172.21.15.98:0/810854299 >> 172.21.15.98:0/810854299 conn(0x7f3f300b34e0 msgr2=0x7f3f3006e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:40.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.722+0000 7f3f359b0640 1 -- 172.21.15.98:0/810854299 shutdown_connections 2024-05-09T10:51:40.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.722+0000 7f3f359b0640 1 -- 172.21.15.98:0/810854299 wait complete. 2024-05-09T10:51:40.726 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-09T10:51:40.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.833+0000 7fdcc4007640 1 Processor -- start 2024-05-09T10:51:40.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.834+0000 7fdcc4007640 1 -- start start 2024-05-09T10:51:40.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.834+0000 7fdcc4007640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdcbc155000 0x7fdcbc0713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.834+0000 7fdcc4007640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdcbc071930 0x7fdcbc06e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.834+0000 7fdcc4007640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdcbc06eb70 0x7fdcbc0af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.834+0000 7fdcc4007640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcbc06f000 con 0x7fdcbc071930 2024-05-09T10:51:40.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.834+0000 7fdcc4007640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcbc071d90 con 0x7fdcbc155000 2024-05-09T10:51:40.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.834+0000 7fdcc4007640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc0b4430 con 0x7fdcbc06eb70 2024-05-09T10:51:40.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.834+0000 7fdcc15a2640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdcbc071930 0x7fdcbc06e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:40.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.835+0000 7fdcc15a2640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdcbc071930 0x7fdcbc06e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54632/0 (socket says 172.21.15.98:54632) 2024-05-09T10:51:40.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.835+0000 7fdcc15a2640 1 -- 172.21.15.98:0/2018607097 learned_addr learned my addr 172.21.15.98:0/2018607097 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:40.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.835+0000 7fdcc1da3640 1 --2- 172.21.15.98:0/2018607097 >> v2:172.21.15.105:3300/0 conn(0x7fdcbc155000 0x7fdcbc0713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:40.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.835+0000 7fdcc25a4640 1 --2- 172.21.15.98:0/2018607097 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdcbc06eb70 0x7fdcbc0af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:40.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.835+0000 7fdcc15a2640 1 -- 172.21.15.98:0/2018607097 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdcbc06eb70 msgr2=0x7fdcbc0af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:40.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.835+0000 7fdcc15a2640 1 --2- 172.21.15.98:0/2018607097 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdcbc06eb70 0x7fdcbc0af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.835+0000 7fdcc15a2640 1 -- 172.21.15.98:0/2018607097 >> v2:172.21.15.105:3300/0 conn(0x7fdcbc155000 msgr2=0x7fdcbc0713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:40.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.835+0000 7fdcc15a2640 1 --2- 172.21.15.98:0/2018607097 >> v2:172.21.15.105:3300/0 conn(0x7fdcbc155000 0x7fdcbc0713f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.835+0000 7fdcc15a2640 1 -- 172.21.15.98:0/2018607097 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcbc1553e0 con 0x7fdcbc071930 2024-05-09T10:51:40.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.836+0000 7fdcc15a2640 1 --2- 172.21.15.98:0/2018607097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdcbc071930 0x7fdcbc06e630 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0b3440 tx=0x7fdcb002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f8298256fb2601 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:40.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.836+0000 7fdcc0da1640 1 -- 172.21.15.98:0/2018607097 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0037070 con 0x7fdcbc071930 2024-05-09T10:51:40.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.836+0000 7fdcc0da1640 1 -- 172.21.15.98:0/2018607097 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb0032440 con 0x7fdcbc071930 2024-05-09T10:51:40.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.836+0000 7fdcc0da1640 1 -- 172.21.15.98:0/2018607097 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0032810 con 0x7fdcbc071930 2024-05-09T10:51:40.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.836+0000 7fdcc4007640 1 -- 172.21.15.98:0/2018607097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdcbc071930 msgr2=0x7fdcbc06e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.836+0000 7fdcc4007640 1 --2- 172.21.15.98:0/2018607097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdcbc071930 0x7fdcbc06e630 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0b3440 tx=0x7fdcb002a1e0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.836+0000 7fdcc4007640 1 -- 172.21.15.98:0/2018607097 shutdown_connections 2024-05-09T10:51:40.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.836+0000 7fdcc4007640 1 --2- 172.21.15.98:0/2018607097 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdcbc06eb70 0x7fdcbc0af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.836+0000 7fdcc4007640 1 --2- 172.21.15.98:0/2018607097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdcbc071930 0x7fdcbc06e630 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.836+0000 7fdcc4007640 1 --2- 172.21.15.98:0/2018607097 >> v2:172.21.15.105:3300/0 conn(0x7fdcbc155000 0x7fdcbc0713f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.836+0000 7fdcc4007640 1 -- 172.21.15.98:0/2018607097 >> 172.21.15.98:0/2018607097 conn(0x7fdcbc0b24c0 msgr2=0x7fdcbc0b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:40.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.836+0000 7fdcc4007640 1 -- 172.21.15.98:0/2018607097 shutdown_connections 2024-05-09T10:51:40.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.836+0000 7fdcc4007640 1 -- 172.21.15.98:0/2018607097 wait complete. 2024-05-09T10:51:40.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.837+0000 7fdcc4007640 1 Processor -- start 2024-05-09T10:51:40.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.838+0000 7fdcc4007640 1 -- start start 2024-05-09T10:51:40.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.838+0000 7fdcc4007640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdcbc06eb70 0x7fdcbc13f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.838+0000 7fdcc4007640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcbc071930 0x7fdcbc156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.838+0000 7fdcc4007640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdcbc155000 0x7fdcbc153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.838+0000 7fdcc4007640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc0b3de0 con 0x7fdcbc155000 2024-05-09T10:51:40.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.838+0000 7fdcc4007640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc0b3860 con 0x7fdcbc071930 2024-05-09T10:51:40.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.838+0000 7fdcc4007640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc0b40c0 con 0x7fdcbc06eb70 2024-05-09T10:51:40.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.838+0000 7fdcc25a4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdcbc155000 0x7fdcbc153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:40.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.838+0000 7fdcc25a4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdcbc155000 0x7fdcbc153240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54638/0 (socket says 172.21.15.98:54638) 2024-05-09T10:51:40.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.838+0000 7fdcc25a4640 1 -- 172.21.15.98:0/3012178780 learned_addr learned my addr 172.21.15.98:0/3012178780 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:40.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.839+0000 7fdcc25a4640 1 -- 172.21.15.98:0/3012178780 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdcbc06eb70 msgr2=0x7fdcbc13f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.839+0000 7fdcc25a4640 1 --2- 172.21.15.98:0/3012178780 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdcbc06eb70 0x7fdcbc13f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.839+0000 7fdcc25a4640 1 -- 172.21.15.98:0/3012178780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcbc071930 msgr2=0x7fdcbc156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:40.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.839+0000 7fdcc25a4640 1 --2- 172.21.15.98:0/3012178780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcbc071930 0x7fdcbc156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:40.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.839+0000 7fdcc25a4640 1 -- 172.21.15.98:0/3012178780 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcbc13f810 con 0x7fdcbc155000 2024-05-09T10:51:40.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.839+0000 7fdcc25a4640 1 --2- 172.21.15.98:0/3012178780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdcbc155000 0x7fdcbc153240 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400c580 tx=0x7fdcb400ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:40.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.839+0000 7fdcba7fc640 1 -- 172.21.15.98:0/3012178780 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4015070 con 0x7fdcbc155000 2024-05-09T10:51:40.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.839+0000 7fdcba7fc640 1 -- 172.21.15.98:0/3012178780 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb4004220 con 0x7fdcbc155000 2024-05-09T10:51:40.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.839+0000 7fdcba7fc640 1 -- 172.21.15.98:0/3012178780 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4003610 con 0x7fdcbc155000 2024-05-09T10:51:40.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.839+0000 7fdcc4007640 1 -- 172.21.15.98:0/3012178780 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcbc1f4ea0 con 0x7fdcbc155000 2024-05-09T10:51:40.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.839+0000 7fdcc4007640 1 -- 172.21.15.98:0/3012178780 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcbc1f5460 con 0x7fdcbc155000 2024-05-09T10:51:40.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.840+0000 7fdcc4007640 1 -- 172.21.15.98:0/3012178780 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcbc069cb0 con 0x7fdcbc155000 2024-05-09T10:51:40.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.840+0000 7fdcba7fc640 1 -- 172.21.15.98:0/3012178780 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7fdcb40037b0 con 0x7fdcbc155000 2024-05-09T10:51:40.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.841+0000 7fdcba7fc640 1 --2- 172.21.15.98:0/3012178780 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fdca806b680 0x7fdca806db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:40.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.841+0000 7fdcba7fc640 1 -- 172.21.15.98:0/3012178780 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdcb408a0e0 con 0x7fdcbc155000 2024-05-09T10:51:40.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.841+0000 7fdcc1da3640 1 --2- 172.21.15.98:0/3012178780 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fdca806b680 0x7fdca806db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:40.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.841+0000 7fdcc1da3640 1 --2- 172.21.15.98:0/3012178780 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fdca806b680 0x7fdca806db40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009540 tx=0x7fdcac009450 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:40.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:40.845+0000 7fdcba7fc640 1 -- 172.21.15.98:0/3012178780 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdcb4058ee0 con 0x7fdcbc155000 2024-05-09T10:51:41.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.017+0000 7fdcc4007640 1 -- 172.21.15.98:0/3012178780 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fdcbc074480 con 0x7fdca806b680 2024-05-09T10:51:41.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.018+0000 7fdcba7fc640 1 -- 172.21.15.98:0/3012178780 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdcbc074480 con 0x7fdca806b680 2024-05-09T10:51:41.011 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 4 to deep-scrub 2024-05-09T10:51:41.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.019+0000 7fdcc4007640 1 -- 172.21.15.98:0/3012178780 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fdca806b680 msgr2=0x7fdca806db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.019+0000 7fdcc4007640 1 --2- 172.21.15.98:0/3012178780 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fdca806b680 0x7fdca806db40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009540 tx=0x7fdcac009450 comp rx=0 tx=0).stop 2024-05-09T10:51:41.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.019+0000 7fdcc4007640 1 -- 172.21.15.98:0/3012178780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdcbc155000 msgr2=0x7fdcbc153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.019+0000 7fdcc4007640 1 --2- 172.21.15.98:0/3012178780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdcbc155000 0x7fdcbc153240 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400c580 tx=0x7fdcb400ca40 comp rx=0 tx=0).stop 2024-05-09T10:51:41.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.019+0000 7fdcc4007640 1 -- 172.21.15.98:0/3012178780 shutdown_connections 2024-05-09T10:51:41.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.019+0000 7fdcc4007640 1 --2- 172.21.15.98:0/3012178780 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7fdca806b680 0x7fdca806db40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.019+0000 7fdcc4007640 1 --2- 172.21.15.98:0/3012178780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdcbc155000 0x7fdcbc153240 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.019+0000 7fdcc4007640 1 --2- 172.21.15.98:0/3012178780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcbc071930 0x7fdcbc156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.019+0000 7fdcc4007640 1 --2- 172.21.15.98:0/3012178780 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdcbc06eb70 0x7fdcbc13f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.020+0000 7fdcc4007640 1 -- 172.21.15.98:0/3012178780 >> 172.21.15.98:0/3012178780 conn(0x7fdcbc0b24c0 msgr2=0x7fdcbc0b5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:41.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.020+0000 7fdcc4007640 1 -- 172.21.15.98:0/3012178780 shutdown_connections 2024-05-09T10:51:41.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.020+0000 7fdcc4007640 1 -- 172.21.15.98:0/3012178780 wait complete. 2024-05-09T10:51:41.024 INFO:tasks.ceph:Scrubbing osd.5 2024-05-09T10:51:41.024 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-09T10:51:41.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.134+0000 7f3ab035c640 1 Processor -- start 2024-05-09T10:51:41.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.134+0000 7f3ab035c640 1 -- start start 2024-05-09T10:51:41.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.135+0000 7f3ab035c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3aa80d2820 0x7f3aa80d4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.135+0000 7f3ab035c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3aa80b2a30 0x7f3aa80b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.135+0000 7f3ab035c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa8161830 con 0x7f3aa8161450 2024-05-09T10:51:41.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.135+0000 7f3ab035c640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa80b0760 con 0x7f3aa80d2820 2024-05-09T10:51:41.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.135+0000 7f3ab035c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa80cf2f0 con 0x7f3aa80b2a30 2024-05-09T10:51:41.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.136+0000 7f3aad8f7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3aa80d2820 0x7f3aa80d4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:41.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.136+0000 7f3aad8f7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3aa80d2820 0x7f3aa80d4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54654/0 (socket says 172.21.15.98:54654) 2024-05-09T10:51:41.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.136+0000 7f3aae0f8640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f3aa8161450 0x7f3aa80d2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52826/0 (socket says 172.21.15.98:52826) 2024-05-09T10:51:41.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.136+0000 7f3aad8f7640 1 -- 172.21.15.98:0/2412260353 learned_addr learned my addr 172.21.15.98:0/2412260353 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:41.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.136+0000 7f3aae8f9640 1 --2- 172.21.15.98:0/2412260353 >> v2:172.21.15.105:3300/0 conn(0x7f3aa80b2a30 0x7f3aa80b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:41.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.136+0000 7f3aad8f7640 1 -- 172.21.15.98:0/2412260353 >> v2:172.21.15.105:3300/0 conn(0x7f3aa80b2a30 msgr2=0x7f3aa80b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:41.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.136+0000 7f3aad8f7640 1 --2- 172.21.15.98:0/2412260353 >> v2:172.21.15.105:3300/0 conn(0x7f3aa80b2a30 0x7f3aa80b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.136+0000 7f3aad8f7640 1 -- 172.21.15.98:0/2412260353 >> v1:172.21.15.98:6789/0 conn(0x7f3aa8161450 legacy=0x7f3aa80d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.136+0000 7f3aad0f6640 1 -- 172.21.15.98:0/2412260353 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409321896 0 0) 0x7f3aa4001490 con 0x7f3aa8161450 2024-05-09T10:51:41.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.136+0000 7f3aad8f7640 1 -- 172.21.15.98:0/2412260353 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa80cf5d0 con 0x7f3aa80d2820 2024-05-09T10:51:41.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.137+0000 7f3aad8f7640 1 --2- 172.21.15.98:0/2412260353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3aa80d2820 0x7f3aa80d4650 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3aa808cea0 tx=0x7f3a9802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32316abfcc1fac8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:41.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.137+0000 7f3aad0f6640 1 -- 172.21.15.98:0/2412260353 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a98037070 con 0x7f3aa80d2820 2024-05-09T10:51:41.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.137+0000 7f3aad0f6640 1 -- 172.21.15.98:0/2412260353 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a98032440 con 0x7f3aa80d2820 2024-05-09T10:51:41.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.137+0000 7f3aad0f6640 1 -- 172.21.15.98:0/2412260353 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a98032810 con 0x7f3aa80d2820 2024-05-09T10:51:41.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.137+0000 7f3ab035c640 1 -- 172.21.15.98:0/2412260353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3aa80d2820 msgr2=0x7f3aa80d4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.137+0000 7f3ab035c640 1 --2- 172.21.15.98:0/2412260353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3aa80d2820 0x7f3aa80d4650 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3aa808cea0 tx=0x7f3a9802a1e0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.137+0000 7f3ab035c640 1 -- 172.21.15.98:0/2412260353 shutdown_connections 2024-05-09T10:51:41.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.137+0000 7f3ab035c640 1 --2- 172.21.15.98:0/2412260353 >> v2:172.21.15.105:3300/0 conn(0x7f3aa80b2a30 0x7f3aa80b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.137+0000 7f3ab035c640 1 --2- 172.21.15.98:0/2412260353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3aa80d2820 0x7f3aa80d4650 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.137+0000 7f3ab035c640 1 -- 172.21.15.98:0/2412260353 >> 172.21.15.98:0/2412260353 conn(0x7f3aa80d3b30 msgr2=0x7f3aa80d3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:41.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.137+0000 7f3ab035c640 1 -- 172.21.15.98:0/2412260353 shutdown_connections 2024-05-09T10:51:41.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.137+0000 7f3ab035c640 1 -- 172.21.15.98:0/2412260353 wait complete. 2024-05-09T10:51:41.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.138+0000 7f3ab035c640 1 Processor -- start 2024-05-09T10:51:41.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.138+0000 7f3ab035c640 1 -- start start 2024-05-09T10:51:41.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.139+0000 7f3ab035c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3aa80b2a30 0x7f3aa81e79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.139+0000 7f3ab035c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3aa80d2820 0x7f3aa814db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.139+0000 7f3ab035c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa8161450 0x7f3aa814e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.139+0000 7f3ab035c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80b1410 con 0x7f3aa80b2a30 2024-05-09T10:51:41.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.139+0000 7f3ab035c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80b09b0 con 0x7f3aa8161450 2024-05-09T10:51:41.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.139+0000 7f3ab035c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80b1590 con 0x7f3aa80d2820 2024-05-09T10:51:41.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.139+0000 7f3aae0f8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3aa80b2a30 0x7f3aa81e79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:41.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.139+0000 7f3aae0f8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3aa80b2a30 0x7f3aa81e79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54656/0 (socket says 172.21.15.98:54656) 2024-05-09T10:51:41.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.139+0000 7f3aae0f8640 1 -- 172.21.15.98:0/982137626 learned_addr learned my addr 172.21.15.98:0/982137626 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:41.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.139+0000 7f3aae0f8640 1 -- 172.21.15.98:0/982137626 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3aa80d2820 msgr2=0x7f3aa814db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.139+0000 7f3aae0f8640 1 --2- 172.21.15.98:0/982137626 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3aa80d2820 0x7f3aa814db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.140+0000 7f3aae0f8640 1 -- 172.21.15.98:0/982137626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa8161450 msgr2=0x7f3aa814e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.140+0000 7f3aae0f8640 1 --2- 172.21.15.98:0/982137626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa8161450 0x7f3aa814e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.140+0000 7f3aae0f8640 1 -- 172.21.15.98:0/982137626 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa815dcc0 con 0x7f3aa80b2a30 2024-05-09T10:51:41.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.140+0000 7f3aae0f8640 1 --2- 172.21.15.98:0/982137626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3aa80b2a30 0x7f3aa81e79e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400c7c0 tx=0x7f3aa400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:41.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.140+0000 7f3a9effd640 1 -- 172.21.15.98:0/982137626 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4018070 con 0x7f3aa80b2a30 2024-05-09T10:51:41.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.140+0000 7f3a9effd640 1 -- 172.21.15.98:0/982137626 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa400a640 con 0x7f3aa80b2a30 2024-05-09T10:51:41.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.140+0000 7f3a9effd640 1 -- 172.21.15.98:0/982137626 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa400aa30 con 0x7f3aa80b2a30 2024-05-09T10:51:41.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.140+0000 7f3ab035c640 1 -- 172.21.15.98:0/982137626 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa80b2ea0 con 0x7f3aa80b2a30 2024-05-09T10:51:41.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.140+0000 7f3ab035c640 1 -- 172.21.15.98:0/982137626 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa80b3460 con 0x7f3aa80b2a30 2024-05-09T10:51:41.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.141+0000 7f3ab035c640 1 -- 172.21.15.98:0/982137626 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3aa80a1ae0 con 0x7f3aa80b2a30 2024-05-09T10:51:41.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.141+0000 7f3a9effd640 1 -- 172.21.15.98:0/982137626 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f3aa401b070 con 0x7f3aa80b2a30 2024-05-09T10:51:41.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.142+0000 7f3a9effd640 1 --2- 172.21.15.98:0/982137626 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3a8c06b680 0x7f3a8c06db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.142+0000 7f3a9effd640 1 -- 172.21.15.98:0/982137626 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3aa4089840 con 0x7f3aa80b2a30 2024-05-09T10:51:41.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.142+0000 7f3aad8f7640 1 --2- 172.21.15.98:0/982137626 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3a8c06b680 0x7f3a8c06db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:41.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.142+0000 7f3a9effd640 1 --2- 172.21.15.98:0/982137626 >> [v2:172.21.15.105:6800/2110551679,v1:172.21.15.105:6801/2110551679] conn(0x7f3a8c071540 0x7f3a8c073960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.142+0000 7f3a9effd640 1 -- 172.21.15.98:0/982137626 --> [v2:172.21.15.105:6800/2110551679,v1:172.21.15.105:6801/2110551679] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3aa4017ec0 con 0x7f3a8c071540 2024-05-09T10:51:41.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.142+0000 7f3a9effd640 1 -- 172.21.15.98:0/982137626 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3aa408e2a0 con 0x7f3aa80b2a30 2024-05-09T10:51:41.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.142+0000 7f3aad8f7640 1 --2- 172.21.15.98:0/982137626 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3a8c06b680 0x7f3a8c06db40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3a98034000 tx=0x7f3a9802c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:41.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.142+0000 7f3aae8f9640 1 --2- 172.21.15.98:0/982137626 >> [v2:172.21.15.105:6800/2110551679,v1:172.21.15.105:6801/2110551679] conn(0x7f3a8c071540 0x7f3a8c073960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:41.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.143+0000 7f3aae8f9640 1 --2- 172.21.15.98:0/982137626 >> [v2:172.21.15.105:6800/2110551679,v1:172.21.15.105:6801/2110551679] conn(0x7f3a8c071540 0x7f3a8c073960 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:41.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.143+0000 7f3a9effd640 1 -- 172.21.15.98:0/982137626 <== osd.5 v2:172.21.15.105:6800/2110551679 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3aa4017ec0 con 0x7f3a8c071540 2024-05-09T10:51:41.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.156+0000 7f3ab035c640 1 -- 172.21.15.98:0/982137626 --> [v2:172.21.15.105:6800/2110551679,v1:172.21.15.105:6801/2110551679] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3aa80b27f0 con 0x7f3a8c071540 2024-05-09T10:51:41.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.157+0000 7f3a9effd640 1 -- 172.21.15.98:0/982137626 <== osd.5 v2:172.21.15.105:6800/2110551679 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f3aa80b27f0 con 0x7f3a8c071540 2024-05-09T10:51:41.150 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-05-09T10:51:41.150 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T10:51:41.150 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-05-09T10:51:41.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.157+0000 7f3ab035c640 1 -- 172.21.15.98:0/982137626 >> [v2:172.21.15.105:6800/2110551679,v1:172.21.15.105:6801/2110551679] conn(0x7f3a8c071540 msgr2=0x7f3a8c073960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.157+0000 7f3ab035c640 1 --2- 172.21.15.98:0/982137626 >> [v2:172.21.15.105:6800/2110551679,v1:172.21.15.105:6801/2110551679] conn(0x7f3a8c071540 0x7f3a8c073960 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.157+0000 7f3ab035c640 1 -- 172.21.15.98:0/982137626 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3a8c06b680 msgr2=0x7f3a8c06db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.157+0000 7f3ab035c640 1 --2- 172.21.15.98:0/982137626 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f3a8c06b680 0x7f3a8c06db40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3a98034000 tx=0x7f3a9802c000 comp rx=0 tx=0).stop 2024-05-09T10:51:41.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.157+0000 7f3ab035c640 1 -- 172.21.15.98:0/982137626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3aa80b2a30 msgr2=0x7f3aa81e79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.157+0000 7f3ab035c640 1 --2- 172.21.15.98:0/982137626 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3aa80b2a30 0x7f3aa81e79e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400c7c0 tx=0x7f3aa400cc80 comp rx=0 tx=0).stop 2024-05-09T10:51:41.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.157+0000 7f3aae8f9640 1 -- 172.21.15.98:0/982137626 reap_dead start 2024-05-09T10:51:41.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.157+0000 7f3ab035c640 1 -- 172.21.15.98:0/982137626 shutdown_connections 2024-05-09T10:51:41.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.157+0000 7f3ab035c640 1 -- 172.21.15.98:0/982137626 >> 172.21.15.98:0/982137626 conn(0x7f3aa80d3b30 msgr2=0x7f3aa80d0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:41.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.157+0000 7f3ab035c640 1 -- 172.21.15.98:0/982137626 shutdown_connections 2024-05-09T10:51:41.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.157+0000 7f3ab035c640 1 -- 172.21.15.98:0/982137626 wait complete. 2024-05-09T10:51:41.161 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-09T10:51:41.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.268+0000 7f210e522640 1 Processor -- start 2024-05-09T10:51:41.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.268+0000 7f210e522640 1 -- start start 2024-05-09T10:51:41.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.269+0000 7f210e522640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f210815bdc0 0x7f21080cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.269+0000 7f210e522640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21080ceb70 0x7f21080b4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.269+0000 7f210e522640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21080cc4c0 con 0x7f21080b4d40 2024-05-09T10:51:41.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.269+0000 7f210e522640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f210815c1a0 con 0x7f210815bdc0 2024-05-09T10:51:41.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.269+0000 7f210e522640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21080cf3c0 con 0x7f21080ceb70 2024-05-09T10:51:41.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.269+0000 7f2107fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f210815bdc0 0x7f21080cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:41.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.269+0000 7f210cabf640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f21080b4d40 0x7f210815f820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52842/0 (socket says 172.21.15.98:52842) 2024-05-09T10:51:41.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.269+0000 7f210cabf640 1 -- 172.21.15.98:0/3437958784 learned_addr learned my addr 172.21.15.98:0/3437958784 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:41.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.269+0000 7f21077fe640 1 --2- 172.21.15.98:0/3437958784 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21080ceb70 0x7f21080b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:41.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.269+0000 7f2107fff640 1 -- 172.21.15.98:0/3437958784 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21080ceb70 msgr2=0x7f21080b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:41.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.269+0000 7f2107fff640 1 --2- 172.21.15.98:0/3437958784 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21080ceb70 0x7f21080b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.269+0000 7f2107fff640 1 -- 172.21.15.98:0/3437958784 >> v1:172.21.15.98:6789/0 conn(0x7f21080b4d40 legacy=0x7f210815f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.269+0000 7f2107fff640 1 -- 172.21.15.98:0/3437958784 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21080cf0e0 con 0x7f210815bdc0 2024-05-09T10:51:41.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.269+0000 7f2106ffd640 1 -- 172.21.15.98:0/3437958784 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4177984680 0 0) 0x7f21080cc4c0 con 0x7f21080b4d40 2024-05-09T10:51:41.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.269+0000 7f21077fe640 1 --2- 172.21.15.98:0/3437958784 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21080ceb70 0x7f21080b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:41.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.270+0000 7f2107fff640 1 --2- 172.21.15.98:0/3437958784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f210815bdc0 0x7f21080cd240 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f20f80097a0 tx=0x7f20f802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a67e5e02cab65ab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:41.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.270+0000 7f2106ffd640 1 -- 172.21.15.98:0/3437958784 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f8037070 con 0x7f210815bdc0 2024-05-09T10:51:41.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.270+0000 7f2106ffd640 1 -- 172.21.15.98:0/3437958784 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f8032440 con 0x7f210815bdc0 2024-05-09T10:51:41.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.270+0000 7f2106ffd640 1 -- 172.21.15.98:0/3437958784 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f8032810 con 0x7f210815bdc0 2024-05-09T10:51:41.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.271+0000 7f210e522640 1 -- 172.21.15.98:0/3437958784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f210815bdc0 msgr2=0x7f21080cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.271+0000 7f210e522640 1 --2- 172.21.15.98:0/3437958784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f210815bdc0 0x7f21080cd240 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f20f80097a0 tx=0x7f20f802a180 comp rx=0 tx=0).stop 2024-05-09T10:51:41.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.271+0000 7f210e522640 1 -- 172.21.15.98:0/3437958784 shutdown_connections 2024-05-09T10:51:41.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.271+0000 7f210e522640 1 --2- 172.21.15.98:0/3437958784 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21080ceb70 0x7f21080b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.271+0000 7f210e522640 1 --2- 172.21.15.98:0/3437958784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f210815bdc0 0x7f21080cd240 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.271+0000 7f210e522640 1 -- 172.21.15.98:0/3437958784 >> 172.21.15.98:0/3437958784 conn(0x7f21080b34e0 msgr2=0x7f21080b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:41.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.271+0000 7f210e522640 1 -- 172.21.15.98:0/3437958784 shutdown_connections 2024-05-09T10:51:41.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.271+0000 7f210e522640 1 -- 172.21.15.98:0/3437958784 wait complete. 2024-05-09T10:51:41.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.272+0000 7f210e522640 1 Processor -- start 2024-05-09T10:51:41.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.272+0000 7f210e522640 1 -- start start 2024-05-09T10:51:41.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.273+0000 7f210e522640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21080b4d40 0x7f21080c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.273+0000 7f210e522640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21080ceb70 0x7f21081f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.273+0000 7f210e522640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f210815bdc0 0x7f21081f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.273+0000 7f210e522640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21080ccd20 con 0x7f210815bdc0 2024-05-09T10:51:41.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.273+0000 7f210e522640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21080cca40 con 0x7f21080b4d40 2024-05-09T10:51:41.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.273+0000 7f210e522640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21080b51d0 con 0x7f21080ceb70 2024-05-09T10:51:41.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.273+0000 7f210cabf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f210815bdc0 0x7f21081f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:41.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.273+0000 7f210cabf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f210815bdc0 0x7f21081f4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54682/0 (socket says 172.21.15.98:54682) 2024-05-09T10:51:41.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.273+0000 7f210cabf640 1 -- 172.21.15.98:0/1197247159 learned_addr learned my addr 172.21.15.98:0/1197247159 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:41.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.273+0000 7f21077fe640 1 --2- 172.21.15.98:0/1197247159 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21080ceb70 0x7f21081f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:41.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.273+0000 7f2107fff640 1 --2- 172.21.15.98:0/1197247159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21080b4d40 0x7f21080c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:41.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.273+0000 7f210cabf640 1 -- 172.21.15.98:0/1197247159 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21080ceb70 msgr2=0x7f21081f1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.273+0000 7f210cabf640 1 --2- 172.21.15.98:0/1197247159 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21080ceb70 0x7f21081f1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.273+0000 7f210cabf640 1 -- 172.21.15.98:0/1197247159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21080b4d40 msgr2=0x7f21080c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.273+0000 7f210cabf640 1 --2- 172.21.15.98:0/1197247159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21080b4d40 0x7f21080c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.273+0000 7f210cabf640 1 -- 172.21.15.98:0/1197247159 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21081f5500 con 0x7f210815bdc0 2024-05-09T10:51:41.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.273+0000 7f210cabf640 1 --2- 172.21.15.98:0/1197247159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f210815bdc0 0x7f21081f4b90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f210000c730 tx=0x7f210000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:41.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.273+0000 7f2104ff9640 1 -- 172.21.15.98:0/1197247159 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2100018070 con 0x7f210815bdc0 2024-05-09T10:51:41.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.274+0000 7f2104ff9640 1 -- 172.21.15.98:0/1197247159 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210000ce40 con 0x7f210815bdc0 2024-05-09T10:51:41.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.274+0000 7f210e522640 1 -- 172.21.15.98:0/1197247159 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21081f57e0 con 0x7f210815bdc0 2024-05-09T10:51:41.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.274+0000 7f2104ff9640 1 -- 172.21.15.98:0/1197247159 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210000a860 con 0x7f210815bdc0 2024-05-09T10:51:41.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.274+0000 7f210e522640 1 -- 172.21.15.98:0/1197247159 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21081f1a20 con 0x7f210815bdc0 2024-05-09T10:51:41.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.275+0000 7f210e522640 1 -- 172.21.15.98:0/1197247159 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2108069cb0 con 0x7f210815bdc0 2024-05-09T10:51:41.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.275+0000 7f2104ff9640 1 -- 172.21.15.98:0/1197247159 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f210000aa00 con 0x7f210815bdc0 2024-05-09T10:51:41.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.275+0000 7f2104ff9640 1 --2- 172.21.15.98:0/1197247159 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f20f406b680 0x7f20f406db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.276+0000 7f2104ff9640 1 -- 172.21.15.98:0/1197247159 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2100089730 con 0x7f210815bdc0 2024-05-09T10:51:41.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.276+0000 7f2107fff640 1 --2- 172.21.15.98:0/1197247159 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f20f406b680 0x7f20f406db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:41.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.276+0000 7f2107fff640 1 --2- 172.21.15.98:0/1197247159 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f20f406b680 0x7f20f406db40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f20f8034000 tx=0x7f20f802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:41.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.280+0000 7f2104ff9640 1 -- 172.21.15.98:0/1197247159 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2100058010 con 0x7f210815bdc0 2024-05-09T10:51:41.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.451+0000 7f210e522640 1 -- 172.21.15.98:0/1197247159 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f21080b7930 con 0x7f20f406b680 2024-05-09T10:51:41.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.452+0000 7f2104ff9640 1 -- 172.21.15.98:0/1197247159 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f21080b7930 con 0x7f20f406b680 2024-05-09T10:51:41.445 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 5 to deep-scrub 2024-05-09T10:51:41.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.453+0000 7f210e522640 1 -- 172.21.15.98:0/1197247159 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f20f406b680 msgr2=0x7f20f406db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.453+0000 7f210e522640 1 --2- 172.21.15.98:0/1197247159 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f20f406b680 0x7f20f406db40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f20f8034000 tx=0x7f20f802e000 comp rx=0 tx=0).stop 2024-05-09T10:51:41.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.454+0000 7f210e522640 1 -- 172.21.15.98:0/1197247159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f210815bdc0 msgr2=0x7f21081f4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.454+0000 7f210e522640 1 --2- 172.21.15.98:0/1197247159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f210815bdc0 0x7f21081f4b90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f210000c730 tx=0x7f210000cbf0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.454+0000 7f210e522640 1 -- 172.21.15.98:0/1197247159 shutdown_connections 2024-05-09T10:51:41.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.454+0000 7f210e522640 1 --2- 172.21.15.98:0/1197247159 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f20f406b680 0x7f20f406db40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.454+0000 7f210e522640 1 --2- 172.21.15.98:0/1197247159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f210815bdc0 0x7f21081f4b90 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.454+0000 7f210e522640 1 --2- 172.21.15.98:0/1197247159 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f21080ceb70 0x7f21081f1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.454+0000 7f210e522640 1 --2- 172.21.15.98:0/1197247159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f21080b4d40 0x7f21080c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.454+0000 7f210e522640 1 -- 172.21.15.98:0/1197247159 >> 172.21.15.98:0/1197247159 conn(0x7f21080b34e0 msgr2=0x7f210806e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:41.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.454+0000 7f210e522640 1 -- 172.21.15.98:0/1197247159 shutdown_connections 2024-05-09T10:51:41.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.454+0000 7f210e522640 1 -- 172.21.15.98:0/1197247159 wait complete. 2024-05-09T10:51:41.458 INFO:tasks.ceph:Scrubbing osd.6 2024-05-09T10:51:41.458 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-09T10:51:41.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.568+0000 7f17475e7640 1 Processor -- start 2024-05-09T10:51:41.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.569+0000 7f17475e7640 1 -- start start 2024-05-09T10:51:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.570+0000 7f17475e7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17400d1fc0 0x7f1740166970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.570+0000 7f17475e7640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17400b2bb0 con 0x7f17400d1be0 2024-05-09T10:51:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.570+0000 7f17475e7640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17400b2e90 con 0x7f17400cf950 2024-05-09T10:51:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.570+0000 7f17475e7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17400af840 con 0x7f17400d1fc0 2024-05-09T10:51:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.570+0000 7f1745383640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f17400d1be0 0x7f17400d11d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52848/0 (socket says 172.21.15.98:52848) 2024-05-09T10:51:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.570+0000 7f1745383640 1 -- 172.21.15.98:0/3698233636 learned_addr learned my addr 172.21.15.98:0/3698233636 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:51:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.570+0000 7f1745b84640 1 --2- 172.21.15.98:0/3698233636 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17400d1fc0 0x7f1740166970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.570+0000 7f1737fff640 1 -- 172.21.15.98:0/3698233636 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 863024158 0 0) 0x7f17400b2bb0 con 0x7f17400d1be0 2024-05-09T10:51:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.570+0000 7f1737fff640 1 -- 172.21.15.98:0/3698233636 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f172c003520 con 0x7f17400d1be0 2024-05-09T10:51:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.570+0000 7f1737fff640 1 -- 172.21.15.98:0/3698233636 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970665715 0 0) 0x7f17400b2e90 con 0x7f17400cf950 2024-05-09T10:51:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.570+0000 7f1737fff640 1 -- 172.21.15.98:0/3698233636 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17400b2bb0 con 0x7f17400cf950 2024-05-09T10:51:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.570+0000 7f1745b84640 1 -- 172.21.15.98:0/3698233636 >> v1:172.21.15.105:6789/0 conn(0x7f17400cf950 legacy=0x7f174015bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.570+0000 7f1745b84640 1 -- 172.21.15.98:0/3698233636 >> v1:172.21.15.98:6789/0 conn(0x7f17400d1be0 legacy=0x7f17400d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.570+0000 7f1737fff640 1 -- 172.21.15.98:0/3698233636 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2988790285 0 0) 0x7f172c003520 con 0x7f17400d1be0 2024-05-09T10:51:41.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.570+0000 7f1745b84640 1 -- 172.21.15.98:0/3698233636 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17400b3170 con 0x7f17400d1fc0 2024-05-09T10:51:41.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.571+0000 7f1745b84640 1 --2- 172.21.15.98:0/3698233636 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17400d1fc0 0x7f1740166970 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1738009dc0 tx=0x7f173802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d90040eedb50812a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:41.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.571+0000 7f1737fff640 1 -- 172.21.15.98:0/3698233636 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c001690 con 0x7f17400d1fc0 2024-05-09T10:51:41.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.571+0000 7f1737fff640 1 -- 172.21.15.98:0/3698233636 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1738007540 con 0x7f17400d1fc0 2024-05-09T10:51:41.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.571+0000 7f1737fff640 1 -- 172.21.15.98:0/3698233636 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17380078f0 con 0x7f17400d1fc0 2024-05-09T10:51:41.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.572+0000 7f17475e7640 1 -- 172.21.15.98:0/3698233636 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17400d1fc0 msgr2=0x7f1740166970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.572+0000 7f17475e7640 1 --2- 172.21.15.98:0/3698233636 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17400d1fc0 0x7f1740166970 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1738009dc0 tx=0x7f173802a5d0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.572+0000 7f17475e7640 1 -- 172.21.15.98:0/3698233636 shutdown_connections 2024-05-09T10:51:41.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.572+0000 7f17475e7640 1 --2- 172.21.15.98:0/3698233636 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17400d1fc0 0x7f1740166970 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.572+0000 7f17475e7640 1 -- 172.21.15.98:0/3698233636 >> 172.21.15.98:0/3698233636 conn(0x7f17400b6db0 msgr2=0x7f17400b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:41.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.572+0000 7f17475e7640 1 -- 172.21.15.98:0/3698233636 shutdown_connections 2024-05-09T10:51:41.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.572+0000 7f17475e7640 1 -- 172.21.15.98:0/3698233636 wait complete. 2024-05-09T10:51:41.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.573+0000 7f17475e7640 1 Processor -- start 2024-05-09T10:51:41.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.573+0000 7f17475e7640 1 -- start start 2024-05-09T10:51:41.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.574+0000 7f17475e7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17400cf950 0x7f1740158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.574+0000 7f17475e7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17400d1be0 0x7f174015cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.574+0000 7f17475e7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17400d1fc0 0x7f174014a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.574+0000 7f17475e7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17400b26b0 con 0x7f17400d1fc0 2024-05-09T10:51:41.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.574+0000 7f17475e7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17400b2190 con 0x7f17400cf950 2024-05-09T10:51:41.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.574+0000 7f17475e7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174015d320 con 0x7f17400d1be0 2024-05-09T10:51:41.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.574+0000 7f1745b84640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17400d1fc0 0x7f174014a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:41.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.574+0000 7f1745b84640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17400d1fc0 0x7f174014a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54692/0 (socket says 172.21.15.98:54692) 2024-05-09T10:51:41.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.574+0000 7f1745b84640 1 -- 172.21.15.98:0/1751215511 learned_addr learned my addr 172.21.15.98:0/1751215511 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:41.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.574+0000 7f1745383640 1 --2- 172.21.15.98:0/1751215511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17400cf950 0x7f1740158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:41.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.574+0000 7f1744b82640 1 --2- 172.21.15.98:0/1751215511 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17400d1be0 0x7f174015cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:41.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.574+0000 7f1745b84640 1 -- 172.21.15.98:0/1751215511 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17400d1be0 msgr2=0x7f174015cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.574+0000 7f1745b84640 1 --2- 172.21.15.98:0/1751215511 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f17400d1be0 0x7f174015cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.574+0000 7f1745b84640 1 -- 172.21.15.98:0/1751215511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17400cf950 msgr2=0x7f1740158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.574+0000 7f1745b84640 1 --2- 172.21.15.98:0/1751215511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17400cf950 0x7f1740158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.574+0000 7f1745b84640 1 -- 172.21.15.98:0/1751215511 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174014ae10 con 0x7f17400d1fc0 2024-05-09T10:51:41.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.574+0000 7f1745b84640 1 --2- 172.21.15.98:0/1751215511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17400d1fc0 0x7f174014a8d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1738009ee0 tx=0x7f1738004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:41.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.574+0000 7f1735ffb640 1 -- 172.21.15.98:0/1751215511 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738040070 con 0x7f17400d1fc0 2024-05-09T10:51:41.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.575+0000 7f1735ffb640 1 -- 172.21.15.98:0/1751215511 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1738031c40 con 0x7f17400d1fc0 2024-05-09T10:51:41.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.575+0000 7f1735ffb640 1 -- 172.21.15.98:0/1751215511 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738009450 con 0x7f17400d1fc0 2024-05-09T10:51:41.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.575+0000 7f17475e7640 1 -- 172.21.15.98:0/1751215511 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174015bd40 con 0x7f17400d1fc0 2024-05-09T10:51:41.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.575+0000 7f17475e7640 1 -- 172.21.15.98:0/1751215511 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174015c300 con 0x7f17400d1fc0 2024-05-09T10:51:41.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.575+0000 7f17475e7640 1 -- 172.21.15.98:0/1751215511 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f174009fbb0 con 0x7f17400d1fc0 2024-05-09T10:51:41.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.576+0000 7f1735ffb640 1 -- 172.21.15.98:0/1751215511 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f1738039040 con 0x7f17400d1fc0 2024-05-09T10:51:41.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.576+0000 7f1735ffb640 1 --2- 172.21.15.98:0/1751215511 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f172406b680 0x7f172406db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.577+0000 7f1745383640 1 --2- 172.21.15.98:0/1751215511 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f172406b680 0x7f172406db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:41.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.577+0000 7f1735ffb640 1 -- 172.21.15.98:0/1751215511 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f17380ab710 con 0x7f17400d1fc0 2024-05-09T10:51:41.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.577+0000 7f1735ffb640 1 --2- 172.21.15.98:0/1751215511 >> [v2:172.21.15.105:6816/721380547,v1:172.21.15.105:6817/721380547] conn(0x7f1724071540 0x7f1724073960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.577+0000 7f1735ffb640 1 -- 172.21.15.98:0/1751215511 --> [v2:172.21.15.105:6816/721380547,v1:172.21.15.105:6817/721380547] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1738007540 con 0x7f1724071540 2024-05-09T10:51:41.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.577+0000 7f1735ffb640 1 -- 172.21.15.98:0/1751215511 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17380b2050 con 0x7f17400d1fc0 2024-05-09T10:51:41.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.577+0000 7f1745383640 1 --2- 172.21.15.98:0/1751215511 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f172406b680 0x7f172406db40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1740070fd0 tx=0x7f173c0014f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:41.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.577+0000 7f1744b82640 1 --2- 172.21.15.98:0/1751215511 >> [v2:172.21.15.105:6816/721380547,v1:172.21.15.105:6817/721380547] conn(0x7f1724071540 0x7f1724073960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:41.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.577+0000 7f1744b82640 1 --2- 172.21.15.98:0/1751215511 >> [v2:172.21.15.105:6816/721380547,v1:172.21.15.105:6817/721380547] conn(0x7f1724071540 0x7f1724073960 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:41.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.578+0000 7f1735ffb640 1 -- 172.21.15.98:0/1751215511 <== osd.6 v2:172.21.15.105:6816/721380547 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1738007540 con 0x7f1724071540 2024-05-09T10:51:41.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.591+0000 7f17475e7640 1 -- 172.21.15.98:0/1751215511 --> [v2:172.21.15.105:6816/721380547,v1:172.21.15.105:6817/721380547] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f174004f000 con 0x7f1724071540 2024-05-09T10:51:41.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.592+0000 7f1735ffb640 1 -- 172.21.15.98:0/1751215511 <== osd.6 v2:172.21.15.105:6816/721380547 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f174004f000 con 0x7f1724071540 2024-05-09T10:51:41.585 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-05-09T10:51:41.585 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T10:51:41.585 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-05-09T10:51:41.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.592+0000 7f17475e7640 1 -- 172.21.15.98:0/1751215511 >> [v2:172.21.15.105:6816/721380547,v1:172.21.15.105:6817/721380547] conn(0x7f1724071540 msgr2=0x7f1724073960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.592+0000 7f17475e7640 1 --2- 172.21.15.98:0/1751215511 >> [v2:172.21.15.105:6816/721380547,v1:172.21.15.105:6817/721380547] conn(0x7f1724071540 0x7f1724073960 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.592+0000 7f17475e7640 1 -- 172.21.15.98:0/1751215511 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f172406b680 msgr2=0x7f172406db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.592+0000 7f17475e7640 1 --2- 172.21.15.98:0/1751215511 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f172406b680 0x7f172406db40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1740070fd0 tx=0x7f173c0014f0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.592+0000 7f17475e7640 1 -- 172.21.15.98:0/1751215511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17400d1fc0 msgr2=0x7f174014a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.592+0000 7f17475e7640 1 --2- 172.21.15.98:0/1751215511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17400d1fc0 0x7f174014a8d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1738009ee0 tx=0x7f1738004a40 comp rx=0 tx=0).stop 2024-05-09T10:51:41.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.592+0000 7f1745b84640 1 -- 172.21.15.98:0/1751215511 reap_dead start 2024-05-09T10:51:41.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.592+0000 7f17475e7640 1 -- 172.21.15.98:0/1751215511 shutdown_connections 2024-05-09T10:51:41.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.592+0000 7f17475e7640 1 -- 172.21.15.98:0/1751215511 >> 172.21.15.98:0/1751215511 conn(0x7f17400b6db0 msgr2=0x7f17400af9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:41.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.592+0000 7f17475e7640 1 -- 172.21.15.98:0/1751215511 shutdown_connections 2024-05-09T10:51:41.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.592+0000 7f17475e7640 1 -- 172.21.15.98:0/1751215511 wait complete. 2024-05-09T10:51:41.596 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-09T10:51:41.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.702+0000 7f660f109640 1 Processor -- start 2024-05-09T10:51:41.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.702+0000 7f660f109640 1 -- start start 2024-05-09T10:51:41.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.703+0000 7f660f109640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f660815bce0 0x7f660806ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.703+0000 7f660f109640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f660806b2f0 0x7f660815c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.703+0000 7f660f109640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66080721a0 con 0x7f66080ce240 2024-05-09T10:51:41.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.703+0000 7f660f109640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f660806d920 con 0x7f660815bce0 2024-05-09T10:51:41.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.703+0000 7f660f109640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6608072480 con 0x7f660806b2f0 2024-05-09T10:51:41.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.703+0000 7f660cea5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f660815bce0 0x7f660806ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:41.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.703+0000 7f660cea5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f660815bce0 0x7f660806ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54696/0 (socket says 172.21.15.98:54696) 2024-05-09T10:51:41.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.703+0000 7f660cea5640 1 -- 172.21.15.98:0/603205830 learned_addr learned my addr 172.21.15.98:0/603205830 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:41.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.703+0000 7f660d6a6640 1 --2- 172.21.15.98:0/603205830 >> v2:172.21.15.105:3300/0 conn(0x7f660806b2f0 0x7f660815c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:41.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.703+0000 7f660cea5640 1 -- 172.21.15.98:0/603205830 >> v2:172.21.15.105:3300/0 conn(0x7f660806b2f0 msgr2=0x7f660815c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:41.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.703+0000 7f660cea5640 1 --2- 172.21.15.98:0/603205830 >> v2:172.21.15.105:3300/0 conn(0x7f660806b2f0 0x7f660815c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.703+0000 7f660cea5640 1 -- 172.21.15.98:0/603205830 >> v1:172.21.15.105:6789/0 conn(0x7f66080ce240 legacy=0x7f660806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.703+0000 7f660cea5640 1 -- 172.21.15.98:0/603205830 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660806d5b0 con 0x7f660815bce0 2024-05-09T10:51:41.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.704+0000 7f660cea5640 1 --2- 172.21.15.98:0/603205830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f660815bce0 0x7f660806ddd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f66080ce760 tx=0x7f65f802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525ea0eabbb42005 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:41.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.704+0000 7f66077fe640 1 -- 172.21.15.98:0/603205830 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f8037070 con 0x7f660815bce0 2024-05-09T10:51:41.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.704+0000 7f66077fe640 1 -- 172.21.15.98:0/603205830 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f8032440 con 0x7f660815bce0 2024-05-09T10:51:41.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.704+0000 7f66077fe640 1 -- 172.21.15.98:0/603205830 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f8032810 con 0x7f660815bce0 2024-05-09T10:51:41.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.705+0000 7f660f109640 1 -- 172.21.15.98:0/603205830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f660815bce0 msgr2=0x7f660806ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.705+0000 7f660f109640 1 --2- 172.21.15.98:0/603205830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f660815bce0 0x7f660806ddd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f66080ce760 tx=0x7f65f802a1e0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.705+0000 7f660f109640 1 -- 172.21.15.98:0/603205830 shutdown_connections 2024-05-09T10:51:41.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.705+0000 7f660f109640 1 --2- 172.21.15.98:0/603205830 >> v2:172.21.15.105:3300/0 conn(0x7f660806b2f0 0x7f660815c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.705+0000 7f660f109640 1 --2- 172.21.15.98:0/603205830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f660815bce0 0x7f660806ddd0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.705+0000 7f660f109640 1 -- 172.21.15.98:0/603205830 >> 172.21.15.98:0/603205830 conn(0x7f6608066f20 msgr2=0x7f660806bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:41.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.705+0000 7f660f109640 1 -- 172.21.15.98:0/603205830 shutdown_connections 2024-05-09T10:51:41.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.705+0000 7f660f109640 1 -- 172.21.15.98:0/603205830 wait complete. 2024-05-09T10:51:41.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.706+0000 7f660f109640 1 Processor -- start 2024-05-09T10:51:41.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.706+0000 7f660f109640 1 -- start start 2024-05-09T10:51:41.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.707+0000 7f660f109640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f660806b2f0 0x7f6608152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.707+0000 7f660f109640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f660815bce0 0x7f660815a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.707+0000 7f660f109640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6608143d80 0x7f6608144230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.707+0000 7f660f109640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66080d0390 con 0x7f660815bce0 2024-05-09T10:51:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.707+0000 7f660f109640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66080d0510 con 0x7f660806b2f0 2024-05-09T10:51:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.707+0000 7f660f109640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660815c7c0 con 0x7f6608143d80 2024-05-09T10:51:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.707+0000 7f6607fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f660815bce0 0x7f660815a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.707+0000 7f6607fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f660815bce0 0x7f660815a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54698/0 (socket says 172.21.15.98:54698) 2024-05-09T10:51:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.707+0000 7f6607fff640 1 -- 172.21.15.98:0/1512264972 learned_addr learned my addr 172.21.15.98:0/1512264972 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.707+0000 7f660cea5640 1 --2- 172.21.15.98:0/1512264972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f660806b2f0 0x7f6608152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.707+0000 7f660d6a6640 1 --2- 172.21.15.98:0/1512264972 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6608143d80 0x7f6608144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.707+0000 7f6607fff640 1 -- 172.21.15.98:0/1512264972 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6608143d80 msgr2=0x7f6608144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.707+0000 7f6607fff640 1 --2- 172.21.15.98:0/1512264972 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6608143d80 0x7f6608144230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.707+0000 7f6607fff640 1 -- 172.21.15.98:0/1512264972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f660806b2f0 msgr2=0x7f6608152960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.707+0000 7f6607fff640 1 --2- 172.21.15.98:0/1512264972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f660806b2f0 0x7f6608152960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.707+0000 7f6607fff640 1 -- 172.21.15.98:0/1512264972 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6608158f40 con 0x7f660815bce0 2024-05-09T10:51:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.708+0000 7f6607fff640 1 --2- 172.21.15.98:0/1512264972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f660815bce0 0x7f660815a550 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00c760 tx=0x7f65fc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.708+0000 7f66057fa640 1 -- 172.21.15.98:0/1512264972 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc016020 con 0x7f660815bce0 2024-05-09T10:51:41.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.708+0000 7f66057fa640 1 -- 172.21.15.98:0/1512264972 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65fc006d50 con 0x7f660815bce0 2024-05-09T10:51:41.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.708+0000 7f660f109640 1 -- 172.21.15.98:0/1512264972 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6608159220 con 0x7f660815bce0 2024-05-09T10:51:41.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.708+0000 7f66057fa640 1 -- 172.21.15.98:0/1512264972 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc003bb0 con 0x7f660815bce0 2024-05-09T10:51:41.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.708+0000 7f660f109640 1 -- 172.21.15.98:0/1512264972 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f660814fdd0 con 0x7f660815bce0 2024-05-09T10:51:41.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.709+0000 7f660f109640 1 -- 172.21.15.98:0/1512264972 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f660814fbf0 con 0x7f660815bce0 2024-05-09T10:51:41.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.709+0000 7f66057fa640 1 -- 172.21.15.98:0/1512264972 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f65fc00a850 con 0x7f660815bce0 2024-05-09T10:51:41.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.710+0000 7f66057fa640 1 --2- 172.21.15.98:0/1512264972 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f65f406b680 0x7f65f406db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:41.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.710+0000 7f66057fa640 1 -- 172.21.15.98:0/1512264972 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f65fc089d90 con 0x7f660815bce0 2024-05-09T10:51:41.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.710+0000 7f660cea5640 1 --2- 172.21.15.98:0/1512264972 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f65f406b680 0x7f65f406db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:41.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.710+0000 7f660cea5640 1 --2- 172.21.15.98:0/1512264972 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f65f406b680 0x7f65f406db40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f65f8034000 tx=0x7f65f802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:41.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.714+0000 7f66057fa640 1 -- 172.21.15.98:0/1512264972 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f660814fbf0 con 0x7f660815bce0 2024-05-09T10:51:41.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.887+0000 7f660f109640 1 -- 172.21.15.98:0/1512264972 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f6608069cb0 con 0x7f65f406b680 2024-05-09T10:51:41.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.888+0000 7f66057fa640 1 -- 172.21.15.98:0/1512264972 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6608069cb0 con 0x7f65f406b680 2024-05-09T10:51:41.881 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 6 to deep-scrub 2024-05-09T10:51:41.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.890+0000 7f660f109640 1 -- 172.21.15.98:0/1512264972 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f65f406b680 msgr2=0x7f65f406db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.890+0000 7f660f109640 1 --2- 172.21.15.98:0/1512264972 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f65f406b680 0x7f65f406db40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f65f8034000 tx=0x7f65f802e000 comp rx=0 tx=0).stop 2024-05-09T10:51:41.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.890+0000 7f660f109640 1 -- 172.21.15.98:0/1512264972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f660815bce0 msgr2=0x7f660815a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:41.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.890+0000 7f660f109640 1 --2- 172.21.15.98:0/1512264972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f660815bce0 0x7f660815a550 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00c760 tx=0x7f65fc00cc20 comp rx=0 tx=0).stop 2024-05-09T10:51:41.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.890+0000 7f660f109640 1 -- 172.21.15.98:0/1512264972 shutdown_connections 2024-05-09T10:51:41.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.890+0000 7f660f109640 1 --2- 172.21.15.98:0/1512264972 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f65f406b680 0x7f65f406db40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.890+0000 7f660f109640 1 --2- 172.21.15.98:0/1512264972 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6608143d80 0x7f6608144230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.890+0000 7f660f109640 1 --2- 172.21.15.98:0/1512264972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f660815bce0 0x7f660815a550 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.890+0000 7f660f109640 1 --2- 172.21.15.98:0/1512264972 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f660806b2f0 0x7f6608152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:41.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.890+0000 7f660f109640 1 -- 172.21.15.98:0/1512264972 >> 172.21.15.98:0/1512264972 conn(0x7f6608066f20 msgr2=0x7f66080b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:41.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.890+0000 7f660f109640 1 -- 172.21.15.98:0/1512264972 shutdown_connections 2024-05-09T10:51:41.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:41.890+0000 7f660f109640 1 -- 172.21.15.98:0/1512264972 wait complete. 2024-05-09T10:51:41.894 INFO:tasks.ceph:Scrubbing osd.7 2024-05-09T10:51:41.894 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-09T10:51:41.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.005+0000 7f6855b14640 1 Processor -- start 2024-05-09T10:51:41.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.005+0000 7f6855b14640 1 -- start start 2024-05-09T10:51:42.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.005+0000 7f6855b14640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f685015a8f0 0x7f685015f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:42.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.005+0000 7f6855b14640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f68500ca530 0x7f68500ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:42.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.005+0000 7f6855b14640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f685015acd0 0x7f68500cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:42.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.005+0000 7f6855b14640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685015f5d0 con 0x7f68500ca530 2024-05-09T10:51:42.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.005+0000 7f6855b14640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68500cb540 con 0x7f685015a8f0 2024-05-09T10:51:42.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.005+0000 7f6855b14640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68500ae960 con 0x7f685015acd0 2024-05-09T10:51:42.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.006+0000 7f684effd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f68500ca530 0x7f68500ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:42.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.006+0000 7f684effd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f68500ca530 0x7f68500ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54702/0 (socket says 172.21.15.98:54702) 2024-05-09T10:51:42.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.006+0000 7f684effd640 1 -- 172.21.15.98:0/4160411089 learned_addr learned my addr 172.21.15.98:0/4160411089 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:42.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.006+0000 7f684f7fe640 1 --2- 172.21.15.98:0/4160411089 >> v2:172.21.15.105:3300/0 conn(0x7f685015a8f0 0x7f685015f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:42.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.006+0000 7f684ffff640 1 --2- 172.21.15.98:0/4160411089 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f685015acd0 0x7f68500cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:42.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.006+0000 7f684effd640 1 -- 172.21.15.98:0/4160411089 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f685015acd0 msgr2=0x7f68500cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:42.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.006+0000 7f684effd640 1 --2- 172.21.15.98:0/4160411089 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f685015acd0 0x7f68500cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.006+0000 7f684effd640 1 -- 172.21.15.98:0/4160411089 >> v2:172.21.15.105:3300/0 conn(0x7f685015a8f0 msgr2=0x7f685015f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:42.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.006+0000 7f684effd640 1 --2- 172.21.15.98:0/4160411089 >> v2:172.21.15.105:3300/0 conn(0x7f685015a8f0 0x7f685015f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.006+0000 7f684effd640 1 -- 172.21.15.98:0/4160411089 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68500ca9c0 con 0x7f68500ca530 2024-05-09T10:51:42.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.006+0000 7f684f7fe640 1 --2- 172.21.15.98:0/4160411089 >> v2:172.21.15.105:3300/0 conn(0x7f685015a8f0 0x7f685015f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:42.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.006+0000 7f684ffff640 1 --2- 172.21.15.98:0/4160411089 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f685015acd0 0x7f68500cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:42.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.007+0000 7f684effd640 1 --2- 172.21.15.98:0/4160411089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68500ca530 0x7f68500ce770 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f68500c47b0 tx=0x7f684402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08430d5cb36b42d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:42.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.007+0000 7f684e7fc640 1 -- 172.21.15.98:0/4160411089 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844037070 con 0x7f68500ca530 2024-05-09T10:51:42.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.007+0000 7f684e7fc640 1 -- 172.21.15.98:0/4160411089 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6844032440 con 0x7f68500ca530 2024-05-09T10:51:42.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.007+0000 7f684e7fc640 1 -- 172.21.15.98:0/4160411089 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844032810 con 0x7f68500ca530 2024-05-09T10:51:42.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.007+0000 7f6855b14640 1 -- 172.21.15.98:0/4160411089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68500ca530 msgr2=0x7f68500ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:42.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.007+0000 7f6855b14640 1 --2- 172.21.15.98:0/4160411089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68500ca530 0x7f68500ce770 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f68500c47b0 tx=0x7f684402a1e0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.007+0000 7f6855b14640 1 -- 172.21.15.98:0/4160411089 shutdown_connections 2024-05-09T10:51:42.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.007+0000 7f6855b14640 1 --2- 172.21.15.98:0/4160411089 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f685015acd0 0x7f68500cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.007+0000 7f6855b14640 1 --2- 172.21.15.98:0/4160411089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68500ca530 0x7f68500ce770 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.007+0000 7f6855b14640 1 --2- 172.21.15.98:0/4160411089 >> v2:172.21.15.105:3300/0 conn(0x7f685015a8f0 0x7f685015f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.007+0000 7f6855b14640 1 -- 172.21.15.98:0/4160411089 >> 172.21.15.98:0/4160411089 conn(0x7f68500cf760 msgr2=0x7f68500cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:42.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.007+0000 7f6855b14640 1 -- 172.21.15.98:0/4160411089 shutdown_connections 2024-05-09T10:51:42.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.007+0000 7f6855b14640 1 -- 172.21.15.98:0/4160411089 wait complete. 2024-05-09T10:51:42.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.008+0000 7f6855b14640 1 Processor -- start 2024-05-09T10:51:42.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.009+0000 7f6855b14640 1 -- start start 2024-05-09T10:51:42.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.009+0000 7f6855b14640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f685015a8f0 0x7f685015d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:42.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.009+0000 7f6855b14640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f685015acd0 0x7f68500702a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:42.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.009+0000 7f6855b14640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68500707e0 0x7f685015b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:42.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.009+0000 7f6855b14640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68500cb260 con 0x7f68500707e0 2024-05-09T10:51:42.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.009+0000 7f6855b14640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68500caf80 con 0x7f685015acd0 2024-05-09T10:51:42.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.009+0000 7f6855b14640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685006abe0 con 0x7f685015a8f0 2024-05-09T10:51:42.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.010+0000 7f684ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68500707e0 0x7f685015b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:42.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.010+0000 7f684ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68500707e0 0x7f685015b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54710/0 (socket says 172.21.15.98:54710) 2024-05-09T10:51:42.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.010+0000 7f684ffff640 1 -- 172.21.15.98:0/741338790 learned_addr learned my addr 172.21.15.98:0/741338790 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:42.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.010+0000 7f684effd640 1 --2- 172.21.15.98:0/741338790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f685015acd0 0x7f68500702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:42.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.010+0000 7f684f7fe640 1 --2- 172.21.15.98:0/741338790 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f685015a8f0 0x7f685015d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:42.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.010+0000 7f684ffff640 1 -- 172.21.15.98:0/741338790 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f685015a8f0 msgr2=0x7f685015d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:42.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.010+0000 7f684ffff640 1 --2- 172.21.15.98:0/741338790 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f685015a8f0 0x7f685015d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.010+0000 7f684ffff640 1 -- 172.21.15.98:0/741338790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f685015acd0 msgr2=0x7f68500702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:42.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.010+0000 7f684ffff640 1 --2- 172.21.15.98:0/741338790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f685015acd0 0x7f68500702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.010+0000 7f684ffff640 1 -- 172.21.15.98:0/741338790 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685015bd30 con 0x7f68500707e0 2024-05-09T10:51:42.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.010+0000 7f684effd640 1 --2- 172.21.15.98:0/741338790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f685015acd0 0x7f68500702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T10:51:42.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.010+0000 7f684ffff640 1 --2- 172.21.15.98:0/741338790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68500707e0 0x7f685015b100 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f684800c5c0 tx=0x7f684800ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:42.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.010+0000 7f6837fff640 1 -- 172.21.15.98:0/741338790 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6848015070 con 0x7f68500707e0 2024-05-09T10:51:42.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.010+0000 7f6837fff640 1 -- 172.21.15.98:0/741338790 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6848011820 con 0x7f68500707e0 2024-05-09T10:51:42.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.010+0000 7f6837fff640 1 -- 172.21.15.98:0/741338790 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6848011c10 con 0x7f68500707e0 2024-05-09T10:51:42.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.010+0000 7f6855b14640 1 -- 172.21.15.98:0/741338790 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685015c010 con 0x7f68500707e0 2024-05-09T10:51:42.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.010+0000 7f6855b14640 1 -- 172.21.15.98:0/741338790 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685015c5d0 con 0x7f68500707e0 2024-05-09T10:51:42.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.011+0000 7f6855b14640 1 -- 172.21.15.98:0/741338790 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f68500b3330 con 0x7f68500707e0 2024-05-09T10:51:42.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.012+0000 7f6837fff640 1 -- 172.21.15.98:0/741338790 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f6848018070 con 0x7f68500707e0 2024-05-09T10:51:42.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.012+0000 7f6837fff640 1 --2- 172.21.15.98:0/741338790 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f683c06b6b0 0x7f683c06db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:42.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.012+0000 7f6837fff640 1 -- 172.21.15.98:0/741338790 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f684808a130 con 0x7f68500707e0 2024-05-09T10:51:42.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.012+0000 7f684f7fe640 1 --2- 172.21.15.98:0/741338790 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f683c06b6b0 0x7f683c06db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:42.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.012+0000 7f6837fff640 1 --2- 172.21.15.98:0/741338790 >> [v2:172.21.15.105:6824/3129650451,v1:172.21.15.105:6825/3129650451] conn(0x7f683c071570 0x7f683c073990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:42.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.012+0000 7f6837fff640 1 -- 172.21.15.98:0/741338790 --> [v2:172.21.15.105:6824/3129650451,v1:172.21.15.105:6825/3129650451] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f683c074040 con 0x7f683c071570 2024-05-09T10:51:42.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.012+0000 7f6837fff640 1 -- 172.21.15.98:0/741338790 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6848091050 con 0x7f68500707e0 2024-05-09T10:51:42.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.012+0000 7f684f7fe640 1 --2- 172.21.15.98:0/741338790 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f683c06b6b0 0x7f683c06db70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f68501f5250 tx=0x7f68400084f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:42.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.013+0000 7f684effd640 1 --2- 172.21.15.98:0/741338790 >> [v2:172.21.15.105:6824/3129650451,v1:172.21.15.105:6825/3129650451] conn(0x7f683c071570 0x7f683c073990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:42.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.013+0000 7f684effd640 1 --2- 172.21.15.98:0/741338790 >> [v2:172.21.15.105:6824/3129650451,v1:172.21.15.105:6825/3129650451] conn(0x7f683c071570 0x7f683c073990 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:42.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.014+0000 7f6837fff640 1 -- 172.21.15.98:0/741338790 <== osd.7 v2:172.21.15.105:6824/3129650451 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f683c074040 con 0x7f683c071570 2024-05-09T10:51:42.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.027+0000 7f6855b14640 1 -- 172.21.15.98:0/741338790 --> [v2:172.21.15.105:6824/3129650451,v1:172.21.15.105:6825/3129650451] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6850072c80 con 0x7f683c071570 2024-05-09T10:51:42.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.027+0000 7f6837fff640 1 -- 172.21.15.98:0/741338790 <== osd.7 v2:172.21.15.105:6824/3129650451 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6850072c80 con 0x7f683c071570 2024-05-09T10:51:42.020 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-05-09T10:51:42.021 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T10:51:42.021 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-05-09T10:51:42.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.027+0000 7f6855b14640 1 -- 172.21.15.98:0/741338790 >> [v2:172.21.15.105:6824/3129650451,v1:172.21.15.105:6825/3129650451] conn(0x7f683c071570 msgr2=0x7f683c073990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:42.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.027+0000 7f6855b14640 1 --2- 172.21.15.98:0/741338790 >> [v2:172.21.15.105:6824/3129650451,v1:172.21.15.105:6825/3129650451] conn(0x7f683c071570 0x7f683c073990 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.028+0000 7f6855b14640 1 -- 172.21.15.98:0/741338790 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f683c06b6b0 msgr2=0x7f683c06db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:42.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.028+0000 7f6855b14640 1 --2- 172.21.15.98:0/741338790 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f683c06b6b0 0x7f683c06db70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f68501f5250 tx=0x7f68400084f0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.028+0000 7f6855b14640 1 -- 172.21.15.98:0/741338790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68500707e0 msgr2=0x7f685015b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:42.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.028+0000 7f6855b14640 1 --2- 172.21.15.98:0/741338790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68500707e0 0x7f685015b100 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f684800c5c0 tx=0x7f684800ca80 comp rx=0 tx=0).stop 2024-05-09T10:51:42.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.028+0000 7f684ffff640 1 -- 172.21.15.98:0/741338790 reap_dead start 2024-05-09T10:51:42.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.028+0000 7f6855b14640 1 -- 172.21.15.98:0/741338790 shutdown_connections 2024-05-09T10:51:42.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.028+0000 7f6855b14640 1 -- 172.21.15.98:0/741338790 >> 172.21.15.98:0/741338790 conn(0x7f68500cf760 msgr2=0x7f685006d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:42.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.028+0000 7f6855b14640 1 -- 172.21.15.98:0/741338790 shutdown_connections 2024-05-09T10:51:42.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.028+0000 7f6855b14640 1 -- 172.21.15.98:0/741338790 wait complete. 2024-05-09T10:51:42.031 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-09T10:51:42.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.139+0000 7f8e77f42640 1 Processor -- start 2024-05-09T10:51:42.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.140+0000 7f8e77f42640 1 -- start start 2024-05-09T10:51:42.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.140+0000 7f8e77f42640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8e7015f4e0 0x7f8e700ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:42.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.140+0000 7f8e77f42640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e700afe40 0x7f8e700b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:42.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.140+0000 7f8e77f42640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8e700b07e0 0x7f8e700b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:42.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.140+0000 7f8e77f42640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e700b1190 con 0x7f8e700b07e0 2024-05-09T10:51:42.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.140+0000 7f8e77f42640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e700aeda0 con 0x7f8e7015f4e0 2024-05-09T10:51:42.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.140+0000 7f8e77f42640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e700af080 con 0x7f8e700afe40 2024-05-09T10:51:42.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.140+0000 7f8e764df640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8e700b07e0 0x7f8e700b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:42.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.141+0000 7f8e764df640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8e700b07e0 0x7f8e700b0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54718/0 (socket says 172.21.15.98:54718) 2024-05-09T10:51:42.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.141+0000 7f8e764df640 1 -- 172.21.15.98:0/213047580 learned_addr learned my addr 172.21.15.98:0/213047580 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:42.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.141+0000 7f8e754dd640 1 --2- 172.21.15.98:0/213047580 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e700afe40 0x7f8e700b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:42.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.141+0000 7f8e75cde640 1 --2- 172.21.15.98:0/213047580 >> v2:172.21.15.105:3300/0 conn(0x7f8e7015f4e0 0x7f8e700ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:42.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.141+0000 7f8e764df640 1 -- 172.21.15.98:0/213047580 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e700afe40 msgr2=0x7f8e700b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:42.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.141+0000 7f8e764df640 1 --2- 172.21.15.98:0/213047580 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e700afe40 0x7f8e700b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.141+0000 7f8e764df640 1 -- 172.21.15.98:0/213047580 >> v2:172.21.15.105:3300/0 conn(0x7f8e7015f4e0 msgr2=0x7f8e700ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:42.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.141+0000 7f8e764df640 1 --2- 172.21.15.98:0/213047580 >> v2:172.21.15.105:3300/0 conn(0x7f8e7015f4e0 0x7f8e700ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.141+0000 7f8e764df640 1 -- 172.21.15.98:0/213047580 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e700af360 con 0x7f8e700b07e0 2024-05-09T10:51:42.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.141+0000 7f8e764df640 1 --2- 172.21.15.98:0/213047580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e700b07e0 0x7f8e700b0c50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8e70070900 tx=0x7f8e6802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76563eb2573d202 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:42.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.142+0000 7f8e74cdc640 1 -- 172.21.15.98:0/213047580 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e68037070 con 0x7f8e700b07e0 2024-05-09T10:51:42.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.142+0000 7f8e74cdc640 1 -- 172.21.15.98:0/213047580 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e68032440 con 0x7f8e700b07e0 2024-05-09T10:51:42.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.142+0000 7f8e74cdc640 1 -- 172.21.15.98:0/213047580 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e68032810 con 0x7f8e700b07e0 2024-05-09T10:51:42.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.142+0000 7f8e77f42640 1 -- 172.21.15.98:0/213047580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e700b07e0 msgr2=0x7f8e700b0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:42.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.142+0000 7f8e77f42640 1 --2- 172.21.15.98:0/213047580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e700b07e0 0x7f8e700b0c50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8e70070900 tx=0x7f8e6802a140 comp rx=0 tx=0).stop 2024-05-09T10:51:42.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.142+0000 7f8e77f42640 1 -- 172.21.15.98:0/213047580 shutdown_connections 2024-05-09T10:51:42.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.142+0000 7f8e77f42640 1 --2- 172.21.15.98:0/213047580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e700b07e0 0x7f8e700b0c50 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.142+0000 7f8e77f42640 1 --2- 172.21.15.98:0/213047580 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e700afe40 0x7f8e700b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.142+0000 7f8e77f42640 1 --2- 172.21.15.98:0/213047580 >> v2:172.21.15.105:3300/0 conn(0x7f8e7015f4e0 0x7f8e700ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.142+0000 7f8e77f42640 1 -- 172.21.15.98:0/213047580 >> 172.21.15.98:0/213047580 conn(0x7f8e700b34e0 msgr2=0x7f8e700b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:42.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.142+0000 7f8e77f42640 1 -- 172.21.15.98:0/213047580 shutdown_connections 2024-05-09T10:51:42.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.142+0000 7f8e77f42640 1 -- 172.21.15.98:0/213047580 wait complete. 2024-05-09T10:51:42.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.143+0000 7f8e77f42640 1 Processor -- start 2024-05-09T10:51:42.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.144+0000 7f8e77f42640 1 -- start start 2024-05-09T10:51:42.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.144+0000 7f8e77f42640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e700afe40 0x7f8e70150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:42.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.144+0000 7f8e77f42640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e700b07e0 0x7f8e701513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:42.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.144+0000 7f8e77f42640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e7015f4e0 0x7f8e7014ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:42.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.144+0000 7f8e77f42640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e700cbcf0 con 0x7f8e7015f4e0 2024-05-09T10:51:42.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.144+0000 7f8e77f42640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e700af920 con 0x7f8e700afe40 2024-05-09T10:51:42.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.144+0000 7f8e77f42640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7015f8c0 con 0x7f8e700b07e0 2024-05-09T10:51:42.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.144+0000 7f8e764df640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e7015f4e0 0x7f8e7014ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:42.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.144+0000 7f8e764df640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e7015f4e0 0x7f8e7014ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54732/0 (socket says 172.21.15.98:54732) 2024-05-09T10:51:42.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.144+0000 7f8e764df640 1 -- 172.21.15.98:0/2136085425 learned_addr learned my addr 172.21.15.98:0/2136085425 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:42.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.144+0000 7f8e754dd640 1 --2- 172.21.15.98:0/2136085425 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e700b07e0 0x7f8e701513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:42.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.144+0000 7f8e75cde640 1 --2- 172.21.15.98:0/2136085425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e700afe40 0x7f8e70150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:42.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.145+0000 7f8e764df640 1 -- 172.21.15.98:0/2136085425 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e700b07e0 msgr2=0x7f8e701513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:42.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.145+0000 7f8e764df640 1 --2- 172.21.15.98:0/2136085425 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e700b07e0 0x7f8e701513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.145+0000 7f8e764df640 1 -- 172.21.15.98:0/2136085425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e700afe40 msgr2=0x7f8e70150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:42.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.145+0000 7f8e764df640 1 --2- 172.21.15.98:0/2136085425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e700afe40 0x7f8e70150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.145+0000 7f8e764df640 1 -- 172.21.15.98:0/2136085425 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e70150fb0 con 0x7f8e7015f4e0 2024-05-09T10:51:42.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.145+0000 7f8e764df640 1 --2- 172.21.15.98:0/2136085425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e7015f4e0 0x7f8e7014ed90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8e680090b0 tx=0x7f8e680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:42.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.145+0000 7f8e667fc640 1 -- 172.21.15.98:0/2136085425 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e68041020 con 0x7f8e7015f4e0 2024-05-09T10:51:42.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.145+0000 7f8e667fc640 1 -- 172.21.15.98:0/2136085425 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6803d640 con 0x7f8e7015f4e0 2024-05-09T10:51:42.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.145+0000 7f8e77f42640 1 -- 172.21.15.98:0/2136085425 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e7014f9d0 con 0x7f8e7015f4e0 2024-05-09T10:51:42.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.145+0000 7f8e667fc640 1 -- 172.21.15.98:0/2136085425 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e68037040 con 0x7f8e7015f4e0 2024-05-09T10:51:42.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.145+0000 7f8e77f42640 1 -- 172.21.15.98:0/2136085425 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7014ff90 con 0x7f8e7015f4e0 2024-05-09T10:51:42.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.146+0000 7f8e77f42640 1 -- 172.21.15.98:0/2136085425 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e7015e420 con 0x7f8e7015f4e0 2024-05-09T10:51:42.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.146+0000 7f8e667fc640 1 -- 172.21.15.98:0/2136085425 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f8e68044020 con 0x7f8e7015f4e0 2024-05-09T10:51:42.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.147+0000 7f8e667fc640 1 --2- 172.21.15.98:0/2136085425 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8e5406b6b0 0x7f8e5406db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:42.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.147+0000 7f8e667fc640 1 -- 172.21.15.98:0/2136085425 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8e680aa670 con 0x7f8e7015f4e0 2024-05-09T10:51:42.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.147+0000 7f8e75cde640 1 --2- 172.21.15.98:0/2136085425 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8e5406b6b0 0x7f8e5406db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:42.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.147+0000 7f8e75cde640 1 --2- 172.21.15.98:0/2136085425 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8e5406b6b0 0x7f8e5406db70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0113e0 tx=0x7f8e6c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:42.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.151+0000 7f8e667fc640 1 -- 172.21.15.98:0/2136085425 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8e6804b030 con 0x7f8e7015f4e0 2024-05-09T10:51:42.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.321+0000 7f8e77f42640 1 -- 172.21.15.98:0/2136085425 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e7015de60 con 0x7f8e5406b6b0 2024-05-09T10:51:42.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.321+0000 7f8e667fc640 1 -- 172.21.15.98:0/2136085425 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8e7015de60 con 0x7f8e5406b6b0 2024-05-09T10:51:42.315 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 7 to deep-scrub 2024-05-09T10:51:42.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.323+0000 7f8e77f42640 1 -- 172.21.15.98:0/2136085425 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8e5406b6b0 msgr2=0x7f8e5406db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:42.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.323+0000 7f8e77f42640 1 --2- 172.21.15.98:0/2136085425 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8e5406b6b0 0x7f8e5406db70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0113e0 tx=0x7f8e6c00b040 comp rx=0 tx=0).stop 2024-05-09T10:51:42.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.323+0000 7f8e77f42640 1 -- 172.21.15.98:0/2136085425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e7015f4e0 msgr2=0x7f8e7014ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:42.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.323+0000 7f8e77f42640 1 --2- 172.21.15.98:0/2136085425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e7015f4e0 0x7f8e7014ed90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8e680090b0 tx=0x7f8e680046c0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.323+0000 7f8e77f42640 1 -- 172.21.15.98:0/2136085425 shutdown_connections 2024-05-09T10:51:42.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.323+0000 7f8e77f42640 1 --2- 172.21.15.98:0/2136085425 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f8e5406b6b0 0x7f8e5406db70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.323+0000 7f8e77f42640 1 --2- 172.21.15.98:0/2136085425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e7015f4e0 0x7f8e7014ed90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.323+0000 7f8e77f42640 1 --2- 172.21.15.98:0/2136085425 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e700b07e0 0x7f8e701513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.323+0000 7f8e77f42640 1 --2- 172.21.15.98:0/2136085425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e700afe40 0x7f8e70150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.323+0000 7f8e77f42640 1 -- 172.21.15.98:0/2136085425 >> 172.21.15.98:0/2136085425 conn(0x7f8e700b34e0 msgr2=0x7f8e700b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:42.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.323+0000 7f8e77f42640 1 -- 172.21.15.98:0/2136085425 shutdown_connections 2024-05-09T10:51:42.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.323+0000 7f8e77f42640 1 -- 172.21.15.98:0/2136085425 wait complete. 2024-05-09T10:51:42.327 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T10:51:42.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.440+0000 7f48829e7640 1 Processor -- start 2024-05-09T10:51:42.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.440+0000 7f48829e7640 1 -- start start 2024-05-09T10:51:42.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.441+0000 7f48829e7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f487c06eb30 0x7f487c0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:42.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.441+0000 7f48829e7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f487c06d570 0x7f487c06ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:42.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.441+0000 7f48829e7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f487c15ddc0 0x7f487c15e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:42.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.441+0000 7f48829e7640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f487c0d0db0 con 0x7f487c15ddc0 2024-05-09T10:51:42.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.441+0000 7f48829e7640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f487c06f450 con 0x7f487c06eb30 2024-05-09T10:51:42.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.441+0000 7f48829e7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487c0d07a0 con 0x7f487c06d570 2024-05-09T10:51:42.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.441+0000 7f4880f84640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f487c15ddc0 0x7f487c15e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:42.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.441+0000 7f4880f84640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f487c15ddc0 0x7f487c15e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54748/0 (socket says 172.21.15.98:54748) 2024-05-09T10:51:42.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.441+0000 7f4880f84640 1 -- 172.21.15.98:0/4131852394 learned_addr learned my addr 172.21.15.98:0/4131852394 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:42.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.441+0000 7f487b7fe640 1 --2- 172.21.15.98:0/4131852394 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f487c06d570 0x7f487c06ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:42.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.441+0000 7f487bfff640 1 --2- 172.21.15.98:0/4131852394 >> v2:172.21.15.105:3300/0 conn(0x7f487c06eb30 0x7f487c0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:42.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.441+0000 7f4880f84640 1 -- 172.21.15.98:0/4131852394 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f487c06d570 msgr2=0x7f487c06ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:42.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.441+0000 7f4880f84640 1 --2- 172.21.15.98:0/4131852394 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f487c06d570 0x7f487c06ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.441+0000 7f4880f84640 1 -- 172.21.15.98:0/4131852394 >> v2:172.21.15.105:3300/0 conn(0x7f487c06eb30 msgr2=0x7f487c0b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:51:42.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.441+0000 7f4880f84640 1 --2- 172.21.15.98:0/4131852394 >> v2:172.21.15.105:3300/0 conn(0x7f487c06eb30 0x7f487c0b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.441+0000 7f4880f84640 1 -- 172.21.15.98:0/4131852394 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c06f9b0 con 0x7f487c15ddc0 2024-05-09T10:51:42.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.442+0000 7f4880f84640 1 --2- 172.21.15.98:0/4131852394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f487c15ddc0 0x7f487c15e270 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f487c071640 tx=0x7f487402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af181381ef00398 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:42.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.442+0000 7f487affd640 1 -- 172.21.15.98:0/4131852394 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4874036070 con 0x7f487c15ddc0 2024-05-09T10:51:42.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.442+0000 7f487affd640 1 -- 172.21.15.98:0/4131852394 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4874031600 con 0x7f487c15ddc0 2024-05-09T10:51:42.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.442+0000 7f487affd640 1 -- 172.21.15.98:0/4131852394 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48740319d0 con 0x7f487c15ddc0 2024-05-09T10:51:42.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.442+0000 7f48829e7640 1 -- 172.21.15.98:0/4131852394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f487c15ddc0 msgr2=0x7f487c15e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:42.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.442+0000 7f48829e7640 1 --2- 172.21.15.98:0/4131852394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f487c15ddc0 0x7f487c15e270 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f487c071640 tx=0x7f487402a610 comp rx=0 tx=0).stop 2024-05-09T10:51:42.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.442+0000 7f48829e7640 1 -- 172.21.15.98:0/4131852394 shutdown_connections 2024-05-09T10:51:42.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.442+0000 7f48829e7640 1 --2- 172.21.15.98:0/4131852394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f487c15ddc0 0x7f487c15e270 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.442+0000 7f48829e7640 1 --2- 172.21.15.98:0/4131852394 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f487c06d570 0x7f487c06ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.442+0000 7f48829e7640 1 --2- 172.21.15.98:0/4131852394 >> v2:172.21.15.105:3300/0 conn(0x7f487c06eb30 0x7f487c0b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.442+0000 7f48829e7640 1 -- 172.21.15.98:0/4131852394 >> 172.21.15.98:0/4131852394 conn(0x7f487c0b34e0 msgr2=0x7f487c0b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:42.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.443+0000 7f48829e7640 1 -- 172.21.15.98:0/4131852394 shutdown_connections 2024-05-09T10:51:42.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.443+0000 7f48829e7640 1 -- 172.21.15.98:0/4131852394 wait complete. 2024-05-09T10:51:42.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.444+0000 7f48829e7640 1 Processor -- start 2024-05-09T10:51:42.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.444+0000 7f48829e7640 1 -- start start 2024-05-09T10:51:42.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.445+0000 7f48829e7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f487c06d570 0x7f487c149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:42.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.445+0000 7f48829e7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f487c06eb30 0x7f487c149500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:42.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.445+0000 7f48829e7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f487c15ddc0 0x7f487c1e44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:42.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.445+0000 7f48829e7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c15e7b0 con 0x7f487c15ddc0 2024-05-09T10:51:42.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.445+0000 7f48829e7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c0d0a40 con 0x7f487c06eb30 2024-05-09T10:51:42.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.445+0000 7f48829e7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487c0ce250 con 0x7f487c06d570 2024-05-09T10:51:42.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.445+0000 7f4880f84640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f487c15ddc0 0x7f487c1e44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:42.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.445+0000 7f4880f84640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f487c15ddc0 0x7f487c1e44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54756/0 (socket says 172.21.15.98:54756) 2024-05-09T10:51:42.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.445+0000 7f4880f84640 1 -- 172.21.15.98:0/2403100417 learned_addr learned my addr 172.21.15.98:0/2403100417 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:51:42.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.445+0000 7f487bfff640 1 --2- 172.21.15.98:0/2403100417 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f487c06d570 0x7f487c149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:42.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.445+0000 7f487b7fe640 1 --2- 172.21.15.98:0/2403100417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f487c06eb30 0x7f487c149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:42.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.445+0000 7f4880f84640 1 -- 172.21.15.98:0/2403100417 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f487c06d570 msgr2=0x7f487c149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:42.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.445+0000 7f4880f84640 1 --2- 172.21.15.98:0/2403100417 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f487c06d570 0x7f487c149da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.445+0000 7f4880f84640 1 -- 172.21.15.98:0/2403100417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f487c06eb30 msgr2=0x7f487c149500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:42.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.445+0000 7f4880f84640 1 --2- 172.21.15.98:0/2403100417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f487c06eb30 0x7f487c149500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.445+0000 7f4880f84640 1 -- 172.21.15.98:0/2403100417 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c1e3fa0 con 0x7f487c15ddc0 2024-05-09T10:51:42.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.445+0000 7f4880f84640 1 --2- 172.21.15.98:0/2403100417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f487c15ddc0 0x7f487c1e44a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4874034040 tx=0x7f4874004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:42.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.445+0000 7f4878ff9640 1 -- 172.21.15.98:0/2403100417 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4874041020 con 0x7f487c15ddc0 2024-05-09T10:51:42.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.445+0000 7f4878ff9640 1 -- 172.21.15.98:0/2403100417 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48740077a0 con 0x7f487c15ddc0 2024-05-09T10:51:42.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.445+0000 7f48829e7640 1 -- 172.21.15.98:0/2403100417 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f487c0c8140 con 0x7f487c15ddc0 2024-05-09T10:51:42.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.445+0000 7f4878ff9640 1 -- 172.21.15.98:0/2403100417 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4874036040 con 0x7f487c15ddc0 2024-05-09T10:51:42.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.445+0000 7f48829e7640 1 -- 172.21.15.98:0/2403100417 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f487c0b7290 con 0x7f487c15ddc0 2024-05-09T10:51:42.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.446+0000 7f48829e7640 1 -- 172.21.15.98:0/2403100417 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f487c0b6d90 con 0x7f487c15ddc0 2024-05-09T10:51:42.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.447+0000 7f4878ff9640 1 -- 172.21.15.98:0/2403100417 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f4874044020 con 0x7f487c15ddc0 2024-05-09T10:51:42.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.447+0000 7f4878ff9640 1 --2- 172.21.15.98:0/2403100417 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f486806b680 0x7f486806db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:51:42.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.447+0000 7f4878ff9640 1 -- 172.21.15.98:0/2403100417 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f48740aadd0 con 0x7f487c15ddc0 2024-05-09T10:51:42.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.447+0000 7f487bfff640 1 --2- 172.21.15.98:0/2403100417 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f486806b680 0x7f486806db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:51:42.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.448+0000 7f487bfff640 1 --2- 172.21.15.98:0/2403100417 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f486806b680 0x7f486806db40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f486c006fd0 tx=0x7f486c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:51:42.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.451+0000 7f4878ff9640 1 -- 172.21.15.98:0/2403100417 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f487404b030 con 0x7f487c15ddc0 2024-05-09T10:51:42.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.617+0000 7f48829e7640 1 -- 172.21.15.98:0/2403100417 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f487c0b7850 con 0x7f486806b680 2024-05-09T10:51:42.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.617+0000 7f4878ff9640 1 -- 172.21.15.98:0/2403100417 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13464 (secure 0 0 0) 0x7f487c0b7850 con 0x7f486806b680 2024-05-09T10:51:42.611 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:51:42.611 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-05-09T10:51:42.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.619+0000 7f48829e7640 1 -- 172.21.15.98:0/2403100417 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f486806b680 msgr2=0x7f486806db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:42.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.619+0000 7f48829e7640 1 --2- 172.21.15.98:0/2403100417 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f486806b680 0x7f486806db40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f486c006fd0 tx=0x7f486c008040 comp rx=0 tx=0).stop 2024-05-09T10:51:42.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.619+0000 7f48829e7640 1 -- 172.21.15.98:0/2403100417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f487c15ddc0 msgr2=0x7f487c1e44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:51:42.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.619+0000 7f48829e7640 1 --2- 172.21.15.98:0/2403100417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f487c15ddc0 0x7f487c1e44a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4874034040 tx=0x7f4874004a40 comp rx=0 tx=0).stop 2024-05-09T10:51:42.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.619+0000 7f48829e7640 1 -- 172.21.15.98:0/2403100417 shutdown_connections 2024-05-09T10:51:42.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.619+0000 7f48829e7640 1 --2- 172.21.15.98:0/2403100417 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f486806b680 0x7f486806db40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.619+0000 7f48829e7640 1 --2- 172.21.15.98:0/2403100417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f487c15ddc0 0x7f487c1e44a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.619+0000 7f48829e7640 1 --2- 172.21.15.98:0/2403100417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f487c06eb30 0x7f487c149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.619+0000 7f48829e7640 1 --2- 172.21.15.98:0/2403100417 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f487c06d570 0x7f487c149da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:51:42.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.619+0000 7f48829e7640 1 -- 172.21.15.98:0/2403100417 >> 172.21.15.98:0/2403100417 conn(0x7f487c0b34e0 msgr2=0x7f487c15d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:51:42.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.619+0000 7f48829e7640 1 -- 172.21.15.98:0/2403100417 shutdown_connections 2024-05-09T10:51:42.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:51:42.619+0000 7f48829e7640 1 -- 172.21.15.98:0/2403100417 wait complete. 2024-05-09T10:51:42.624 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-05-09T10:51:41.380555+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220424,"kb_used_data":4232,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754296,"statfs":{"total":773094113280,"available":772868399104,"internally_reserved":0,"allocated":4333568,"data_stored":2394936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-6,"num_write_kb":-3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.509660"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":196,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-09T10:51:32.879116+0000","last_change":"2024-05-09T10:50:08.410364+0000","last_active":"2024-05-09T10:51:32.879116+0000","last_peered":"2024-05-09T10:51:32.879116+0000","last_clean":"2024-05-09T10:51:32.879116+0000","last_became_active":"2024-05-09T10:50:08.409457+0000","last_became_peered":"2024-05-09T10:50:08.409457+0000","last_unstale":"2024-05-09T10:51:32.879116+0000","last_undegraded":"2024-05-09T10:51:32.879116+0000","last_fullsized":"2024-05-09T10:51:32.879116+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T10:50:07.396510+0000","last_clean_scrub_stamp":"2024-05-09T10:50:07.396510+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:03.566698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":987136,"data_stored":742143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":151775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":151775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":151775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":151775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":151775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":151775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":987136,"data_stored":742143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T10:51:42.628 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-09T10:50:07.396510+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=10, tm_min=50, tm_sec=7, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=10, tm_min=51, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T10:51:42.628 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-09T10:51:49.454 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T10:51:49.458 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T10:52:02.629 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T10:52:02.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.748+0000 7f9708322640 1 Processor -- start 2024-05-09T10:52:02.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.748+0000 7f9708322640 1 -- start start 2024-05-09T10:52:02.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.749+0000 7f9708322640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f97000b40b0 0x7f970015de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:52:02.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.749+0000 7f9708322640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f970006dc50 0x7f970006c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:52:02.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.749+0000 7f9708322640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9700071ad0 con 0x7f970015a3f0 2024-05-09T10:52:02.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.749+0000 7f9708322640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97000718a0 con 0x7f97000b40b0 2024-05-09T10:52:02.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.749+0000 7f9708322640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97000af120 con 0x7f970006dc50 2024-05-09T10:52:02.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.749+0000 7f97060be640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f970015a3f0 0x7f970006d540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:59184/0 (socket says 172.21.15.98:59184) 2024-05-09T10:52:02.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.749+0000 7f97060be640 1 -- 172.21.15.98:0/2240989466 learned_addr learned my addr 172.21.15.98:0/2240989466 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:52:02.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.749+0000 7f97068bf640 1 --2- 172.21.15.98:0/2240989466 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f970006dc50 0x7f970006c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:52:02.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.749+0000 7f97058bd640 1 --2- 172.21.15.98:0/2240989466 >> v2:172.21.15.105:3300/0 conn(0x7f97000b40b0 0x7f970015de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:52:02.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.749+0000 7f97050bc640 1 -- 172.21.15.98:0/2240989466 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1262475628 0 0) 0x7f9700071ad0 con 0x7f970015a3f0 2024-05-09T10:52:02.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.750+0000 7f97050bc640 1 -- 172.21.15.98:0/2240989466 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96ec003520 con 0x7f970015a3f0 2024-05-09T10:52:02.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.750+0000 7f97050bc640 1 -- 172.21.15.98:0/2240989466 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1306239668 0 0) 0x7f96ec003520 con 0x7f970015a3f0 2024-05-09T10:52:02.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.750+0000 7f97050bc640 1 -- 172.21.15.98:0/2240989466 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f970006dc50 msgr2=0x7f970006c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:52:02.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.750+0000 7f97050bc640 1 --2- 172.21.15.98:0/2240989466 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f970006dc50 0x7f970006c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:02.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.750+0000 7f97050bc640 1 -- 172.21.15.98:0/2240989466 >> v2:172.21.15.105:3300/0 conn(0x7f97000b40b0 msgr2=0x7f970015de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:52:02.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.750+0000 7f97050bc640 1 --2- 172.21.15.98:0/2240989466 >> v2:172.21.15.105:3300/0 conn(0x7f97000b40b0 0x7f970015de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:02.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.750+0000 7f97050bc640 1 -- 172.21.15.98:0/2240989466 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9700071db0 con 0x7f970015a3f0 2024-05-09T10:52:02.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.750+0000 7f97050bc640 1 -- 172.21.15.98:0/2240989466 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f96fc001670 con 0x7f970015a3f0 2024-05-09T10:52:02.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.750+0000 7f97050bc640 1 -- 172.21.15.98:0/2240989466 >> v1:172.21.15.98:6789/0 conn(0x7f970015a3f0 legacy=0x7f970006d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:52:02.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.750+0000 7f97050bc640 1 --2- 172.21.15.98:0/2240989466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96ec0039b0 0x7f96ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:52:02.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.750+0000 7f97050bc640 1 --2- 172.21.15.98:0/2240989466 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f970006dc50 0x7f96ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:52:02.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.750+0000 7f97050bc640 1 --2- 172.21.15.98:0/2240989466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96ec004d40 0x7f96ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:52:02.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.750+0000 7f97050bc640 1 -- 172.21.15.98:0/2240989466 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97000af120 con 0x7f96ec004d40 2024-05-09T10:52:02.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.750+0000 7f97050bc640 1 -- 172.21.15.98:0/2240989466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec00b9c0 con 0x7f96ec0039b0 2024-05-09T10:52:02.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.750+0000 7f97050bc640 1 -- 172.21.15.98:0/2240989466 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec00bb40 con 0x7f970006dc50 2024-05-09T10:52:02.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.750+0000 7f97058bd640 1 --2- 172.21.15.98:0/2240989466 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f970006dc50 0x7f96ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:52:02.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.750+0000 7f97068bf640 1 --2- 172.21.15.98:0/2240989466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96ec004d40 0x7f96ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:52:02.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.750+0000 7f97060be640 1 --2- 172.21.15.98:0/2240989466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96ec0039b0 0x7f96ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:52:02.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.750+0000 7f97068bf640 1 -- 172.21.15.98:0/2240989466 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f970006dc50 msgr2=0x7f96ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:52:02.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.750+0000 7f97068bf640 1 --2- 172.21.15.98:0/2240989466 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f970006dc50 0x7f96ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:02.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.750+0000 7f97068bf640 1 -- 172.21.15.98:0/2240989466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96ec0039b0 msgr2=0x7f96ec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:52:02.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.750+0000 7f97068bf640 1 --2- 172.21.15.98:0/2240989466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96ec0039b0 0x7f96ec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:02.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.750+0000 7f97068bf640 1 -- 172.21.15.98:0/2240989466 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ec003520 con 0x7f96ec004d40 2024-05-09T10:52:02.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.751+0000 7f97068bf640 1 --2- 172.21.15.98:0/2240989466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96ec004d40 0x7f96ec0051c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009f90 tx=0x7f96ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce87de0e39eabe62 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:52:02.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.751+0000 7f97050bc640 1 -- 172.21.15.98:0/2240989466 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8009c10 con 0x7f96ec004d40 2024-05-09T10:52:02.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.751+0000 7f97050bc640 1 -- 172.21.15.98:0/2240989466 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f8036070 con 0x7f96ec004d40 2024-05-09T10:52:02.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.751+0000 7f97050bc640 1 -- 172.21.15.98:0/2240989466 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8031850 con 0x7f96ec004d40 2024-05-09T10:52:02.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.752+0000 7f9708322640 1 -- 172.21.15.98:0/2240989466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96ec004d40 msgr2=0x7f96ec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:52:02.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.752+0000 7f9708322640 1 --2- 172.21.15.98:0/2240989466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96ec004d40 0x7f96ec0051c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009f90 tx=0x7f96ec005830 comp rx=0 tx=0).stop 2024-05-09T10:52:02.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.752+0000 7f97068bf640 1 -- 172.21.15.98:0/2240989466 reap_dead start 2024-05-09T10:52:02.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.752+0000 7f9708322640 1 -- 172.21.15.98:0/2240989466 shutdown_connections 2024-05-09T10:52:02.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.752+0000 7f9708322640 1 -- 172.21.15.98:0/2240989466 >> 172.21.15.98:0/2240989466 conn(0x7f970006abe0 msgr2=0x7f970006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:52:02.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.752+0000 7f9708322640 1 -- 172.21.15.98:0/2240989466 shutdown_connections 2024-05-09T10:52:02.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.752+0000 7f9708322640 1 -- 172.21.15.98:0/2240989466 wait complete. 2024-05-09T10:52:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.753+0000 7f9708322640 1 Processor -- start 2024-05-09T10:52:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.753+0000 7f9708322640 1 -- start start 2024-05-09T10:52:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.754+0000 7f9708322640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96ec004d40 0x7f9700153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:52:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.754+0000 7f9708322640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97000b26a0 0x7f97001ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:52:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.754+0000 7f9708322640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97001ef820 0x7f9700145b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:52:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.754+0000 7f9708322640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97000aec20 con 0x7f97001ef820 2024-05-09T10:52:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.754+0000 7f9708322640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9700072400 con 0x7f96ec004d40 2024-05-09T10:52:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.754+0000 7f9708322640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97000aeda0 con 0x7f97000b26a0 2024-05-09T10:52:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.754+0000 7f97068bf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97001ef820 0x7f9700145b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:52:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.754+0000 7f97068bf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97001ef820 0x7f9700145b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41180/0 (socket says 172.21.15.98:41180) 2024-05-09T10:52:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.754+0000 7f97068bf640 1 -- 172.21.15.98:0/388580556 learned_addr learned my addr 172.21.15.98:0/388580556 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:52:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.754+0000 7f97058bd640 1 --2- 172.21.15.98:0/388580556 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97000b26a0 0x7f97001ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:52:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.754+0000 7f97060be640 1 --2- 172.21.15.98:0/388580556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96ec004d40 0x7f9700153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:52:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.754+0000 7f97068bf640 1 -- 172.21.15.98:0/388580556 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97000b26a0 msgr2=0x7f97001ef2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:52:02.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.754+0000 7f97068bf640 1 --2- 172.21.15.98:0/388580556 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97000b26a0 0x7f97001ef2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:02.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.754+0000 7f97068bf640 1 -- 172.21.15.98:0/388580556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96ec004d40 msgr2=0x7f9700153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:52:02.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.754+0000 7f97068bf640 1 --2- 172.21.15.98:0/388580556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96ec004d40 0x7f9700153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:02.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.754+0000 7f97068bf640 1 -- 172.21.15.98:0/388580556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970015bf70 con 0x7f97001ef820 2024-05-09T10:52:02.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.754+0000 7f97068bf640 1 --2- 172.21.15.98:0/388580556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97001ef820 0x7f9700145b80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f96f8033040 tx=0x7f96f8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:52:02.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.755+0000 7f96f6ffd640 1 -- 172.21.15.98:0/388580556 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8009c10 con 0x7f97001ef820 2024-05-09T10:52:02.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.755+0000 7f96f6ffd640 1 -- 172.21.15.98:0/388580556 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f8003990 con 0x7f97001ef820 2024-05-09T10:52:02.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.755+0000 7f9708322640 1 -- 172.21.15.98:0/388580556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f970015c250 con 0x7f97001ef820 2024-05-09T10:52:02.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.755+0000 7f96f6ffd640 1 -- 172.21.15.98:0/388580556 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8003d80 con 0x7f97001ef820 2024-05-09T10:52:02.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.755+0000 7f9708322640 1 -- 172.21.15.98:0/388580556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f970015c810 con 0x7f97001ef820 2024-05-09T10:52:02.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.756+0000 7f9708322640 1 -- 172.21.15.98:0/388580556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f970015ca10 con 0x7f97001ef820 2024-05-09T10:52:02.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.756+0000 7f96f6ffd640 1 -- 172.21.15.98:0/388580556 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f96f8005d10 con 0x7f97001ef820 2024-05-09T10:52:02.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.756+0000 7f96f6ffd640 1 --2- 172.21.15.98:0/388580556 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f96e406b5c0 0x7f96e406da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:52:02.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.757+0000 7f96f6ffd640 1 -- 172.21.15.98:0/388580556 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f96f80ab320 con 0x7f97001ef820 2024-05-09T10:52:02.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.757+0000 7f97060be640 1 --2- 172.21.15.98:0/388580556 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f96e406b5c0 0x7f96e406da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:52:02.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.757+0000 7f97060be640 1 --2- 172.21.15.98:0/388580556 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f96e406b5c0 0x7f96e406da80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f96fc001d30 tx=0x7f96fc007970 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:52:02.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.761+0000 7f96f6ffd640 1 -- 172.21.15.98:0/388580556 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f970015ca10 con 0x7f97001ef820 2024-05-09T10:52:02.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.927+0000 7f9708322640 1 -- 172.21.15.98:0/388580556 --> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9700069cb0 con 0x7f96e406b5c0 2024-05-09T10:52:02.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.927+0000 7f96f6ffd640 1 -- 172.21.15.98:0/388580556 <== mgr.4110 v2:172.21.15.98:6832/800971426 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13354 (secure 0 0 0) 0x7f9700069cb0 con 0x7f96e406b5c0 2024-05-09T10:52:02.921 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:52:02.921 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-05-09T10:52:02.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.929+0000 7f9708322640 1 -- 172.21.15.98:0/388580556 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f96e406b5c0 msgr2=0x7f96e406da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:52:02.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.929+0000 7f9708322640 1 --2- 172.21.15.98:0/388580556 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f96e406b5c0 0x7f96e406da80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f96fc001d30 tx=0x7f96fc007970 comp rx=0 tx=0).stop 2024-05-09T10:52:02.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.929+0000 7f9708322640 1 -- 172.21.15.98:0/388580556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97001ef820 msgr2=0x7f9700145b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:52:02.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.929+0000 7f9708322640 1 --2- 172.21.15.98:0/388580556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97001ef820 0x7f9700145b80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f96f8033040 tx=0x7f96f8009750 comp rx=0 tx=0).stop 2024-05-09T10:52:02.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.929+0000 7f9708322640 1 -- 172.21.15.98:0/388580556 shutdown_connections 2024-05-09T10:52:02.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.929+0000 7f9708322640 1 --2- 172.21.15.98:0/388580556 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f96e406b5c0 0x7f96e406da80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:02.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.929+0000 7f9708322640 1 --2- 172.21.15.98:0/388580556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97001ef820 0x7f9700145b80 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:02.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.929+0000 7f9708322640 1 --2- 172.21.15.98:0/388580556 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f97000b26a0 0x7f97001ef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:02.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.929+0000 7f9708322640 1 --2- 172.21.15.98:0/388580556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96ec004d40 0x7f9700153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:02.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.929+0000 7f9708322640 1 -- 172.21.15.98:0/388580556 >> 172.21.15.98:0/388580556 conn(0x7f970006abe0 msgr2=0x7f97000d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:52:02.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.929+0000 7f9708322640 1 -- 172.21.15.98:0/388580556 shutdown_connections 2024-05-09T10:52:02.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:02.929+0000 7f9708322640 1 -- 172.21.15.98:0/388580556 wait complete. 2024-05-09T10:52:02.933 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-05-09T10:52:01.384698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220408,"kb_used_data":4232,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754312,"statfs":{"total":773094113280,"available":772868415488,"internally_reserved":0,"allocated":4333568,"data_stored":2394936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002419"},"pg_stats":[{"pgid":"1.0","version":"34'119","reported_seq":207,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-09T10:51:43.230907+0000","last_change":"2024-05-09T10:51:43.230954+0000","last_active":"2024-05-09T10:51:43.230907+0000","last_peered":"2024-05-09T10:51:43.230907+0000","last_clean":"2024-05-09T10:51:43.230907+0000","last_became_active":"2024-05-09T10:50:08.409457+0000","last_became_peered":"2024-05-09T10:50:08.409457+0000","last_unstale":"2024-05-09T10:51:43.230907+0000","last_undegraded":"2024-05-09T10:51:43.230907+0000","last_fullsized":"2024-05-09T10:51:43.230907+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'119","last_scrub_stamp":"2024-05-09T10:51:43.230808+0000","last_deep_scrub":"34'119","last_deep_scrub_stamp":"2024-05-09T10:51:43.230808+0000","last_clean_scrub_stamp":"2024-05-09T10:51:43.230808+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:41:12.710958+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":987136,"data_stored":742143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":151775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":151775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":151775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":151775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":151775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":151775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":987136,"data_stored":742143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T10:52:02.934 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-09T10:52:03.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.039+0000 7f51a7542640 1 Processor -- start 2024-05-09T10:52:03.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.040+0000 7f51a7542640 1 -- start start 2024-05-09T10:52:03.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.040+0000 7f51a7542640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f51a00b4d80 0x7f51a00713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:52:03.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.040+0000 7f51a7542640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51a0071930 0x7f51a006d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:52:03.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.040+0000 7f51a7542640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a00b01a0 con 0x7f51a01584e0 2024-05-09T10:52:03.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.040+0000 7f51a7542640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a0071d70 con 0x7f51a00b4d80 2024-05-09T10:52:03.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.040+0000 7f51a7542640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a00b42a0 con 0x7f51a0071930 2024-05-09T10:52:03.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.041+0000 7f51a52de640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f51a01584e0 0x7f51a00af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:59188/0 (socket says 172.21.15.98:59188) 2024-05-09T10:52:03.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.041+0000 7f51a52de640 1 -- 172.21.15.98:0/1880304402 learned_addr learned my addr 172.21.15.98:0/1880304402 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:52:03.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.041+0000 7f51a4add640 1 --2- 172.21.15.98:0/1880304402 >> v2:172.21.15.98:3300/0 conn(0x7f51a00b4d80 0x7f51a00713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:52:03.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.041+0000 7f51a5adf640 1 --2- 172.21.15.98:0/1880304402 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51a0071930 0x7f51a006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:52:03.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.041+0000 7f51a4add640 1 -- 172.21.15.98:0/1880304402 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51a0071930 msgr2=0x7f51a006d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:52:03.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.041+0000 7f51a4add640 1 --2- 172.21.15.98:0/1880304402 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51a0071930 0x7f51a006d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:03.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.041+0000 7f51a4add640 1 -- 172.21.15.98:0/1880304402 >> v1:172.21.15.98:6789/0 conn(0x7f51a01584e0 legacy=0x7f51a00af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:52:03.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.041+0000 7f519ffff640 1 -- 172.21.15.98:0/1880304402 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2965116030 0 0) 0x7f51a00b01a0 con 0x7f51a01584e0 2024-05-09T10:52:03.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.041+0000 7f51a4add640 1 -- 172.21.15.98:0/1880304402 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a00b4580 con 0x7f51a00b4d80 2024-05-09T10:52:03.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.042+0000 7f51a4add640 1 --2- 172.21.15.98:0/1880304402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51a00b4d80 0x7f51a00713f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f51940097a0 tx=0x7f519402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69827fd0c722583d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:52:03.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.042+0000 7f519ffff640 1 -- 172.21.15.98:0/1880304402 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194037070 con 0x7f51a00b4d80 2024-05-09T10:52:03.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.042+0000 7f519ffff640 1 -- 172.21.15.98:0/1880304402 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519402ae50 con 0x7f51a00b4d80 2024-05-09T10:52:03.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.042+0000 7f519ffff640 1 -- 172.21.15.98:0/1880304402 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194032660 con 0x7f51a00b4d80 2024-05-09T10:52:03.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.042+0000 7f51a7542640 1 -- 172.21.15.98:0/1880304402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51a00b4d80 msgr2=0x7f51a00713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:52:03.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.042+0000 7f51a7542640 1 --2- 172.21.15.98:0/1880304402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51a00b4d80 0x7f51a00713f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f51940097a0 tx=0x7f519402a100 comp rx=0 tx=0).stop 2024-05-09T10:52:03.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.042+0000 7f51a7542640 1 -- 172.21.15.98:0/1880304402 shutdown_connections 2024-05-09T10:52:03.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.042+0000 7f51a7542640 1 --2- 172.21.15.98:0/1880304402 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51a0071930 0x7f51a006d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:03.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.042+0000 7f51a7542640 1 --2- 172.21.15.98:0/1880304402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51a00b4d80 0x7f51a00713f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:03.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.042+0000 7f51a7542640 1 -- 172.21.15.98:0/1880304402 >> 172.21.15.98:0/1880304402 conn(0x7f51a00d0860 msgr2=0x7f51a00d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:52:03.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.042+0000 7f51a7542640 1 -- 172.21.15.98:0/1880304402 shutdown_connections 2024-05-09T10:52:03.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.042+0000 7f51a7542640 1 -- 172.21.15.98:0/1880304402 wait complete. 2024-05-09T10:52:03.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.043+0000 7f51a7542640 1 Processor -- start 2024-05-09T10:52:03.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.044+0000 7f51a7542640 1 -- start start 2024-05-09T10:52:03.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.044+0000 7f51a7542640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51a0071930 0x7f51a01f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:52:03.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.044+0000 7f51a7542640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51a00b4d80 0x7f51a01e0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:52:03.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.044+0000 7f51a7542640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51a01584e0 0x7f51a01463e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:52:03.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.044+0000 7f51a7542640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a015bde0 con 0x7f51a00b4d80 2024-05-09T10:52:03.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.044+0000 7f51a7542640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a01588c0 con 0x7f51a01584e0 2024-05-09T10:52:03.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.044+0000 7f51a7542640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a00afdf0 con 0x7f51a0071930 2024-05-09T10:52:03.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.044+0000 7f51a4add640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51a00b4d80 0x7f51a01e0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:52:03.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.044+0000 7f51a4add640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51a00b4d80 0x7f51a01e0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41190/0 (socket says 172.21.15.98:41190) 2024-05-09T10:52:03.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.044+0000 7f51a4add640 1 -- 172.21.15.98:0/4122040688 learned_addr learned my addr 172.21.15.98:0/4122040688 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:52:03.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.044+0000 7f51a52de640 1 --2- 172.21.15.98:0/4122040688 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51a0071930 0x7f51a01f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:52:03.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.044+0000 7f51a5adf640 1 --2- 172.21.15.98:0/4122040688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51a01584e0 0x7f51a01463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:52:03.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.045+0000 7f51a4add640 1 -- 172.21.15.98:0/4122040688 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51a0071930 msgr2=0x7f51a01f2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:52:03.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.045+0000 7f51a4add640 1 --2- 172.21.15.98:0/4122040688 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51a0071930 0x7f51a01f2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:03.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.045+0000 7f51a4add640 1 -- 172.21.15.98:0/4122040688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51a01584e0 msgr2=0x7f51a01463e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:52:03.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.045+0000 7f51a4add640 1 --2- 172.21.15.98:0/4122040688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51a01584e0 0x7f51a01463e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:03.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.045+0000 7f51a4add640 1 -- 172.21.15.98:0/4122040688 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a0146920 con 0x7f51a00b4d80 2024-05-09T10:52:03.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.045+0000 7f51a5adf640 1 --2- 172.21.15.98:0/4122040688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51a01584e0 0x7f51a01463e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T10:52:03.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.045+0000 7f51a4add640 1 --2- 172.21.15.98:0/4122040688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51a00b4d80 0x7f51a01e0f10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5194035040 tx=0x7f51940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:52:03.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.045+0000 7f519dffb640 1 -- 172.21.15.98:0/4122040688 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194041070 con 0x7f51a00b4d80 2024-05-09T10:52:03.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.045+0000 7f519dffb640 1 -- 172.21.15.98:0/4122040688 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519403c670 con 0x7f51a00b4d80 2024-05-09T10:52:03.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.045+0000 7f51a7542640 1 -- 172.21.15.98:0/4122040688 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a01f3dc0 con 0x7f51a00b4d80 2024-05-09T10:52:03.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.045+0000 7f519dffb640 1 -- 172.21.15.98:0/4122040688 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194037040 con 0x7f51a00b4d80 2024-05-09T10:52:03.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.045+0000 7f51a7542640 1 -- 172.21.15.98:0/4122040688 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a01f3800 con 0x7f51a00b4d80 2024-05-09T10:52:03.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.046+0000 7f51a7542640 1 -- 172.21.15.98:0/4122040688 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a00673a0 con 0x7f51a00b4d80 2024-05-09T10:52:03.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.047+0000 7f519dffb640 1 -- 172.21.15.98:0/4122040688 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f519403a070 con 0x7f51a00b4d80 2024-05-09T10:52:03.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.047+0000 7f519dffb640 1 --2- 172.21.15.98:0/4122040688 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f518c06b680 0x7f518c06db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:52:03.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.047+0000 7f519dffb640 1 -- 172.21.15.98:0/4122040688 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f51940aa300 con 0x7f51a00b4d80 2024-05-09T10:52:03.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.047+0000 7f51a52de640 1 --2- 172.21.15.98:0/4122040688 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f518c06b680 0x7f518c06db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:52:03.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.048+0000 7f51a52de640 1 --2- 172.21.15.98:0/4122040688 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f518c06b680 0x7f518c06db40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f51900012e0 tx=0x7f5190001630 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:52:03.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.051+0000 7f519dffb640 1 -- 172.21.15.98:0/4122040688 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f51a0146920 con 0x7f51a00b4d80 2024-05-09T10:52:03.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.223+0000 7f51a7542640 1 -- 172.21.15.98:0/4122040688 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f51a0072f30 con 0x7f51a00b4d80 2024-05-09T10:52:03.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.228+0000 7f519dffb640 1 -- 172.21.15.98:0/4122040688 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f5194078c40 con 0x7f51a00b4d80 2024-05-09T10:52:03.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.229+0000 7f519dffb640 1 -- 172.21.15.98:0/4122040688 <== mon.0 v2:172.21.15.98:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f519402ae50 con 0x7f51a00b4d80 2024-05-09T10:52:03.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.231+0000 7f51a7542640 1 -- 172.21.15.98:0/4122040688 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f518c06b680 msgr2=0x7f518c06db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:52:03.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.231+0000 7f51a7542640 1 --2- 172.21.15.98:0/4122040688 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f518c06b680 0x7f518c06db40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f51900012e0 tx=0x7f5190001630 comp rx=0 tx=0).stop 2024-05-09T10:52:03.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.231+0000 7f51a7542640 1 -- 172.21.15.98:0/4122040688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51a00b4d80 msgr2=0x7f51a01e0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:52:03.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.231+0000 7f51a7542640 1 --2- 172.21.15.98:0/4122040688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51a00b4d80 0x7f51a01e0f10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5194035040 tx=0x7f51940046c0 comp rx=0 tx=0).stop 2024-05-09T10:52:03.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.231+0000 7f51a7542640 1 -- 172.21.15.98:0/4122040688 shutdown_connections 2024-05-09T10:52:03.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.231+0000 7f51a7542640 1 --2- 172.21.15.98:0/4122040688 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f518c06b680 0x7f518c06db40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:03.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.231+0000 7f51a7542640 1 --2- 172.21.15.98:0/4122040688 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f51a01584e0 0x7f51a01463e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:03.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.231+0000 7f51a7542640 1 --2- 172.21.15.98:0/4122040688 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51a00b4d80 0x7f51a01e0f10 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:03.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.231+0000 7f51a7542640 1 --2- 172.21.15.98:0/4122040688 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f51a0071930 0x7f51a01f2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:03.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.231+0000 7f51a7542640 1 -- 172.21.15.98:0/4122040688 >> 172.21.15.98:0/4122040688 conn(0x7f51a00d0860 msgr2=0x7f51a0158a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:52:03.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.231+0000 7f51a7542640 1 -- 172.21.15.98:0/4122040688 shutdown_connections 2024-05-09T10:52:03.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.231+0000 7f51a7542640 1 -- 172.21.15.98:0/4122040688 wait complete. 2024-05-09T10:52:03.236 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T10:52:03.236 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T10:52:03.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.342+0000 7f29649fe640 1 Processor -- start 2024-05-09T10:52:03.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.342+0000 7f29649fe640 1 -- start start 2024-05-09T10:52:03.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.343+0000 7f29649fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2960143d70 0x7f29600720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:52:03.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.343+0000 7f29649fe640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2960143b20 con 0x7f2960143740 2024-05-09T10:52:03.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.343+0000 7f29649fe640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f296015f460 con 0x7f29601443e0 2024-05-09T10:52:03.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.343+0000 7f29649fe640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29600b28c0 con 0x7f2960143d70 2024-05-09T10:52:03.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.343+0000 7f295e59c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2960143740 0x7f296015e6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:59200/0 (socket says 172.21.15.98:59200) 2024-05-09T10:52:03.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.343+0000 7f295e59c640 1 -- 172.21.15.98:0/4135792411 learned_addr learned my addr 172.21.15.98:0/4135792411 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-05-09T10:52:03.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.343+0000 7f295ed9d640 1 --2- 172.21.15.98:0/4135792411 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2960143d70 0x7f29600720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:52:03.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.343+0000 7f295d59a640 1 -- 172.21.15.98:0/4135792411 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 116264760 0 0) 0x7f2960143b20 con 0x7f2960143740 2024-05-09T10:52:03.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.343+0000 7f295d59a640 1 -- 172.21.15.98:0/4135792411 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f294c003520 con 0x7f2960143740 2024-05-09T10:52:03.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.344+0000 7f295d59a640 1 -- 172.21.15.98:0/4135792411 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4018364561 0 0) 0x7f294c003520 con 0x7f2960143740 2024-05-09T10:52:03.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.344+0000 7f295d59a640 1 -- 172.21.15.98:0/4135792411 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2960143d70 msgr2=0x7f29600720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:52:03.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.344+0000 7f295d59a640 1 --2- 172.21.15.98:0/4135792411 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2960143d70 0x7f29600720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:03.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.344+0000 7f295d59a640 1 -- 172.21.15.98:0/4135792411 >> v1:172.21.15.105:6789/0 conn(0x7f29601443e0 legacy=0x7f29600b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:52:03.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.344+0000 7f295d59a640 1 -- 172.21.15.98:0/4135792411 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29600b2ba0 con 0x7f2960143740 2024-05-09T10:52:03.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.344+0000 7f295d59a640 1 -- 172.21.15.98:0/4135792411 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1464762042 0 0) 0x7f29500019f0 con 0x7f2960143740 2024-05-09T10:52:03.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.344+0000 7f295d59a640 1 -- 172.21.15.98:0/4135792411 >> v1:172.21.15.98:6789/0 conn(0x7f2960143740 legacy=0x7f296015e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:52:03.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.344+0000 7f295d59a640 1 --2- 172.21.15.98:0/4135792411 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2960143d70 0x7f294c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:52:03.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.344+0000 7f295d59a640 1 --2- 172.21.15.98:0/4135792411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f294c003f20 0x7f294c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:52:03.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.344+0000 7f295d59a640 1 --2- 172.21.15.98:0/4135792411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f294c004d40 0x7f294c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:52:03.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.344+0000 7f295d59a640 1 -- 172.21.15.98:0/4135792411 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29600b28c0 con 0x7f294c003f20 2024-05-09T10:52:03.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.344+0000 7f295d59a640 1 -- 172.21.15.98:0/4135792411 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c00ba20 con 0x7f294c004d40 2024-05-09T10:52:03.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.344+0000 7f295d59a640 1 -- 172.21.15.98:0/4135792411 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294c00bc20 con 0x7f2960143d70 2024-05-09T10:52:03.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.344+0000 7f295dd9b640 1 --2- 172.21.15.98:0/4135792411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f294c003f20 0x7f294c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:52:03.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.344+0000 7f295e59c640 1 --2- 172.21.15.98:0/4135792411 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2960143d70 0x7f294c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:52:03.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.344+0000 7f295ed9d640 1 --2- 172.21.15.98:0/4135792411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f294c004d40 0x7f294c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:52:03.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.344+0000 7f295dd9b640 1 -- 172.21.15.98:0/4135792411 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2960143d70 msgr2=0x7f294c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:52:03.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.344+0000 7f295dd9b640 1 --2- 172.21.15.98:0/4135792411 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2960143d70 0x7f294c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:03.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.344+0000 7f295dd9b640 1 -- 172.21.15.98:0/4135792411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f294c004d40 msgr2=0x7f294c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T10:52:03.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.344+0000 7f295dd9b640 1 --2- 172.21.15.98:0/4135792411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f294c004d40 0x7f294c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:03.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.344+0000 7f295dd9b640 1 -- 172.21.15.98:0/4135792411 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c003520 con 0x7f294c003f20 2024-05-09T10:52:03.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.345+0000 7f295dd9b640 1 --2- 172.21.15.98:0/4135792411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f294c003f20 0x7f294c004800 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2948001860 tx=0x7f294c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50dc789731717a30 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:52:03.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.345+0000 7f295d59a640 1 -- 172.21.15.98:0/4135792411 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948037070 con 0x7f294c003f20 2024-05-09T10:52:03.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.345+0000 7f295d59a640 1 -- 172.21.15.98:0/4135792411 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f294802ab60 con 0x7f294c003f20 2024-05-09T10:52:03.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.345+0000 7f295d59a640 1 -- 172.21.15.98:0/4135792411 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948001430 con 0x7f294c003f20 2024-05-09T10:52:03.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.345+0000 7f29649fe640 1 -- 172.21.15.98:0/4135792411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f294c003f20 msgr2=0x7f294c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:52:03.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.345+0000 7f29649fe640 1 --2- 172.21.15.98:0/4135792411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f294c003f20 0x7f294c004800 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2948001860 tx=0x7f294c005830 comp rx=0 tx=0).stop 2024-05-09T10:52:03.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.345+0000 7f295ed9d640 1 -- 172.21.15.98:0/4135792411 reap_dead start 2024-05-09T10:52:03.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.346+0000 7f29649fe640 1 -- 172.21.15.98:0/4135792411 shutdown_connections 2024-05-09T10:52:03.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.346+0000 7f29649fe640 1 -- 172.21.15.98:0/4135792411 >> 172.21.15.98:0/4135792411 conn(0x7f296006abe0 msgr2=0x7f296006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:52:03.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.346+0000 7f29649fe640 1 -- 172.21.15.98:0/4135792411 shutdown_connections 2024-05-09T10:52:03.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.346+0000 7f29649fe640 1 -- 172.21.15.98:0/4135792411 wait complete. 2024-05-09T10:52:03.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.347+0000 7f29649fe640 1 Processor -- start 2024-05-09T10:52:03.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.347+0000 7f29649fe640 1 -- start start 2024-05-09T10:52:03.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.348+0000 7f29649fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f294c003f20 0x7f29601e4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:52:03.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.348+0000 7f29649fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f296013f8f0 0x7f296013fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:52:03.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.348+0000 7f29649fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29601f02a0 0x7f29601f0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:52:03.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.348+0000 7f29649fe640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2960072430 con 0x7f296013f8f0 2024-05-09T10:52:03.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.348+0000 7f29649fe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29600722b0 con 0x7f29601f02a0 2024-05-09T10:52:03.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.348+0000 7f29649fe640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296015edf0 con 0x7f294c003f20 2024-05-09T10:52:03.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.348+0000 7f295dd9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f296013f8f0 0x7f296013fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:52:03.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.348+0000 7f295dd9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f296013f8f0 0x7f296013fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41218/0 (socket says 172.21.15.98:41218) 2024-05-09T10:52:03.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.348+0000 7f295dd9b640 1 -- 172.21.15.98:0/3413026008 learned_addr learned my addr 172.21.15.98:0/3413026008 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-05-09T10:52:03.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.348+0000 7f295dd9b640 1 -- 172.21.15.98:0/3413026008 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f294c003f20 msgr2=0x7f29601e4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:52:03.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.348+0000 7f295dd9b640 1 --2- 172.21.15.98:0/3413026008 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f294c003f20 0x7f29601e4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:03.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.348+0000 7f295dd9b640 1 -- 172.21.15.98:0/3413026008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29601f02a0 msgr2=0x7f29601f0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:52:03.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.348+0000 7f295dd9b640 1 --2- 172.21.15.98:0/3413026008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29601f02a0 0x7f29601f0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:03.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.348+0000 7f295dd9b640 1 -- 172.21.15.98:0/3413026008 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29601e53c0 con 0x7f296013f8f0 2024-05-09T10:52:03.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.348+0000 7f295dd9b640 1 --2- 172.21.15.98:0/3413026008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f296013f8f0 0x7f296013fd30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f29480018c0 tx=0x7f2948001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:52:03.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.348+0000 7f2956ffd640 1 -- 172.21.15.98:0/3413026008 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294803f020 con 0x7f296013f8f0 2024-05-09T10:52:03.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.348+0000 7f2956ffd640 1 -- 172.21.15.98:0/3413026008 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f2948005270 con 0x7f296013f8f0 2024-05-09T10:52:03.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.348+0000 7f29649fe640 1 -- 172.21.15.98:0/3413026008 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29601f1030 con 0x7f296013f8f0 2024-05-09T10:52:03.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.348+0000 7f2956ffd640 1 -- 172.21.15.98:0/3413026008 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948037040 con 0x7f296013f8f0 2024-05-09T10:52:03.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.348+0000 7f29649fe640 1 -- 172.21.15.98:0/3413026008 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29601f15f0 con 0x7f296013f8f0 2024-05-09T10:52:03.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.349+0000 7f29649fe640 1 -- 172.21.15.98:0/3413026008 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2960069cb0 con 0x7f296013f8f0 2024-05-09T10:52:03.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.350+0000 7f2956ffd640 1 -- 172.21.15.98:0/3413026008 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) v1 ==== 89933+0+0 (secure 0 0 0) 0x7f2948042020 con 0x7f296013f8f0 2024-05-09T10:52:03.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.350+0000 7f2956ffd640 1 --2- 172.21.15.98:0/3413026008 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f294006b680 0x7f294006db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T10:52:03.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.350+0000 7f2956ffd640 1 -- 172.21.15.98:0/3413026008 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f29480aa040 con 0x7f296013f8f0 2024-05-09T10:52:03.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.350+0000 7f295e59c640 1 --2- 172.21.15.98:0/3413026008 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f294006b680 0x7f294006db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T10:52:03.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.351+0000 7f295e59c640 1 --2- 172.21.15.98:0/3413026008 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f294006b680 0x7f294006db40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f29500012e0 tx=0x7f2950001250 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T10:52:03.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.354+0000 7f2956ffd640 1 -- 172.21.15.98:0/3413026008 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2948049030 con 0x7f296013f8f0 2024-05-09T10:52:03.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.560+0000 7f29649fe640 1 -- 172.21.15.98:0/3413026008 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f296015e450 con 0x7f296013f8f0 2024-05-09T10:52:03.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.561+0000 7f2956ffd640 1 -- 172.21.15.98:0/3413026008 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f2948078860 con 0x7f296013f8f0 2024-05-09T10:52:03.554 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-05-09T10:52:03.554 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":16,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/2332373437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2332373437},{"type":"v1","addr":"172.21.15.105:6834","nonce":2332373437}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4849,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/4092257628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":4092257628},{"type":"v1","addr":"172.21.15.98:6836","nonce":4092257628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4855,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3957569958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3957569958},{"type":"v1","addr":"172.21.15.105:6835","nonce":3957569958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/1423896295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":1423896295},{"type":"v1","addr":"172.21.15.98:6837","nonce":1423896295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":16}],"filesystems":[]} 2024-05-09T10:52:03.554 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 16 2024-05-09T10:52:03.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.563+0000 7f29649fe640 1 -- 172.21.15.98:0/3413026008 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f294006b680 msgr2=0x7f294006db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:52:03.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.563+0000 7f29649fe640 1 --2- 172.21.15.98:0/3413026008 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f294006b680 0x7f294006db40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f29500012e0 tx=0x7f2950001250 comp rx=0 tx=0).stop 2024-05-09T10:52:03.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.563+0000 7f29649fe640 1 -- 172.21.15.98:0/3413026008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f296013f8f0 msgr2=0x7f296013fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T10:52:03.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.563+0000 7f29649fe640 1 --2- 172.21.15.98:0/3413026008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f296013f8f0 0x7f296013fd30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f29480018c0 tx=0x7f2948001230 comp rx=0 tx=0).stop 2024-05-09T10:52:03.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.563+0000 7f29649fe640 1 -- 172.21.15.98:0/3413026008 shutdown_connections 2024-05-09T10:52:03.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.563+0000 7f29649fe640 1 --2- 172.21.15.98:0/3413026008 >> [v2:172.21.15.98:6832/800971426,v1:172.21.15.98:6833/800971426] conn(0x7f294006b680 0x7f294006db40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:03.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.563+0000 7f29649fe640 1 --2- 172.21.15.98:0/3413026008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29601f02a0 0x7f29601f0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:03.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.563+0000 7f29649fe640 1 --2- 172.21.15.98:0/3413026008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f296013f8f0 0x7f296013fd30 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:03.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.563+0000 7f29649fe640 1 --2- 172.21.15.98:0/3413026008 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f294c003f20 0x7f29601e4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T10:52:03.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.563+0000 7f29649fe640 1 -- 172.21.15.98:0/3413026008 >> 172.21.15.98:0/3413026008 conn(0x7f296006abe0 msgr2=0x7f2960140f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T10:52:03.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.563+0000 7f29649fe640 1 -- 172.21.15.98:0/3413026008 shutdown_connections 2024-05-09T10:52:03.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-05-09T10:52:03.563+0000 7f29649fe640 1 -- 172.21.15.98:0/3413026008 wait complete. 2024-05-09T10:52:03.567 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-09T10:52:03.567 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-09T10:52:03.567 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-09T10:52:03.568 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T10:52:03.577 INFO:tasks.ceph.mds.a:Stopped 2024-05-09T10:52:03.577 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-09T10:52:03.577 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T10:52:03.626 INFO:tasks.ceph.mds.c:Stopped 2024-05-09T10:52:03.627 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-09T10:52:03.627 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T10:52:03.639 INFO:tasks.ceph.mds.b:Stopped 2024-05-09T10:52:03.639 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-09T10:52:03.639 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T10:52:03.692 INFO:tasks.ceph.mds.d:Stopped 2024-05-09T10:52:03.692 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-09T10:52:03.693 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-09T10:52:03.693 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T10:52:03.724 INFO:tasks.ceph.osd.0:Stopped 2024-05-09T10:52:03.725 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-09T10:52:03.725 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T10:52:03.789 INFO:tasks.ceph.osd.1:Stopped 2024-05-09T10:52:03.789 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-09T10:52:03.789 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T10:52:03.857 INFO:tasks.ceph.osd.2:Stopped 2024-05-09T10:52:03.857 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-09T10:52:03.858 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T10:52:03.926 INFO:tasks.ceph.osd.3:Stopped 2024-05-09T10:52:03.926 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-09T10:52:03.926 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T10:52:03.969 INFO:tasks.ceph.osd.4:Stopped 2024-05-09T10:52:03.969 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-09T10:52:03.969 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T10:52:04.007 INFO:tasks.ceph.osd.5:Stopped 2024-05-09T10:52:04.007 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-09T10:52:04.007 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T10:52:04.076 INFO:tasks.ceph.osd.6:Stopped 2024-05-09T10:52:04.076 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-09T10:52:04.077 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T10:52:04.144 INFO:tasks.ceph.osd.7:Stopped 2024-05-09T10:52:04.144 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-09T10:52:04.144 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-09T10:52:04.144 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T10:52:04.170 INFO:tasks.ceph.mgr.y:Stopped 2024-05-09T10:52:04.171 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-09T10:52:04.171 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T10:52:04.210 INFO:tasks.ceph.mgr.x:Stopped 2024-05-09T10:52:04.210 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-09T10:52:04.210 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-09T10:52:04.210 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T10:52:04.222 INFO:tasks.ceph.mon.a:Stopped 2024-05-09T10:52:04.222 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-09T10:52:04.222 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T10:52:04.262 INFO:tasks.ceph.mon.b:Stopped 2024-05-09T10:52:04.262 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-09T10:52:04.262 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T10:52:04.314 INFO:tasks.ceph.mon.c:Stopped 2024-05-09T10:52:04.314 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-09T10:52:04.314 DEBUG:teuthology.orchestra.run.smithi098:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-05-09T10:52:04.345 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi098.front.sepia.ceph.com 2024-05-09T10:52:04.345 DEBUG:teuthology.orchestra.run.smithi098:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-09T10:52:04.546 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi098.front.sepia.ceph.com 2024-05-09T10:52:04.546 DEBUG:teuthology.orchestra.run.smithi098:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-09T10:52:04.652 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi098.front.sepia.ceph.com 2024-05-09T10:52:04.652 DEBUG:teuthology.orchestra.run.smithi098:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-09T10:52:04.709 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi098.front.sepia.ceph.com 2024-05-09T10:52:04.710 DEBUG:teuthology.orchestra.run.smithi098:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-09T10:52:04.759 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi105.front.sepia.ceph.com 2024-05-09T10:52:04.759 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-09T10:52:04.965 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi105.front.sepia.ceph.com 2024-05-09T10:52:04.965 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-09T10:52:05.040 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi105.front.sepia.ceph.com 2024-05-09T10:52:05.041 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-09T10:52:05.091 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi105.front.sepia.ceph.com 2024-05-09T10:52:05.091 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-09T10:52:05.149 INFO:tasks.ceph:Archiving mon data... 2024-05-09T10:52:05.150 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2024-05-09_10:23:18-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7699683/data/mon.a.tgz 2024-05-09T10:52:05.150 DEBUG:teuthology.orchestra.run.smithi098:> mktemp 2024-05-09T10:52:05.167 INFO:teuthology.orchestra.run.smithi098.stdout:/tmp/tmp.EpS3B5tIdW 2024-05-09T10:52:05.167 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.EpS3B5tIdW 2024-05-09T10:52:05.299 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 0666 /tmp/tmp.EpS3B5tIdW 2024-05-09T10:52:05.382 DEBUG:teuthology.orchestra.remote:smithi098:/tmp/tmp.EpS3B5tIdW is 340KB 2024-05-09T10:52:05.477 DEBUG:teuthology.orchestra.run.smithi098:> rm -fr /tmp/tmp.EpS3B5tIdW 2024-05-09T10:52:05.492 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2024-05-09_10:23:18-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7699683/data/mon.b.tgz 2024-05-09T10:52:05.493 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-05-09T10:52:05.510 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.3t3gGElloM 2024-05-09T10:52:05.510 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.3t3gGElloM 2024-05-09T10:52:05.679 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.3t3gGElloM 2024-05-09T10:52:05.762 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.3t3gGElloM is 467KB 2024-05-09T10:52:05.832 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.3t3gGElloM 2024-05-09T10:52:05.847 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2024-05-09_10:23:18-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7699683/data/mon.c.tgz 2024-05-09T10:52:05.847 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-05-09T10:52:05.902 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.VuEAr9IE2a 2024-05-09T10:52:05.903 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.VuEAr9IE2a 2024-05-09T10:52:06.046 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.VuEAr9IE2a 2024-05-09T10:52:06.130 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.VuEAr9IE2a is 467KB 2024-05-09T10:52:06.198 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.VuEAr9IE2a 2024-05-09T10:52:06.213 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-09T10:52:06.214 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-09T10:52:06.216 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-09T10:52:06.298 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-09T10:52:06.333 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-09T10:52:06.333 INFO:tasks.ceph:Archiving crash dumps... 2024-05-09T10:52:06.335 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-05-09_10:23:18-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7699683/remote/smithi098/crash 2024-05-09T10:52:06.336 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-09T10:52:06.364 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-05-09_10:23:18-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7699683/remote/smithi105/crash 2024-05-09T10:52:06.365 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-09T10:52:06.397 INFO:tasks.ceph:Shutting down logrotate 2024-05-09T10:52:06.398 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T10:52:06.408 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T10:52:06.497 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-09T10:52:06.523 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-09T10:52:06.547 INFO:tasks.ceph:Compressing logs... 2024-05-09T10:52:06.547 DEBUG:teuthology.orchestra.run.smithi098:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T10:52:06.565 DEBUG:teuthology.orchestra.run.smithi105:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T10:52:06.587 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50484.log 2024-05-09T10:52:06.587 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46919.log 2024-05-09T10:52:06.637 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43185.log 2024-05-09T10:52:06.637 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42204.log 2024-05-09T10:52:06.637 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.1.43185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35671.log 2024-05-09T10:52:06.638 INFO:teuthology.orchestra.run.smithi105.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43185.log.gz 2024-05-09T10:52:06.638 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.1.42204.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-09T10:52:06.638 INFO:teuthology.orchestra.run.smithi105.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42204.log.gz 2024-05-09T10:52:06.638 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43189.log 2024-05-09T10:52:06.638 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-09T10:52:06.638 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-09T10:52:06.638 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35671.log: /var/log/ceph/ceph.log: /var/log/ceph/ceph-client.1.43189.log: /var/log/ceph/ceph.audit.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43189.log.gz 2024-05-09T10:52:06.638 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-09T10:52:06.638 INFO:teuthology.orchestra.run.smithi105.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35671.log.gz 2024-05-09T10:52:06.638 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-09T10:52:06.638 INFO:teuthology.orchestra.run.smithi105.stderr: 91.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-09T10:52:06.638 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42592.log 2024-05-09T10:52:06.638 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-09T10:52:06.638 INFO:teuthology.orchestra.run.smithi105.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-09T10:52:06.638 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.3.42592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35499.log 2024-05-09T10:52:06.638 INFO:teuthology.orchestra.run.smithi105.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42592.log.gz 2024-05-09T10:52:06.639 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42199.log 2024-05-09T10:52:06.639 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.4.log: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-09T10:52:06.639 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35751.log 2024-05-09T10:52:06.639 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.1.42199.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35499.log.gz 2024-05-09T10:52:06.639 INFO:teuthology.orchestra.run.smithi105.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42199.log.gz 2024-05-09T10:52:06.639 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-09T10:52:06.639 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35751.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-09T10:52:06.639 INFO:teuthology.orchestra.run.smithi105.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.35751.log.gz 2024-05-09T10:52:06.639 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-09T10:52:06.639 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35459.log 2024-05-09T10:52:06.639 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42596.log 2024-05-09T10:52:06.639 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35711.log 2024-05-09T10:52:06.639 INFO:teuthology.orchestra.run.smithi105.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.35459.log.gz 2024-05-09T10:52:06.639 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.3.42596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35540.log 2024-05-09T10:52:06.639 INFO:teuthology.orchestra.run.smithi105.stderr: 0.0%/var/log/ceph/ceph-client.admin.35711.log: -- replaced with /var/log/ceph/ceph-client.3.42596.log.gz 2024-05-09T10:52:06.639 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35711.log.gz 2024-05-09T10:52:06.640 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35631.log 2024-05-09T10:52:06.640 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-05-09T10:52:06.640 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35631.log: /var/log/ceph/ceph-client.admin.35540.log: 87.8% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35631.log.gz 2024-05-09T10:52:06.640 INFO:teuthology.orchestra.run.smithi105.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-09T10:52:06.640 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35540.log.gz 2024-05-09T10:52:06.640 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-09T10:52:06.640 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35584.log 2024-05-09T10:52:06.640 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.35584.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35584.log.gz 2024-05-09T10:52:06.667 INFO:teuthology.orchestra.run.smithi105.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-09T10:52:06.695 INFO:teuthology.orchestra.run.smithi105.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-09T10:52:06.701 INFO:teuthology.orchestra.run.smithi105.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-09T10:52:06.701 INFO:teuthology.orchestra.run.smithi105.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-09T10:52:06.722 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51695.log 2024-05-09T10:52:06.722 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50484.log: /var/log/ceph/ceph-client.admin.46919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47207.log 2024-05-09T10:52:06.722 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50484.log.gz 2024-05-09T10:52:06.723 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51695.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46919.log.gz 2024-05-09T10:52:06.723 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45992.log 2024-05-09T10:52:06.723 INFO:teuthology.orchestra.run.smithi098.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51695.log.gz 2024-05-09T10:52:06.723 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41643.log 2024-05-09T10:52:06.723 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45992.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47207.log.gz 2024-05-09T10:52:06.723 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49440.log 2024-05-09T10:52:06.723 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41643.log: gzip 89.0% -5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.45992.log.gz 2024-05-09T10:52:06.723 INFO:teuthology.orchestra.run.smithi098.stderr: /var/log/ceph/ceph-client.admin.48644.log 2024-05-09T10:52:06.723 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7%/var/log/ceph/ceph-client.admin.49440.log: -- replaced with /var/log/ceph/ceph-client.admin.41643.log.gz 2024-05-09T10:52:06.723 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45278.log 2024-05-09T10:52:06.723 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48644.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49440.log.gz 2024-05-09T10:52:06.723 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42977.log 2024-05-09T10:52:06.724 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45278.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48644.log.gz 2024-05-09T10:52:06.724 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47998.log 2024-05-09T10:52:06.724 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45278.log.gz 2024-05-09T10:52:06.724 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50358.log 2024-05-09T10:52:06.724 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47998.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42977.log.gz 2024-05-09T10:52:06.724 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48209.log 2024-05-09T10:52:06.724 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48083.log 2024-05-09T10:52:06.724 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47998.log.gz 2024-05-09T10:52:06.724 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45530.log 2024-05-09T10:52:06.724 INFO:teuthology.orchestra.run.smithi098.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50358.log.gz 2024-05-09T10:52:06.724 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48083.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48209.log.gz 2024-05-09T10:52:06.724 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35376.log 2024-05-09T10:52:06.724 INFO:teuthology.orchestra.run.smithi098.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48083.log.gz 2024-05-09T10:52:06.724 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48686.log 2024-05-09T10:52:06.725 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35376.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45530.log.gz 2024-05-09T10:52:06.725 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42636.log 2024-05-09T10:52:06.725 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52325.log 2024-05-09T10:52:06.725 INFO:teuthology.orchestra.run.smithi098.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35376.log.gz 2024-05-09T10:52:06.725 INFO:teuthology.orchestra.run.smithi098.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48686.log.gz 2024-05-09T10:52:06.725 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46877.log 2024-05-09T10:52:06.725 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52325.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42636.log.gz 2024-05-09T10:52:06.725 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50752.log 2024-05-09T10:52:06.725 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52325.log.gz 2024-05-09T10:52:06.725 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42422.log 2024-05-09T10:52:06.725 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45194.log 2024-05-09T10:52:06.725 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46877.log.gz 2024-05-09T10:52:06.725 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42422.log: gzip 87.8% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.50752.log.gz 2024-05-09T10:52:06.725 INFO:teuthology.orchestra.run.smithi098.stderr: -- /var/log/ceph/ceph-client.admin.43556.log 2024-05-09T10:52:06.726 INFO:teuthology.orchestra.run.smithi098.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42422.log.gz 2024-05-09T10:52:06.726 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46961.log 2024-05-09T10:52:06.726 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43556.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45194.log.gz 2024-05-09T10:52:06.726 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51989.log 2024-05-09T10:52:06.726 INFO:teuthology.orchestra.run.smithi098.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43556.log.gz 2024-05-09T10:52:06.726 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41373.log 2024-05-09T10:52:06.726 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46961.log.gz/var/log/ceph/ceph-client.admin.51989.log: 2024-05-09T10:52:06.726 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47459.log 2024-05-09T10:52:06.726 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51989.log.gz 2024-05-09T10:52:06.726 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44144.log 2024-05-09T10:52:06.726 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47459.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-09T10:52:06.726 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41373.log.gz 2024-05-09T10:52:06.726 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44144.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47459.log.gz 2024-05-09T10:52:06.727 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52367.log 2024-05-09T10:52:06.727 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47956.log 2024-05-09T10:52:06.727 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52367.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44144.log.gz 2024-05-09T10:52:06.727 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45656.log 2024-05-09T10:52:06.727 INFO:teuthology.orchestra.run.smithi098.stderr: 84.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-09T10:52:06.727 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52367.log.gz 2024-05-09T10:52:06.727 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46271.log 2024-05-09T10:52:06.727 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47956.log.gz 2024-05-09T10:52:06.727 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45572.log 2024-05-09T10:52:06.727 INFO:teuthology.orchestra.run.smithi098.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45656.log.gz 2024-05-09T10:52:06.727 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.46271.log: /var/log/ceph/ceph-client.admin.48558.log 2024-05-09T10:52:06.727 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45572.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-09T10:52:06.727 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46271.log.gz 2024-05-09T10:52:06.727 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45572.log.gz 2024-05-09T10:52:06.728 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45065.log 2024-05-09T10:52:06.728 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-osd.2.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48558.log.gz 2024-05-09T10:52:06.728 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50878.log 2024-05-09T10:52:06.728 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.2.45065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45614.log 2024-05-09T10:52:06.728 INFO:teuthology.orchestra.run.smithi098.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45065.log.gz 2024-05-09T10:52:06.728 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49064.log 2024-05-09T10:52:06.728 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47875.log 2024-05-09T10:52:06.728 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50878.log.gz 2024-05-09T10:52:06.728 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49064.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45614.log.gz 2024-05-09T10:52:06.728 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43317.log 2024-05-09T10:52:06.728 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47753.log 2024-05-09T10:52:06.728 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49064.log.gz 2024-05-09T10:52:06.728 INFO:teuthology.orchestra.run.smithi098.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47875.log.gz 2024-05-09T10:52:06.728 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46397.log 2024-05-09T10:52:06.729 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47753.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43317.log.gz 2024-05-09T10:52:06.729 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42044.log 2024-05-09T10:52:06.729 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46397.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47753.log.gz 2024-05-09T10:52:06.729 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44102.log 2024-05-09T10:52:06.729 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42044.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46397.log.gz 2024-05-09T10:52:06.729 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42254.log 2024-05-09T10:52:06.729 INFO:teuthology.orchestra.run.smithi098.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42044.log.gz 2024-05-09T10:52:06.729 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50920.log 2024-05-09T10:52:06.729 INFO:teuthology.orchestra.run.smithi098.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44102.log.gz 2024-05-09T10:52:06.729 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47081.log 2024-05-09T10:52:06.729 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50920.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42254.log.gz 2024-05-09T10:52:06.729 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50400.log 2024-05-09T10:52:06.730 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50920.log.gz 2024-05-09T10:52:06.730 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49958.log 2024-05-09T10:52:06.730 INFO:teuthology.orchestra.run.smithi098.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47081.log.gz 2024-05-09T10:52:06.730 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46703.log 2024-05-09T10:52:06.730 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49958.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-09T10:52:06.730 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50400.log.gz 2024-05-09T10:52:06.730 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46703.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49958.log.gz 2024-05-09T10:52:06.730 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42973.log 2024-05-09T10:52:06.730 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46703.log.gz 2024-05-09T10:52:06.730 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46559.log 2024-05-09T10:52:06.730 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42380.log 2024-05-09T10:52:06.730 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46559.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42973.log.gz 2024-05-09T10:52:06.731 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41685.log 2024-05-09T10:52:06.731 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46559.log.gz 2024-05-09T10:52:06.731 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50794.log 2024-05-09T10:52:06.731 INFO:teuthology.orchestra.run.smithi098.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42380.log.gz 2024-05-09T10:52:06.731 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-09T10:52:06.731 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50710.log 2024-05-09T10:52:06.731 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41685.log.gz 2024-05-09T10:52:06.731 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46661.log 2024-05-09T10:52:06.731 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50794.log.gz 2024-05-09T10:52:06.731 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52427.log 2024-05-09T10:52:06.731 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46661.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50710.log.gz 2024-05-09T10:52:06.731 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43598.log 2024-05-09T10:52:06.731 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46661.log.gz 2024-05-09T10:52:06.732 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46793.log 2024-05-09T10:52:06.732 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52427.log.gz 2024-05-09T10:52:06.732 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42975.log 2024-05-09T10:52:06.732 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51317.log 2024-05-09T10:52:06.732 INFO:teuthology.orchestra.run.smithi098.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43598.log.gz 2024-05-09T10:52:06.732 INFO:teuthology.orchestra.run.smithi098.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46793.log.gz 2024-05-09T10:52:06.732 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49106.log 2024-05-09T10:52:06.732 INFO:teuthology.orchestra.run.smithi098.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42975.log.gz 2024-05-09T10:52:06.732 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51317.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-09T10:52:06.732 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49106.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51317.log.gz 2024-05-09T10:52:06.733 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49874.log 2024-05-09T10:52:06.733 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51020.log 2024-05-09T10:52:06.733 INFO:teuthology.orchestra.run.smithi098.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49106.log.gz 2024-05-09T10:52:06.733 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45404.log 2024-05-09T10:52:06.733 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49874.log.gz 2024-05-09T10:52:06.733 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51107.log 2024-05-09T10:52:06.733 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51020.log: /var/log/ceph/ceph-client.admin.45404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47669.log 2024-05-09T10:52:06.733 INFO:teuthology.orchestra.run.smithi098.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51020.log.gz 2024-05-09T10:52:06.733 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45404.log.gz 2024-05-09T10:52:06.733 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51863.log 2024-05-09T10:52:06.733 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47669.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51107.log.gz 2024-05-09T10:52:06.733 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42212.log 2024-05-09T10:52:06.734 INFO:teuthology.orchestra.run.smithi098.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47669.log.gz 2024-05-09T10:52:06.734 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47375.log 2024-05-09T10:52:06.734 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51863.log: /var/log/ceph/ceph-client.admin.42212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47585.log 2024-05-09T10:52:06.734 INFO:teuthology.orchestra.run.smithi098.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51863.log.gz 2024-05-09T10:52:06.734 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47375.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42212.log.gz 2024-05-09T10:52:06.734 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43311.log 2024-05-09T10:52:06.734 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47375.log.gz 2024-05-09T10:52:06.734 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46439.log 2024-05-09T10:52:06.734 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47585.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47585.log.gz 2024-05-09T10:52:06.734 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49682.log 2024-05-09T10:52:06.734 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41770.log 2024-05-09T10:52:06.735 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46439.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43311.log.gz 2024-05-09T10:52:06.735 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49682.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46439.log.gz 2024-05-09T10:52:06.735 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49626.log 2024-05-09T10:52:06.735 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49682.log.gz 2024-05-09T10:52:06.735 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48896.log 2024-05-09T10:52:06.735 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41770.log: /var/log/ceph/ceph-client.admin.49626.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41770.log.gz 2024-05-09T10:52:06.735 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-09T10:52:06.735 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49626.log.gz 2024-05-09T10:52:06.735 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49542.log 2024-05-09T10:52:06.735 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48896.log: /var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42980.log 2024-05-09T10:52:06.735 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48896.log.gz 2024-05-09T10:52:06.735 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48040.log 2024-05-09T10:52:06.736 INFO:teuthology.orchestra.run.smithi098.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49542.log.gz 2024-05-09T10:52:06.736 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42980.log: 91.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-09T10:52:06.736 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48980.log 2024-05-09T10:52:06.736 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42980.log.gz 2024-05-09T10:52:06.736 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49584.log 2024-05-09T10:52:06.736 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48040.log: /var/log/ceph/ceph-client.admin.48980.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-09T10:52:06.736 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48040.log.gz 2024-05-09T10:52:06.736 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49584.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48980.log.gz 2024-05-09T10:52:06.736 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-09T10:52:06.736 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49584.log.gz 2024-05-09T10:52:06.736 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48167.log 2024-05-09T10:52:06.736 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42971.log 2024-05-09T10:52:06.736 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45824.log 2024-05-09T10:52:06.737 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42971.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48167.log.gz 2024-05-09T10:52:06.737 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48770.log 2024-05-09T10:52:06.737 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0%/var/log/ceph/ceph-client.admin.45824.log: -- replaced with /var/log/ceph/ceph-client.admin.42971.log.gz 2024-05-09T10:52:06.737 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-09T10:52:06.737 INFO:teuthology.orchestra.run.smithi098.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45824.log.gz 2024-05-09T10:52:06.737 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52115.log 2024-05-09T10:52:06.737 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4%/var/log/ceph/ceph-osd.0.log: -- replaced with /var/log/ceph/ceph-client.admin.48770.log.gz 2024-05-09T10:52:06.737 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48421.log 2024-05-09T10:52:06.737 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43514.log 2024-05-09T10:52:06.737 INFO:teuthology.orchestra.run.smithi098.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52115.log.gz 2024-05-09T10:52:06.737 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47039.log 2024-05-09T10:52:06.738 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2%/var/log/ceph/ceph-client.admin.43514.log: -- replaced with /var/log/ceph/ceph-client.admin.48421.log.gz 2024-05-09T10:52:06.738 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45446.log 2024-05-09T10:52:06.738 INFO:teuthology.orchestra.run.smithi098.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43514.log.gz 2024-05-09T10:52:06.738 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48465.log 2024-05-09T10:52:06.738 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47039.log: /var/log/ceph/ceph-client.admin.45446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48377.log 2024-05-09T10:52:06.738 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48465.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47039.log.gz 2024-05-09T10:52:06.738 INFO:teuthology.orchestra.run.smithi098.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45446.log.gz 2024-05-09T10:52:06.738 INFO:teuthology.orchestra.run.smithi098.stderr:gzip 88.4% -5 --verbose -- /var/log/ceph/ceph-client.admin.45362.log 2024-05-09T10:52:06.738 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48465.log.gz 2024-05-09T10:52:06.738 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42646.log 2024-05-09T10:52:06.739 INFO:teuthology.orchestra.run.smithi098.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48377.log.gz/var/log/ceph/ceph-client.admin.45362.log: 2024-05-09T10:52:06.739 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42638.log 2024-05-09T10:52:06.739 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45362.log.gz 2024-05-09T10:52:06.739 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46160.log 2024-05-09T10:52:06.739 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42646.log: /var/log/ceph/ceph-client.admin.42638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51821.log 2024-05-09T10:52:06.739 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46160.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42646.log.gz 2024-05-09T10:52:06.739 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42638.log.gz 2024-05-09T10:52:06.739 INFO:teuthology.orchestra.run.smithi098.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44060.log 2024-05-09T10:52:06.739 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46160.log.gz 2024-05-09T10:52:06.739 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48293.log 2024-05-09T10:52:06.739 INFO:teuthology.orchestra.run.smithi098.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51821.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47417.log 2024-05-09T10:52:06.739 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-05-09T10:52:06.740 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44060.log: /var/log/ceph/ceph-client.admin.48293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51191.log 2024-05-09T10:52:06.740 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47417.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44060.log.gz 2024-05-09T10:52:06.740 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48293.log.gz 2024-05-09T10:52:06.740 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35641.log 2024-05-09T10:52:06.740 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47417.log.gz 2024-05-09T10:52:06.740 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.51191.log: -- /var/log/ceph/ceph-client.admin.50668.log 2024-05-09T10:52:06.740 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35641.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51191.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42296.log 2024-05-09T10:52:06.740 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-05-09T10:52:06.740 INFO:teuthology.orchestra.run.smithi098.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35641.log.gz 2024-05-09T10:52:06.740 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50584.log 2024-05-09T10:52:06.740 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50668.log: /var/log/ceph/ceph-client.admin.42296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52467.log 2024-05-09T10:52:06.740 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50668.log.gz 2024-05-09T10:52:06.740 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50584.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42296.log.gz 2024-05-09T10:52:06.740 INFO:teuthology.orchestra.run.smithi098.stderr:gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50584.log.gz 2024-05-09T10:52:06.741 INFO:teuthology.orchestra.run.smithi098.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51149.log 2024-05-09T10:52:06.741 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45908.log 2024-05-09T10:52:06.741 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52467.log.gz 2024-05-09T10:52:06.741 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48251.log 2024-05-09T10:52:06.741 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51149.log.gz 2024-05-09T10:52:06.741 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42086.log 2024-05-09T10:52:06.741 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45908.log.gz 2024-05-09T10:52:06.741 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51611.log 2024-05-09T10:52:06.741 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48251.log: /var/log/ceph/ceph-client.admin.42086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48854.log 2024-05-09T10:52:06.741 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51611.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48251.log.gz 2024-05-09T10:52:06.741 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42086.log.gz 2024-05-09T10:52:06.741 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51611.log.gz 2024-05-09T10:52:06.741 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43318.log 2024-05-09T10:52:06.742 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.48854.log: -- /var/log/ceph/ceph-client.admin.43892.log 2024-05-09T10:52:06.742 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43318.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-09T10:52:06.742 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48854.log.gz 2024-05-09T10:52:06.742 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43318.log.gz 2024-05-09T10:52:06.742 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48938.log 2024-05-09T10:52:06.742 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42644.log 2024-05-09T10:52:06.742 INFO:teuthology.orchestra.run.smithi098.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43892.log.gz 2024-05-09T10:52:06.742 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49363.log 2024-05-09T10:52:06.742 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42644.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48938.log.gz 2024-05-09T10:52:06.742 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42644.log.gz 2024-05-09T10:52:06.742 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46601.log 2024-05-09T10:52:06.742 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50112.log 2024-05-09T10:52:06.742 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.0.49363.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49363.log.gz 2024-05-09T10:52:06.743 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47165.log 2024-05-09T10:52:06.743 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46601.log.gz 2024-05-09T10:52:06.743 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49916.log 2024-05-09T10:52:06.743 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47165.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50112.log.gz 2024-05-09T10:52:06.743 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47165.log.gzgzip 2024-05-09T10:52:06.743 INFO:teuthology.orchestra.run.smithi098.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41940.log 2024-05-09T10:52:06.743 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45740.log 2024-05-09T10:52:06.743 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41940.log: /var/log/ceph/ceph-client.admin.49916.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49916.log.gz 2024-05-09T10:52:06.743 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41940.log.gz 2024-05-09T10:52:06.743 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50626.log 2024-05-09T10:52:06.743 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44282.log 2024-05-09T10:52:06.743 INFO:teuthology.orchestra.run.smithi098.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45740.log.gz 2024-05-09T10:52:06.744 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45698.log 2024-05-09T10:52:06.744 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44282.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50626.log.gz 2024-05-09T10:52:06.744 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48600.log 2024-05-09T10:52:06.744 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44282.log.gz 2024-05-09T10:52:06.744 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51527.log 2024-05-09T10:52:06.744 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48600.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45698.log.gz 2024-05-09T10:52:06.744 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48600.log.gz 2024-05-09T10:52:06.744 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42128.log 2024-05-09T10:52:06.744 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51779.log 2024-05-09T10:52:06.744 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42128.log: /var/log/ceph/ceph-client.admin.51527.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51527.log.gz 2024-05-09T10:52:06.744 INFO:teuthology.orchestra.run.smithi098.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42128.log.gz 2024-05-09T10:52:06.744 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48125.log 2024-05-09T10:52:06.744 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46355.log 2024-05-09T10:52:06.745 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51779.log.gz 2024-05-09T10:52:06.745 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42002.log 2024-05-09T10:52:06.745 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48125.log.gz 2024-05-09T10:52:06.745 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43850.log 2024-05-09T10:52:06.745 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46355.log.gz/var/log/ceph/ceph-client.admin.42002.log: 2024-05-09T10:52:06.745 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47627.log 2024-05-09T10:52:06.745 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4%/var/log/ceph/ceph-client.admin.43850.log: -- replaced with /var/log/ceph/ceph-client.admin.42002.log.gz 2024-05-09T10:52:06.745 INFO:teuthology.orchestra.run.smithi098.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43850.log.gzgzip 2024-05-09T10:52:06.745 INFO:teuthology.orchestra.run.smithi098.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49766.log 2024-05-09T10:52:06.745 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49359.log 2024-05-09T10:52:06.745 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49766.log: /var/log/ceph/ceph-client.admin.47627.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47627.log.gz 2024-05-09T10:52:06.746 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49766.log.gz 2024-05-09T10:52:06.746 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41562.log 2024-05-09T10:52:06.746 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.0.49359.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47711.log 2024-05-09T10:52:06.746 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.0.49359.log.gz 2024-05-09T10:52:06.746 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41562.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.41562.log.gz 2024-05-09T10:52:06.746 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43976.log 2024-05-09T10:52:06.746 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46517.log 2024-05-09T10:52:06.746 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47711.log.gz 2024-05-09T10:52:06.746 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51947.log 2024-05-09T10:52:06.746 INFO:teuthology.orchestra.run.smithi098.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43976.log.gz 2024-05-09T10:52:06.746 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46517.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46517.log.gz 2024-05-09T10:52:06.746 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41728.log 2024-05-09T10:52:06.747 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44018.log 2024-05-09T10:52:06.747 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51947.log.gz 2024-05-09T10:52:06.747 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49172.log 2024-05-09T10:52:06.747 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44018.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41728.log.gz 2024-05-09T10:52:06.747 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44018.log.gz 2024-05-09T10:52:06.747 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51065.log 2024-05-09T10:52:06.747 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43316.log 2024-05-09T10:52:06.747 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49172.log.gz 2024-05-09T10:52:06.747 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45782.log 2024-05-09T10:52:06.747 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43316.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51065.log.gz 2024-05-09T10:52:06.747 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43316.log.gz 2024-05-09T10:52:06.747 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46313.log 2024-05-09T10:52:06.748 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52241.log 2024-05-09T10:52:06.748 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45782.log.gz 2024-05-09T10:52:06.748 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47795.log 2024-05-09T10:52:06.748 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52241.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46313.log.gz 2024-05-09T10:52:06.748 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52241.log.gz 2024-05-09T10:52:06.748 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42642.log 2024-05-09T10:52:06.748 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48516.log 2024-05-09T10:52:06.748 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47795.log.gz 2024-05-09T10:52:06.748 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50442.log 2024-05-09T10:52:06.748 INFO:teuthology.orchestra.run.smithi098.stderr: 88.6%/var/log/ceph/ceph-client.admin.48516.log: -- replaced with /var/log/ceph/ceph-client.admin.42642.log.gz 2024-05-09T10:52:06.748 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48516.log.gzgzip 2024-05-09T10:52:06.748 INFO:teuthology.orchestra.run.smithi098.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35760.log 2024-05-09T10:52:06.748 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42645.log 2024-05-09T10:52:06.749 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35760.log: /var/log/ceph/ceph-client.admin.50442.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35760.log.gz 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50442.log.gz 2024-05-09T10:52:06.749 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-05-09T10:52:06.749 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49482.log 2024-05-09T10:52:06.749 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45866.log 2024-05-09T10:52:06.749 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42645.log.gz 2024-05-09T10:52:06.749 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43178.log 2024-05-09T10:52:06.749 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45866.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49482.log.gz 2024-05-09T10:52:06.749 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45866.log.gz 2024-05-09T10:52:06.749 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45488.log 2024-05-09T10:52:06.749 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50154.log 2024-05-09T10:52:06.749 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43178.log.gz 2024-05-09T10:52:06.750 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50238.log 2024-05-09T10:52:06.750 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50154.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45488.log.gz 2024-05-09T10:52:06.750 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50154.log.gz 2024-05-09T10:52:06.750 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42982.log 2024-05-09T10:52:06.750 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41896.log 2024-05-09T10:52:06.750 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50238.log.gz 2024-05-09T10:52:06.750 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45061.log 2024-05-09T10:52:06.750 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41896.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42982.log.gz 2024-05-09T10:52:06.750 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41896.log.gz 2024-05-09T10:52:06.750 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42979.log 2024-05-09T10:52:06.750 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.2.45061.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45061.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45236.log 2024-05-09T10:52:06.750 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-05-09T10:52:06.750 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41466.log 2024-05-09T10:52:06.751 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45236.log: /var/log/ceph/ceph-client.admin.42979.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45236.log.gz 2024-05-09T10:52:06.751 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42979.log.gz 2024-05-09T10:52:06.751 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52157.log 2024-05-09T10:52:06.751 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41466.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.29302.log 2024-05-09T10:52:06.751 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41466.log.gz 2024-05-09T10:52:06.751 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51905.log 2024-05-09T10:52:06.751 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph.tmp-client.admin.29302.log: 87.9% 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.29302.log.gz 2024-05-09T10:52:06.751 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52157.log.gz 2024-05-09T10:52:06.751 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52031.log 2024-05-09T10:52:06.751 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47123.log 2024-05-09T10:52:06.751 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51905.log: /var/log/ceph/ceph-client.admin.52031.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51905.log.gz 2024-05-09T10:52:06.752 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41812.log 2024-05-09T10:52:06.752 INFO:teuthology.orchestra.run.smithi098.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52031.log.gz/var/log/ceph/ceph-client.admin.47123.log: 2024-05-09T10:52:06.752 INFO:teuthology.orchestra.run.smithi098.stderr: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42464.log 2024-05-09T10:52:06.752 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47123.log.gz 2024-05-09T10:52:06.752 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43934.log 2024-05-09T10:52:06.752 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41812.log: /var/log/ceph/ceph-client.admin.42464.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41812.log.gz 2024-05-09T10:52:06.752 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42464.log.gz 2024-05-09T10:52:06.752 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43313.log 2024-05-09T10:52:06.752 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47291.log 2024-05-09T10:52:06.752 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43934.log: /var/log/ceph/ceph-client.admin.43313.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43934.log.gz 2024-05-09T10:52:06.752 INFO:teuthology.orchestra.run.smithi098.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43313.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46034.log 2024-05-09T10:52:06.753 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-05-09T10:52:06.753 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46118.log 2024-05-09T10:52:06.753 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46034.log: /var/log/ceph/ceph-client.admin.47291.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47291.log.gz 2024-05-09T10:52:06.753 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46034.log.gz 2024-05-09T10:52:06.753 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47543.log 2024-05-09T10:52:06.753 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43724.log 2024-05-09T10:52:06.753 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46118.log: /var/log/ceph/ceph-client.admin.47543.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46118.log.gz 2024-05-09T10:52:06.753 INFO:teuthology.orchestra.run.smithi098.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47543.log.gz 2024-05-09T10:52:06.753 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51233.log 2024-05-09T10:52:06.753 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52509.log 2024-05-09T10:52:06.753 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51233.log: /var/log/ceph/ceph-client.admin.43724.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43724.log.gz 2024-05-09T10:52:06.753 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51233.log.gz 2024-05-09T10:52:06.754 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44631.log 2024-05-09T10:52:06.754 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5/var/log/ceph/ceph-client.admin.52509.log: --verbose -- /var/log/ceph/ceph-client.admin.50542.log 2024-05-09T10:52:06.754 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7%/var/log/ceph/ceph-client.0.44631.log: -- replaced with /var/log/ceph/ceph-client.admin.52509.log.gz 2024-05-09T10:52:06.754 INFO:teuthology.orchestra.run.smithi098.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44631.log.gz 2024-05-09T10:52:06.754 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42640.log 2024-05-09T10:52:06.754 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43766.log 2024-05-09T10:52:06.754 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50542.log.gz 2024-05-09T10:52:06.754 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52073.log 2024-05-09T10:52:06.754 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43766.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42640.log.gz 2024-05-09T10:52:06.754 INFO:teuthology.orchestra.run.smithi098.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43766.log.gz 2024-05-09T10:52:06.754 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49808.log 2024-05-09T10:52:06.755 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51569.log 2024-05-09T10:52:06.755 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52073.log: /var/log/ceph/ceph-client.admin.49808.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52073.log.gz 2024-05-09T10:52:06.755 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49808.log.gz 2024-05-09T10:52:06.755 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42170.log 2024-05-09T10:52:06.755 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45950.log 2024-05-09T10:52:06.755 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42170.log: /var/log/ceph/ceph-client.admin.51569.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51569.log.gz 2024-05-09T10:52:06.755 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42170.log.gz 2024-05-09T10:52:06.755 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42338.log 2024-05-09T10:52:06.755 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48812.log 2024-05-09T10:52:06.755 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45950.log: /var/log/ceph/ceph-client.admin.42338.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45950.log.gz 2024-05-09T10:52:06.755 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42338.log.gz 2024-05-09T10:52:06.755 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49022.log 2024-05-09T10:52:06.755 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50836.log 2024-05-09T10:52:06.756 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48812.log: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-09T10:52:06.756 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49022.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48812.log.gz 2024-05-09T10:52:06.756 INFO:teuthology.orchestra.run.smithi098.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49022.log.gz 2024-05-09T10:52:06.756 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47333.log 2024-05-09T10:52:06.756 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50836.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-09T10:52:06.756 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47333.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50836.log.gz 2024-05-09T10:52:06.756 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46835.log 2024-05-09T10:52:06.756 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47333.log.gz 2024-05-09T10:52:06.756 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50962.log 2024-05-09T10:52:06.756 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.46835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42978.log 2024-05-09T10:52:06.757 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50962.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46835.log.gz 2024-05-09T10:52:06.757 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48335.log 2024-05-09T10:52:06.757 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50962.log.gz 2024-05-09T10:52:06.757 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51653.log 2024-05-09T10:52:06.757 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7%/var/log/ceph/ceph-client.admin.48335.log: -- replaced with /var/log/ceph/ceph-client.admin.42978.log.gz 2024-05-09T10:52:06.757 INFO:teuthology.orchestra.run.smithi098.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48335.log.gzgzip 2024-05-09T10:52:06.757 INFO:teuthology.orchestra.run.smithi098.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50070.log 2024-05-09T10:52:06.757 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51275.log 2024-05-09T10:52:06.757 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50070.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51653.log.gz 2024-05-09T10:52:06.757 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50316.log 2024-05-09T10:52:06.758 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50070.log.gz 2024-05-09T10:52:06.758 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43808.log 2024-05-09T10:52:06.758 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50316.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51275.log.gz 2024-05-09T10:52:06.758 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44761.log 2024-05-09T10:52:06.758 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50316.log.gz 2024-05-09T10:52:06.758 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47249.log 2024-05-09T10:52:06.758 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43808.log.gz/var/log/ceph/ceph-client.admin.44761.log: 2024-05-09T10:52:06.758 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52283.log 2024-05-09T10:52:06.758 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44761.log.gz 2024-05-09T10:52:06.758 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49724.log 2024-05-09T10:52:06.758 INFO:teuthology.orchestra.run.smithi098.stderr: 87.9%/var/log/ceph/ceph-client.admin.52283.log: -- replaced with /var/log/ceph/ceph-client.admin.47249.log.gz 2024-05-09T10:52:06.759 INFO:teuthology.orchestra.run.smithi098.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52283.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46076.log 2024-05-09T10:52:06.759 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-05-09T10:52:06.759 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43640.log 2024-05-09T10:52:06.759 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46076.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49724.log.gz 2024-05-09T10:52:06.759 INFO:teuthology.orchestra.run.smithi098.stderr:gzip 88.9% -5 --verbose -- /var/log/ceph/ceph-client.admin.42643.log 2024-05-09T10:52:06.759 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46076.log.gz 2024-05-09T10:52:06.759 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45320.log 2024-05-09T10:52:06.759 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42643.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43640.log.gz 2024-05-09T10:52:06.759 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42643.log.gzgzip 2024-05-09T10:52:06.759 INFO:teuthology.orchestra.run.smithi098.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43682.log 2024-05-09T10:52:06.759 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52199.log 2024-05-09T10:52:06.759 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43682.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45320.log.gz 2024-05-09T10:52:06.759 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44626.log 2024-05-09T10:52:06.760 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43682.log.gz 2024-05-09T10:52:06.760 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50196.log 2024-05-09T10:52:06.760 INFO:teuthology.orchestra.run.smithi098.stderr: 87.5%/var/log/ceph/ceph-client.0.44626.log: -- replaced with /var/log/ceph/ceph-client.admin.52199.log.gz 2024-05-09T10:52:06.760 INFO:teuthology.orchestra.run.smithi098.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44626.log.gz 2024-05-09T10:52:06.760 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51737.log 2024-05-09T10:52:06.760 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47501.log 2024-05-09T10:52:06.760 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51737.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50196.log.gz 2024-05-09T10:52:06.760 INFO:teuthology.orchestra.run.smithi098.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51737.log.gz 2024-05-09T10:52:06.760 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43314.log 2024-05-09T10:52:06.760 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43315.log 2024-05-09T10:52:06.760 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43314.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47501.log.gz 2024-05-09T10:52:06.760 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41854.log 2024-05-09T10:52:06.760 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43314.log.gz 2024-05-09T10:52:06.760 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48728.log 2024-05-09T10:52:06.761 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43315.log.gz 2024-05-09T10:52:06.761 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41854.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41854.log.gz 2024-05-09T10:52:06.761 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48728.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48728.log.gz 2024-05-09T10:52:06.761 INFO:teuthology.orchestra.run.smithi098.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-09T10:52:06.762 INFO:teuthology.orchestra.run.smithi105.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-09T10:52:06.762 INFO:teuthology.orchestra.run.smithi105.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-09T10:52:06.762 INFO:teuthology.orchestra.run.smithi098.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-09T10:52:06.762 INFO:teuthology.orchestra.run.smithi098.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-09T10:52:06.762 INFO:teuthology.orchestra.run.smithi098.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-09T10:52:06.767 INFO:teuthology.orchestra.run.smithi105.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-09T10:52:06.769 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-05-09T10:52:06.769 INFO:teuthology.orchestra.run.smithi105.stderr:real 0m0.167s 2024-05-09T10:52:06.769 INFO:teuthology.orchestra.run.smithi105.stderr:user 0m0.681s 2024-05-09T10:52:06.769 INFO:teuthology.orchestra.run.smithi105.stderr:sys 0m0.046s 2024-05-09T10:52:06.804 INFO:teuthology.orchestra.run.smithi098.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-09T10:52:06.812 INFO:teuthology.orchestra.run.smithi098.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-09T10:52:06.814 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-05-09T10:52:06.815 INFO:teuthology.orchestra.run.smithi098.stderr:real 0m0.237s 2024-05-09T10:52:06.815 INFO:teuthology.orchestra.run.smithi098.stderr:user 0m0.968s 2024-05-09T10:52:06.815 INFO:teuthology.orchestra.run.smithi098.stderr:sys 0m0.182s 2024-05-09T10:52:06.815 INFO:tasks.ceph:Archiving logs... 2024-05-09T10:52:06.815 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-05-09_10:23:18-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7699683/remote/smithi098/log 2024-05-09T10:52:06.816 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T10:52:07.181 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-05-09_10:23:18-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7699683/remote/smithi105/log 2024-05-09T10:52:07.181 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T10:52:07.307 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-09T10:52:07.327 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-09T10:52:07.327 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T10:52:07.330 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T10:52:07.395 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-09T10:52:07.404 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-09T10:52:07.404 DEBUG:teuthology.orchestra.run.smithi098:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T10:52:07.407 DEBUG:teuthology.orchestra.run.smithi105:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T10:52:07.419 INFO:teuthology.orchestra.run.smithi098.stderr:bash: line 1: ntpq: command not found 2024-05-09T10:52:07.421 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-05-09T10:52:07.423 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T10:52:07.423 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2024-05-09T10:52:07.423 INFO:teuthology.orchestra.run.smithi098.stdout:^* hv01.front.sepia.ceph.com 3 6 377 9 +710us[ +710us] +/- 55ms 2024-05-09T10:52:07.423 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 9 -1183us[-1183us] +/- 64ms 2024-05-09T10:52:07.423 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 +450us[ +450us] +/- 81ms 2024-05-09T10:52:07.424 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T10:52:07.425 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T10:52:07.425 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-05-09T10:52:07.425 INFO:teuthology.orchestra.run.smithi105.stdout:^* hv01.front.sepia.ceph.com 3 6 377 9 +715us[ +720us] +/- 55ms 2024-05-09T10:52:07.425 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 10 -1176us[-1171us] +/- 64ms 2024-05-09T10:52:07.425 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 +461us[ +461us] +/- 81ms 2024-05-09T10:52:07.426 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T10:52:07.426 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-09T10:52:07.436 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-09T10:52:07.436 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-09T10:52:07.444 DEBUG:teuthology.orchestra.run.smithi098:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T10:52:07.530 DEBUG:teuthology.orchestra.run.smithi105:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T10:52:07.591 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-09T10:52:07.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T10:52:07.619 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-09T10:52:07.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T10:52:07.646 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-09T10:52:07.656 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-09T10:52:07.713 INFO:teuthology.task.internal:Duration was 511.587617 seconds 2024-05-09T10:52:07.713 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-09T10:52:07.722 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-09T10:52:07.722 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T10:52:07.724 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T10:52:07.760 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T10:52:07.762 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T10:52:08.073 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-09T10:52:08.073 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi098.front.sepia.ceph.com 2024-05-09T10:52:08.074 DEBUG:teuthology.orchestra.run.smithi098:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T10:52:08.094 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi105.front.sepia.ceph.com 2024-05-09T10:52:08.094 DEBUG:teuthology.orchestra.run.smithi105:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T10:52:08.136 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-09T10:52:08.137 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T10:52:08.139 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T10:52:08.203 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-09T10:52:08.203 DEBUG:teuthology.orchestra.run.smithi098:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T10:52:08.205 DEBUG:teuthology.orchestra.run.smithi105:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T10:52:08.340 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-09T10:52:08.350 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-09T10:52:08.350 DEBUG:teuthology.orchestra.run.smithi098:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T10:52:08.375 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T10:52:08.400 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-09T10:52:08.410 DEBUG:teuthology.orchestra.run.smithi098:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T10:52:08.418 DEBUG:teuthology.orchestra.run.smithi105:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T10:52:08.439 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = core 2024-05-09T10:52:08.463 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = core 2024-05-09T10:52:08.476 DEBUG:teuthology.orchestra.run.smithi098:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T10:52:08.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T10:52:08.507 DEBUG:teuthology.orchestra.run.smithi105:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T10:52:08.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T10:52:08.531 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-09T10:52:08.540 INFO:teuthology.task.internal:Transferring archived files... 2024-05-09T10:52:08.541 DEBUG:teuthology.misc:Transferring archived files from smithi098:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-05-09_10:23:18-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7699683/remote/smithi098 2024-05-09T10:52:08.541 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T10:52:08.585 DEBUG:teuthology.misc:Transferring archived files from smithi105:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-05-09_10:23:18-fs:volumes-wip-neesingh-testing1-20240507-distro-default-smithi/7699683/remote/smithi105 2024-05-09T10:52:08.586 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T10:52:08.619 INFO:teuthology.task.internal:Removing archive directory... 2024-05-09T10:52:08.620 DEBUG:teuthology.orchestra.run.smithi098:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T10:52:08.622 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T10:52:08.669 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-09T10:52:08.683 INFO:teuthology.task.internal:Not uploading archives. 2024-05-09T10:52:08.683 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-09T10:52:08.694 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-09T10:52:08.694 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T10:52:08.696 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T10:52:08.709 INFO:teuthology.orchestra.run.smithi098.stdout: 265585 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 10:52 /home/ubuntu/cephtest 2024-05-09T10:52:08.725 INFO:teuthology.orchestra.run.smithi105.stdout: 265307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 10:52 /home/ubuntu/cephtest 2024-05-09T10:52:08.726 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-09T10:52:08.737 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-09T10:52:08.763 INFO:teuthology.run:Summary data: description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}} duration: 511.58761739730835 flavor: default owner: scheduled_neesingh@teuthology success: true 2024-05-09T10:52:08.763 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T10:52:08.850 INFO:teuthology.run:pass